Olimpia Mosteanu writes about the research approach the team has taken to evaluate We Walworth, reflecting on what we have learned since the beginning of the project, and the challenges of evaluating place-based systems approaches.
1. The beginning of the research journey
We Walworth is a place-based project that recognises that systemic change is needed to develop more responsive and integrated central-local collaborations. Through these, the project hopes to better support the capacity of existing local relationships and resources. Food was key to creating solidarity in Walworth during the COVID-19 pandemic. By drawing on those experiences, the project aims to bring people together to tackle food insecurity, boosting community resilience and strengthening social infrastructure.
At the beginning of the project, a theory of change was developed to understand the place-based ecosystem of resources, relationships, and people with a stake in this work, including central government representatives, local authorities and people living in Walworth. Three key questions tied together the outcomes of the theory of change we developed for We Walworth:
The evaluation research is allowing the team to track the key project outcomes, including changes in relationships and competencies across different stakeholders, and changes in the density of engagement across the neighbourhood. In turn, these insights are guiding how we approach the project’s problem statements:
Like other place-based projects (or complex interventions) that take on complex issues, We Walworth has used an agile and iterative approach to achieve its goals. As the project has shifted, the research has flexed around it. This way of working together has had important implications for the research journey and resulted in a robust yet flexible research design. Others have called this approach “slow cooked evaluation” and the label resonates with my experiences on this evaluation research.
2. Embracing a flexible, open research design
For this project, the research team have built on our experiences with open research designs, which were key to our work during the COVID-19 pandemic. We adopted a flexible research design that helped us (1) develop more responsive ways of managing fieldwork and (2) continuously build rapport with those involved in the project. This flexibility has allowed us to respond quickly to the emerging challenges of an agile place-based systems change project. A key responsibility for the research team has been making sure the research does not disrupt engagement activities. The challenge for this role has been remaining embedded in project activities, while also standing back and examining the stories we collected from all those involved in the project. Key to this work is to incorporate all perspectives and to make sure everyone is part of the learning journey.
The evaluation research has three key moments: (a) baseline research, which took place in the first months of the project, (b) case study research that is unfolding at the moment and it is focused on the activities of the working group, and (c) end-of-project research to take place as the project wraps up in early Spring. An important part of this work is to map local resources and networks. This method is allowing us to track existing relationships in the neighbourhood and understand how community services, open spaces, amenities, among other social infrastructure providers, are used and by whom. Social network analysis is used to reveal the local ecosystem of relationships and assets. In turn, this helps us discover missing (or underexplored) links and where possibilities for action lie in the ecosystem.
The baseline and end-of-project research will provide insight into the project journey. The evaluation research combines a focus on easily comparable quantitative indicators with qualitative approaches focused on capturing progress stories.
3. What we have learned so far
This blog post is accompanied by a slide deck that presents in more detail the results of the baseline research. Here, I focus on the emerging answers to two of the research questions.
Questions 1: How do the engagement activities support or hinder the development of new models of collaborations between local government, central government, and the local community?
The baseline research has given us detailed insight into the main barriers that central and local government face to improving social capital in Walworth. The key barriers cited by local and central government participants in the research are lack of resources (funding, staff, investment in clear measurements about social capital), lack of “on the ground” knowledge and distance from community issues, communication (with the local community, across gov sectors), and competing priorities.
Moving from barriers to what it would take to achieve the goals of We Walworth, building partnerships across government and with communities and developing new ways of working were common threads in central and local government representatives’ narratives about how success would look like at the end of the project. The quote below illustrates the key themes that emerged when we asked them what would be gained by forming partnerships across government and with communities in Walworth.
“Everything! The system is ripe for reform and could be improved upon. Lots of duplication centrally that leads to frustration locally - duplication across outcomes, duplication across funding, and general duplication of resources. Bits of Whitehall looking at an area like Walworth with a telescope, community looking back and seeing a kaleidoscope.” (Interview with central government participant)
Progress stories from the residents involved in We Walworth showed us that they started the journey with different skills, expectations, and understandings of the project. They highlighted that activities joined so far have been both a learning experience and an opportunity to meet new people. We also learned that residents need to be reassured that their views and experiences matter. Transparency, clear communication, sharing the information gathered through consultations are key to increasing local participation. In the words of one of the residents involved in the project, this is what would make residents become more involved to shape the local area:
“Holding sessions where people are invited to participate, communicate, and be part of decision making. Rather than having a consultation - which in my experience, they do only once everything has already been put together. We’re asked to respond, but we never know how it worked out or what the results were. I remember once I was at a consultation about changing the bidding process for council accommodation - but I don’t think it changed anything." (Interview with resident involved in the project)
Question 2: How do the engagement activities support or hinder the development of social capital in the local area?
The baseline research explored what government representatives personally hoped to achieve by getting involved in We Walworth through progress stories. The central government representatives we interviewed spoke about “a better understanding of community needs", "[being] closer to communities, [being] able to understand what’s going on, and [engaging] with people and [feeding] the learning into conversations with policymakers."
Among the local and central government representatives we interviewed, there is shared awareness that central government work is detached from local communities and We Walworth could help them better understand inner-city diverse neighbourhoods like Walworth. The quotes below neatly illustrates this key theme:
“Increasing the trust would be really good and breaking people’s stereotypes about people that work in government. I try to break the mould - I like when people don’t believe I work for government! Breaking the stereotypes and increasing trust would be amazing, although it’ll be difficult. […] It’s not like people should trust government for no reason, we need to build it. […] It’s about being open with things; people shouldn’t trust for no reason.” (Interview with local government participant)
“I think a better understanding of the needs of those communities, and place-based aspects. […] Also it would be useful for communities to learn about how government and civil servants work, the things they have to do to make things happen.” (Interview with central government participant)
The baseline research also collected progress stories from residents who are not involved in We Walworth but might indirectly benefit from its outcomes. Those narratives showed that We Walworth engagement activities could support the development of social capital in the local area and help build stronger communities around the project.
"My own back yard and my home. There’s nothing around except Walworth Living Room. There was more a few years ago. Sure Start got taken away and so did the youth clubs, the bingo hall, the bowling - got shut down. Surrey Quays is the nearest place for that kind of thing. Your support space is your own home. The interior is strong, the exterior is disgusting.” (Interview with resident who uses the Walworth Living Room, where many of the project’s activities take place, but it is not directly involved in We Walworth)
The emerging themes from the baseline research will be compared to the accounts we will gather at the end of project. This will help us understand the impact of We Walworth on those who have been directly and indirectly involved in the project.
4. Concluding reflections on the research journey
A number of themes stood out to me while we were compiling and mapping out the journeys of those involved in We Walworth. One of them was the conversation about the barriers to improving social capital in Walworth. The central and local government representatives we interviewed noted that the lack of resources is a significant barrier. Yet, when we spoke about what they would do to build social capital in Walworth, their focus shifted from financial resources to providing a platform for community engagement and identifying community needs. I am curious to learn whether (and how) these perceptions change in coming months.
Similarly, the issue of trust stood out. There is consensus among those interviewed that rebuilding the trust of local people (an important dimension of social capital) in central and local government takes time and requires sustained engagement. At the moment, there is less clarity on how fast levels of trust can shift but, once again, there is shared agreement that behaviours and mindsets are being changed by working in partnership across government and with communities.
The work we have done so far shows that a flexible and mixed-methods research design is a better approach to evaluating an agile place-based project that tackles systemic issues. A single method or a rigid research design cannot robustly document how place-based ecosystems may shift power relationships and change ways of thinking and working in partnership. The strength of this evaluation research has come from the mix of approaches and our collective readiness to keep adjusting them. As the research journey continues, I hope this will allow us to adequately capture the complex journeys of those who have embarked on the project.
