Chris Foye blogs about the challenges of carrying out policy research in a time of crisis. He reflects on his own experiences, and the experiences of other researchers.
Chris is a Knowledge Exchange Associate at CaCHE - the the UK Collaborative Centre for Housing Evidence.
In July, Alex Marsh and I started a research project - “The rented sector, evictions and activism”- exploring how policymaking in relation to the private rented sector had responded to the COVID-19 pandemic. The first wave of interviews, from September to November, included 15 people who were involved in, or sought to influence, private rented sector policymaking in England and Scotland. The findings from it will be published in February 2021. The second and final wave will be conducted in Spring 2021.
This is part of a multi-stranded research project by the UK Collaborative Centre for Housing Evidence (CaCHE), which is exploring different dimensions of the housing policy response to COVID-19.
In this blogpost, I talk about the challenges encountered when conducting policy research in a time of crisis. I reflect both on our own experiences, but also on the experiences of those people we interviewed, many of whom conducted research themselves.
Three features of a crisis
Before doing so though, we must identify the defining features of crisis: urgency, the need for state intervention and uncertainty.
First, crises are perceived to constitute an urgent threat to a society’s basic structures or fundamental values. Second, and consequently, they are understood as requiring decisive intervention from the state.
Taken together, these two characteristics imply that crises carry with them a certain degree of political contingency and opportunity. Policy develops rapidly and also, potentially, radically. Policies which were previously considered too interventionist may suddenly be deemed tolerable or even necessary.
Such interventions, however, are made difficult by the third defining feature of crisis: uncertainty. In times of crisis, actors and agencies may simply not know what is going on, what is going to happen next, or what the effect of a policy intervention is likely to be. Factors which, in normal times, would inform policy positions are suddenly in flux.
Taken together, these three features present a number of challenges to researchers looking to study the policymaking process.
Challenges of conducting policy research in a time of crisis
The first challenge relates to timescales and the potential relevance of the research. Because policy is developing much more rapidly than usual, it also means that policy research goes out of date more swiftly than usual. This puts pressure on researchers to publish fast, to exploit the tight timescale over which a piece of research is likely to be relevant.
Publishing fast, however, raises challenges of its own. Most obviously there is a quality vs relevance trade-off: it takes time to produce a piece of research that is empirically robust, and/or theoretically compelling. Publishing fast in a time of crisis also raises potential ethical issues. Because policy is developing more rapidly and unpredictably than usual, the potential impact of a piece of research is arguably more difficult to gauge. If research subjects believe that, when published, the research is likely to shape the policy response, then this may affect their engagement .
Some civil servants have been reluctant to engage in our research, perhaps because they are aware their responses (which will be included in our interim report, due to be published in February 2021) will be scrutinised more than usual. Other actors more peripherally involved in the policymaking process have been very forthcoming: but while we might assume this is in good spirit, can we rule out completely the possibility that they are looking to use the research output as another channel through which to advance their preferred framing of the policy response?
The final challenge relates to the lack of data. One of the most interesting findings from our research is how scarce, and therefore valuable, data is during a crisis. On reflection, this is actually quite obvious. Because a crisis dramatically alters outcomes, data collected before the crisis are suddenly rendered “out of date”. Consequently, organisations that can provide “up to date” data, in the form of hastily commissioned surveys or local knowledge, are likely to be more in demand both from those making policy in the dark, and from the media looking to ascertain and communicate the current state of affairs.
Organisations which can provide this data therefore acquire a certain amount of influence both over policy and the media. For example, several organisations we spoke to mentioned how important polling evidence was in shaping narratives on the extent of rent arrears accrued in the private rented sector.
Overall then, the defining features of a crisis - change, uncertainty and contingency - have significant implications for the policy research process: they apply pressure to publish fast; they potentially raise the stakes of the research, affecting the way in which interviewees respond; and they render existing data sources out of date, meaning ‘new’ data becomes a much more valuable commodity and what counts as useful data can come to be viewed more flexibly.
Photo credit: Random Institute on Unsplash
