Challenges and benefits of using participatory research methods in evaluation

Not so long ago I finished a research project that incorporated a participatory approach. This ranged from designing the evaluation over the data collection phase to reporting the study. This means, that we applied the fundamental principle of involvement and empowerment of people being researched. In this post, I will share with you some points to consider during designing participatory research, what benefits and challenges you can expect. 

An ‘orientation to the inquiry’
The first step in participatory design is to always think through the purpose of involving participants and who they might be. This is something that has to be clear from the beginning and be aligned with the evaluation design. In this certain case, our research aimed to unveil the reasons behind the deficiencies of a program within an organization. It was a formative evaluation and the client expected us to provide pieces of evidence for development. The whole process started with a secondary analysis of some former results related to the issue and context analysis.

We had a definite timeline with a limited number of components that needed to be assessed, however, the research problem and the objectives of the program were not clearly defined.  So the complexity of the evaluation was given from this unstructured nature of the research topic and also from its embeddedness into other processes of the organization. These aspects justified the application of participative methods already from the designing stage in order to get access to the knowledge of the participants and also to win their engagement. In this case, we decided not to involve a big group of people in one single method but appointed a couple of key informants to participate in the entire process. If the focus is not on organizational development but on the social impacts of a program, different ways of involvement might be also reasonable. The research goals and feasibility of participative methods are always points of departure.

Gaining different perspectives
The second step was to define how the participants could join in the research. The inception phase can already provide many opportunities for participants’ engagement. A document review and secondary analysis of data can serve as catalysts throughout the logic model process: The synthesis of data from multiple sources can help to understand the cause-effect relationships among variables and contextual factors better. However, key informants can enrich logic modeling with valuable insights and hidden parameters, that otherwise could not be obtained.

Gaining access to knowledge
Similarly, we conducted stakeholder analysis also in a participative manner, which means, that the key informants not only validated the results but performed each analytical step. In addition, we incorporated the participative research method to capture KPIs to measure performance. Here the top management level had to be also involved in order to validate the content of the indicators and the way how we use them for evaluation. We let our key informants partake in designing the details of the qualitative phase too, thus they finalized the research questions and approved the proposed qualitative methods we used to gain answers for these questions. This solution can have many benefits since members of an organization have more knowledge about the organizational culture, the social network, and the capacities within the company.

Stepping back
Third, it is always an issue what level of control we hand over from the researcher to the participants in order to maximize the effectiveness of the approach. This might seem a minor detail, but in practice, it can be strategically significant and can influence the outcome. Maybe it is because of my personality, but as a researcher, for me, it is always challenging to play the role of the facilitator properly and give room for the participants to prevail. Stepping back cognitively from the familiar routines of performing analytics and remaining more passive in terms of questioning, handling interactions, and power relations is easy only at the first sight. Empowering the participants, however, can have obvious benefits: this can allow us to rethink established interpretations of given situations and strategies and also to discover new elements easier.

Giving safe space
In this case, ethical considerations were also important. Although key informants provided valuable information during the inception phase, and they participated in the planning and management process of the qualitative research phase as well, it had major importance to secure anonymity for the focus group discussions and in-depth interviews. We call it the concept of ‘safe space’ that can support the willingness on the part of participants to disclose their personal views of the situation, their own opinions, and experiences. Since we researched an organization, acting as a third party, being an independent consultancy firm had opened the door for the employees to share untold stories and confusing details that would not have come to surface anyway. So we determined a high level of involvement and control given for the key informants during the inception phase but a low level in the analytical phase that was limited only to the sample selection and organizing the focus group discussions/ in-depth interviews within the organization.


Conclusions
Conducting a participative research session- let it be logic modeling, stakeholder analysis, or any other method like system mapping, action research, world café, scenario planning collaborative autoethnography, just to mention some- has many challenges, like deciding about the research design, defining the participants, doing facilitation properly or handling ethical issues. However, these techniques promise unique values, like giving access to the organizational/group knowledge, providing a different perspective and engaging the participants toward a program/project/issue, just to list the most significant ones. The good news is, that practice makes perfect, the more experience a researcher has in this field the better outcome she or he can achieve. I also keep learning with every session I have, and I do it consciously. I pay special attention to make a mental note about the lessons I learnt and use every occasion to develop professionally.

Recommended readings on this topic:
Bergold, J., & Thomas, S. (2012). Participatory research methods: A methodological approach in motion. Historical Social Research/Historische Sozialforschung, 191-222.
Cornwall, A., & Jewkes, R. (1995). What is participatory research? Social science & medicine, 41(12), 1667-1676.