11th of October is the International Day of the Girl Child, which gave an opportunity for activists, celebrities, influencers and ordinary women to talk about inequity and exclusion gap of girls this week. In apropos of this today I would like to write about equity as an approach in program evaluation.
Program evaluation and impact evaluation are basically about informing founders or policymakers about the effectiveness and performance of social programs designed to help those who needed it most. The current pillars of evaluation practice are evidence-based decision-making, rigorous scientific research methods, and validated non-biased results. However, if we focus exclusively on questions like generalizability or cost-efficiency, we might miss reflecting on the core values of the organization, like equity, equality, or social justice. Equitable evaluation addresses this problem by applying an equity approach in evaluation practices.
But why is this important, how equitable evaluation is connected to the conception of ‘culturally competent’ or ‘culturally responsive’ evaluation and what can be the role of participative research here? Today I was thinking and reading about these questions, and now I would like to share my answers with you.
Equity is a conception that has been always relevant and has a long history in the philanthropic sector, where paying attention to the positive or negative impacts of an evaluation on equity is common and required. However, there is an ongoing shift in evaluation practice that is connected not only to philanthropy but to nonprofits in general (Equitable Evaluation Initiative, 2017). In 2011 for example, the American Evaluation Association released a statement declaring cultural competence in evaluation to be “an ethical imperative” and necessary for guaranteeing the validity of findings. In 2018, AEA published its Guiding Principles for Evaluators, which also includes the concepts of cultural competence and equity in evaluation: one out of the five guiding principles is ‘Common Good and Equity: Evaluators strive to contribute to the common good and advancement of an equitable and just society’.
The equitable approach
But what is equitable evaluation more precisely? Compared to the traditional evaluation, the shift happens in the direction of being conscious of inequities. Applying an explicit equity dimension when it is about monitoring and assessment, helps us to exceed the conventional data analytics, and collect (more) information on the difficult-to-reach or socially marginalized groups (Stern, 2019). In addition, engagement in self-reflection and questioning traditional evaluation methods can lead to deeper understanding.
Equitable vs. culturally responsive evaluation
Adopting equitable evaluation principles requires that evaluators recognize that life experiences and perspectives might differ, and that linguistic, historical, and socioeconomic differences can lead to differences in perceptions and interpretations. Equitable approach means, that the evaluator is critical in connection with equity aspects of the evaluative process from asking questions to giving recommendations. This goes beyond culturally responsive evaluation because it pays attention to evaluating system drivers of inequalities. However, culturally responsive or competent evaluation is more in a sense, that it actively searches for cultural grounding, appreciates the role of sovereignty, seeks knowledge of the particular community and builds relationships (Public Policy Associates PPA, 2015). It rejects culture-free evaluation and recognizes that culturally defined values and beliefs lie at the heart of evaluation efforts. (Hood, 2015)
Equitable approach in practice
There is a growing practice of evaluation methods like participatory and empowerment evaluation that explicitly account for the power imbalances. Participative research methods are especially suitable for amplifying the voices, knowledge, expertise, capacity, and experiences of all evaluation participants and stakeholders, particularly people of marginalized groups. Participative research applies the fundamental principle of involvement and empowerment of people being researched, so these methods can be especially fruitful if it is about recognizing hidden disparities, ensuring diversity and realizing inclusion. In addition to this, the increasing use of systems thinking in evaluation helps evaluators to understand how to identify and evaluate changes related to the systemic drivers of inequity.
The steps that an evaluator can apply are similar to the culturally competent evaluation practice: we can select a diverse evaluation team where members have different skills, identities, experiences. Second, the historical and contextual background should be understood before phrasing research questions, and documenting and analyzing systemic and institutional patterns of inequity, including racism and poverty should be incorporated. Finally, stakeholders play a crucial role in being able to address equity issues, so participative methods can help in maximizing the accuracy and validity.
Conclusion
Evaluators, including the American Evaluation Association argue, that equity approach is not only an option but an essential characteristic of the high-quality evaluation (Stern, 2019). A culturally responsive evaluation recognizes that researchers, practitioners, theorists and philanthropists must challenge and address cultural biases in evaluation. Equitable evaluation applies in addition to this, equity-focus; reaches out for the marginalized groups and handles equity issues on a structural level. One effective way to do so is the application of participative methods already from the planning stage of the evaluation project.
Closing this post, I would like to draw your attention to Project Implicit. It is a non-profit organization founded by scientists at Harvard/University of Virginia/ University of Washington. It aims to educate the public about bias and disparities. Series of tests on their website help us to raise our self-awareness and be conscious of our implicit attitudes and beliefs that we did not know about, but that might be connected to an evaluation project.
References:
Hood, S., Hopson, R. K., & Kirkhart, K. E. (2015): Culturally responsive evaluation. Handbook of practical program evaluation, 281.
Equitable Evaluation Initiative, Center for Evaluation Innovation, Institute for Foundation and Donor Learning, Dorothy A Johnson Center for Philanthropy and Luminare Group (2017): Equitable Evaluation Framing Paper
Stern, A., Guckenburg, S., Persson, H., Petrosino, A., & Poirier, J. (2019): Reflections on applying principles of equitable evaluation.
Public Policy Associates (2015): Considerations for Conducting Evaluation Using a Culturally Responsive and Racial Equity