As an independent researcher and evaluator, I recently attended the European Evaluation Society's 15th Biennial Conference in Rimini, Italy. The theme, "Better Together: Collaborative thought and action for better evaluation," set the stage for rich discussions and critical reflections. Here are my key reflections:
1. Decolonizing Evaluation in Action
❓The Issue: "Decolonizing evaluation" is a frequent topic of discussion, but practical guidance on how to actually walk the talk and implement this approach remains scarce.
💡 The Opportunity: A standout presentation by Minji Cho of Claremont Graduate University demonstrated how to integrate a decolonizing evaluation framework into all aspects of a project—from budgeting and team selection to funder relationships. This was a clear example of how principles (Responsivity, Deconstruction, Holistic and Relational, Reconciliation, Self Determination, and Learning) can guide practical implementation.
🚩 Implication for Evaluators: Decolonizing evaluation in practice means stepping out of our comfort zones. It requires not only rethinking practices but also engaging in extra work—educating others, negotiating differences, and addressing complex questions like fair and sustainable fee structures, especially in international projects that span diverse economic contexts.2.
Climate Change & Evaluation: No Longer Optional
❓The Issue: Climate change is no longer an optional consideration for evaluators. It has evolved from a niche concern to something that needs to be integrated into our evaluation plans, frameworks, and indicators.
💡The Opportunity: At the EES conference, a professional development workshop on climate change evaluation encouraged us to reflect on the role of climate in our work. We made a commitment to think about how the workshop material relates to our work on the ground. I will share my thoughts after I finish reading a big pile of papers.
🚩 Implication for Evaluators: We need to learn about the conceptual frameworks, landmark policies, and unique evaluation challenges when it comes to evaluating climate change interventions. We also need to learn how to bring a ‘climate change’ lens to all stages of our work, just like we have been doing with ‘gender analysis’ or ‘human rights analysis’.
3. We Need to Talk About AI But (For Now) Mostly LLMs
❓The Issue: Evaluators within large organizations like UNICEF, DEval, and the World Bank IEG are at the forefront of experimentation with AI (e.g. how well do certain LLMs code data compared to others), finding use cases (e.g. creating counterfactuals for impact evaluations), and establishing standards (e.g. A presentation by German Institute for Development Evaluation showcased five standards and discussed the risks and mitigation strategies associated with each). These large organizations are working with data scientists and learning how to create and play the ‘broker’ role. But smaller evaluation firms risk being left behind due to resource constraints.
💡The Opportunity: The openness of these larger agencies to share their tools and lessons, is encouraging. We talked about pushing for more ‘public goods’ that can come out of such experimentation with AI. The conversation will need to continue.
🚩 Implication for Evaluators: Contrary to some perspectives at the conference, I am more convinced now that evaluators must upskill. Not all of us need to become data scientists, but we need to know enough to work with them. Whether through learning basic Python, understanding data science pipelines, or exploring prompt engineering, we must upskill if we want to shape the field.
4. Who Wasn't in the Room?
❓The Issue: Despite discussions on transformation and decolonization, representation from Low- and Middle-Income Countries (LMICs) in conferences remains a concern for me. Financial barriers and visa refusals continue to limit participation from these regions. While bursaries and funding guides (like the one EES put together) can help some participants, visa approvals remain largely political and outside direct control of conference organizers.
💡The Opportunity: People like Madhukar Pai, MD, PhD, FCAHS, Canada Research Chair in Epidemiology & Global Health at McGill University, and other global health professionals are raising awareness of ‘visa injustices’. Is there an opportunity for us evaluators to do the same?
🚩The Implication for Evaluators: If we don’t pay attention to this absence, our colleagues facing visa barriers might just stop wanting to participate in these meetings all together. Their absence means our discussions are less rich and relevant. This PLOS article provides good practical recommendations for short-term, medium-term and long-term, ranging from logistics of conference planning to advocacy for brining this issue into the spotlight.
After posting this blog on LinkedIn, one of the readers shared this report by Global Change Center, Praxis UK, and Praxis Institute for Participatory Practices, titled: Equity-centered evaluation of international cooperation efforts: the urgent need to shift unfair power dynamics. It summerizes the findings from the study “Landscape analysis to inform international evaluation in the service of equity,” commissioned by the Ford Foundation.
If you’d like to explore these topics further or have questions, feel free to contact me. I’d be happy to provide additional context or connect you with the relevant experts.
Comentarios