Guest post: The etcetera of Transformative Evaluation

Umberto Eco, in the opening pages to his book The Infinity of Lists provides evaluators with a ticklish challenge: stability supports the definitions of a finite form of list, while instability requires a reforming of an idea with the recognition that the list of its attributes is likely to contain an etcetera. Providing an etcetera is not the laziness of an Evaluator, but a signal that a list needs to be constituted by many hands to make sense of an emerging reality.

Grappling with the instability of our current time is represented in our evaluation literature and indeed Zenda's blogs, signaling that our lists and checklists need reformulation to guide and shift our practice. The transformative challenge is all around the evaluator. Importantly IDEAS has chosen Evaluation for Transformative Change as the theme for their 2019 Global Assembly. IDEAS did this based on the rationale that:

The goal of societal transformation brings with it a challenge for evaluators. How does the role of evaluation need to adapt for the values of the SDGs? What is the list or lists we need? Are lists the appropriate response for evaluators who work in contexts of complexity?

The shift will involve the use of a transformative lens for the evaluation design, implementation and use. Zenda Ofir, in a post in this blog space voiced this challenge as follows: "One can expect that 'transformation' is the point of departure for development planning as well as for evaluation in a development context, which now includes both rich and poor countries. Yet when I look for literature on evaluation for transformation it appears to be meagre."

The response to this challenge in the international development community has focused on systems and complexity theory (see, for example, a recent Webinar recording and Complexity Checklist), without explicitly bringing a transformation lens to evaluation planning, implementation, and use.  As a rejoinder we propose that we start to form two lists that challenge the international development evaluation community to configure transformation with an evaluative perspective. These include works by one of the authors of this post (Donna Mertens), and distinguished thinkers who have written about transformation from the perspective of feminist, culturally responsive and Indigenous evaluation.

Donna has previously proposed four common themes to transformative evaluation approaches:

• underlying assumptions that rely on ethical stances of inclusion and challenging oppressive structures;

• an entry process into the community that is designed to build trust and make goals and strategies transparent;

• dissemination of findings in ways that encourage the use of result to enhance social justice and human rights; and

• addressing intersectionality when culturally responsive, feminist, equity-focused, and Indigenous theories are relevant in the evaluation context.

Work on evaluation in climate change and sustainability challenge us, for example, a recent New Directions special issue, on adaptive management and from the climate investment fund,  to think about how evaluation relates to systemic change, accountability relationships, scale, sustainability...etcetera.

Existing resources on transformative forms of evaluation include:

The work of Donna Mertens developing the concept of transformative evaluation began in 1998 in her presidential address to the American Evaluation Association; the conference theme was Transforming Society Through Evaluation. Subsequently developing the meaning of transformative evaluation in, for example, three books: Transformative Research and Evaluation, Research and Evaluation in Education and Psychology 5th and Program Evaluation Theory and Practice 2^nd ed. Her website provides many resources for evaluators who seek to use a transformative lens

The Center for Culturally Responsive Evaluation and Assessment (CREA), with Stafford Hood and Rodney Hopson as principal faculty, provides many resources that have relevance for international evaluators. Members of CREA have published several books that provide details regarding culturally responsive methodologies, for example, Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice & Tacking Wicked Problems in Complex Ecologies: The Role of Evaluations.

Bagele Chilisa, Professor at the University of Botswana, wrote Indigenous Research Methodologies as a text to guide researchers and evaluators in work conducted in the larger, historical, cultural, and global context, with specific emphasis on historical and cultural traditions of the "third world and indigenous peoples".

The Maori community in New Zealand have written and presented extensively on the Kaupapa Maori approach to research and evaluation; an approach that prioritizes responsiveness to their culture in the historical context of colonization, for example, Decolonizing Methodologies: Research and Indigenous Peoples.

UN Women have also been at the forefront of advocating for equity-focused, gender responsive evaluation.

Feminist Issues in Evaluation are being discussed within a thematic interest group of the AEA

These two lists are incomplete and can be elaborated to help achieve the SDGs in complex contexts. We accept the challenge, and hope that others in the international development evaluation community will as well, engage in the etcetera, shifting our practices so that evaluation can be substantive in achieving transformation, beyond its instrumental support to projects.