Guest Post: How can evaluation step up to the transformational challenge?

This guest post by Sam Buckton gives us a brief insight into what I consider to be a pioneering paper recently published as a result of his PhD work. It provides an excellent overview, based on a large number of carefully selected papers, of how evaluation-focused transformation is unfolding; this can guide all who are interested in how evaluation can support efforts to accelerate the transformations the world needs. More importantly, the paper provides us with a set of principles that we can use to shape evaluation practice if we want to become more relevant for the world of today and tomorrow. I am pleased to be a co-author of this important contribution to our field. Enjoy this short read by Sam, including - for the first time - a haiku of evaluation principles!

"Evaluating transformation means transforming evaluation."

Michael Quinn Patton's words underscore the transformational challenge for evaluation in our polycrisis world. The old world is dying, along with its associated evaluation models and criteria. If evaluation is going to truly drive efforts to transform societies towards regenerative futures where life can thrive, its practice will need to fundamentally change.

Evaluation is both mundane and profound. In a sense, we are all evaluators: evaluative thinking, both conscious and unconscious, underpins every action we take in the world. Evaluation is how we assign quality, value and importance to something. When we choose what to buy, where to live, which candidate to vote for, and whom to make our partner, when we create, design, and plan, when we display our desires, preferences and priorities, we are evaluating. Our evaluative choices are shaped by our values, morals, experiences, beliefs, paradigms and worldviews. The same goes for the relatively systematic, in-depth forms of analysis that characterise the professionalised discipline of evaluation. But the paradigms shaping the traditional Western model of evaluation, that has colonised much of the world particularly since World War II, were not designed for contexts of transformation in complex, contested social-ecological systems.

In today's world, people are increasingly wanting to know the answer to questions like: Is our system transforming towards a desirable future? What are the impacts (both desirable and undesirable) of our systems' transformations? Are we significantly contributing to transformation? Do we have the enabling conditions in place to do so? To what extent are we aligned to transformative or regenerative principles?

The promise of evaluation is to help answer these questions, and more. Many are now grappling with these questions and the corresponding need for new evaluation paradigms. This is often a case of learning from non-Western perspectives that have carried useful ways of thinking and doing for thousands of years.

Recently, I led a team of evaluators and researchers in a large review of disparate evaluation literature (over 2300 documents) to gain a holistic appreciation of how evaluation can most effectively assist transformation efforts. From 138 documents accepted after screening, we drew out lessons for evaluation that were synthesised into twelve principles for transformation-focused evaluation, along with relevant evaluation approaches, methods, frameworks and tools for operationalising the principles.

Principles can be powerful anchors to inform practice in a complex and uncertain world: they provide enough clarity to guide, but enough flexibility to adapt to different and dynamic contexts.

The twelve principles were clustered into three overarching themes of Complexity Principles (how evaluation approaches complex systems), Power Principles (evaluators' power relations), and Purpose Principles (evaluation's deeper purpose and values).

Overall, they paint a picture of evaluation as a more activist, ethically guided practice that embodies the kinds of transformations it seeks to support in the world, and sees itself as part of change efforts rather than removed from them.

The principles are not straightforward, and rightly so. The complexity of transformations demands complex evaluation. The principles are interdependent, and in some cases in tension with each other. I would encourage readers to delve into the paper for more detail, and examine the separate visual summary. But for the purposes of this blog, I've attempted to summarise the principles in a succinct way.

I find the Japanese haiku form (five, seven and five syllables) helpful for distilling words down to their essence, and each principle is therefore crystallised into a single haiku. They are far from refined but I hope that they carry the key messages of the principles.

They are colour-coded according to the three broad categories of principle: green for the Complexity Principles, purple for the Power Principles, and blue for the Purpose Principles.

Ultimately we would like to see these principles become more embedded in evaluation practice, and subjects of principles-focused evaluation in themselves. This will require the development of new competencies and expectations around evaluation, and the overturning of many misconceptions.

It will not be easy, and the principles will be controversial to many. But if we change the way we evaluate - that most fundamental human activity - then a whole new world of possibility emerges.

Twelve principles for transformation-focused evaluation

1

The world is nested,

tangled, interdependent.

See complexity.

2

Adapt to context:

every transformation

has unique features.

3

Be a detective:

use diverse tools, perspectives

on uncertainty.

4

Change is dynamic,

emergent. Remain wary,

agile, adaptive.

5

Look to the future -

the possible, desired -

with a long-term view.

6

Seek justice: uphold

rights, decolonise, honour

the unheard voices.

7

Guide rather than lead:

shift power to the users.

You're one of the team.

8

Beware the '-washers':

stand by your ethical stance.

Speak truth to power.

9

Foster partnerships

across boundaries. Efforts

should align for change.

10

Put the value back

into evaluation:

surface and question.

11

Be accountable

for learning, not pre-planned change.

Fail and learn. Learn deep.

12

Seek transformation:

regenerative futures

where life can flourish.