Competencies for post-normal evaluation, Part 2

**6 min read**

Skill 6. Deep Generalists

In order to be a good evaluation professional, it is essential to see connections. We have to be able to analyse, integrate and synthesise - what we know from past research and evaluation literature, what we derive from different sources and methods used during the evaluation, and what our own experience and intuition tell us. We need to understand as much as possible about which connections matter, the interplay between them, and how this might influence the evaluation findings, conclusions and recommendations.

Studying 'what works' is too limiting. We need to understand much more about 'why' and 'under what circumstances' changes have been or are taking place. This will help us to see connections that matter.

The 2030 Agenda for Sustainable Development reinforces the importance of the many connections between agents, or processes, or goals, or outcomes. Evaluation professionals today most certainly need to operate as 'deep generalists'. We need to be specialists without being confined to our area of specialisation. We have to understand matters beyond our own narrow field, and deeply consider connections in whatever and however we evaluate.

Skill 7. Design Logic

Stowe Boyd: It's not only about imagining things we desire, but also undesirable things - cautionary tales that highlight what might happen if we carelessly introduce [for example] new technologies into society.

His main point is that those of us in the world of work also need to focus on the implications, not just on 'application' or 'implementation' of what they are doing -  implications at the design stage, during implementation, and once done. Tony Dunne and Fiona Raby's argument in Design for Debate about the cultural, social and ethical implications of emerging technologies is equally valid for social contexts: "This shift from thinking about applications to implications creates a need for new design roles, contexts and methods ... It's not only about designing products that can be consumed and used today, but also imaginary ones that might exist in years to come. And, it's not only about imagining things we desire, but also undesirable things -- cautionary tales that highlight what might happen if we carelessly introduce new technologies into society."

My evaluation reports often have a chapter or section about the strategic and/or practical implications of chosen pathways, findings or conclusions. We do not do that often enough. So much of what I see in evaluation reports are descriptive rather than analytical, and stop short of considering the implications of what we find. Social innovations bring new ideas, yes, but their implications need to be explored and methods to do this advanced. There are now swathes of books and articles that urge evaluation professionals to study especially unexpected negative consequences or outcomes of actions and interventions (i.e., externalities). More than that, we need to consider whether trade-offs are worthwhile. Yet the number of evaluation professionals who do this systematically and with some rigor remains extraordinarily low. This means that many assessments are essentially useless, and certainly not credible.

Skill 8. Post-normal Creativity

Innovation or creativity is today the foundation of business, and essential for finding solutions to major challenges. It is also essential for advancing the field of evaluation. Should it be the responsibility of evaluation professionals to understand better not only how to evaluate innovation, but how to be innovative in every evaluation where this makes sense - not just mindlessly following methodological recipes, but advancing the field of evaluation through small and large innovations, and sharing them with our peers?

Skill 9. Posterity - not History, nor the Future

In my view, this is the most important skill to add to our competency frameworks. According to Stowe, "'posterity' implies continuity of society and the obligations of those living now to future inheritors, a living commitment, while 'the future' is a distant land peopled by strangers to whom we have no ties". Instead of being selfish, it is therefore essential to identify "not only with a community in space but also with a community extending over time from the past into the future". Greta Thunberg has highlighted the importance of commitment to future generations, and to the world we will leave them.

This is part of a new ethics for this era. It also requires imagining and 'designing' futures fully cognisant of the people who will inhabit them. Evaluation professionals use the past to help inform the future. But do our evaluation questions and criteria reflect a clear notion of the responsibility of this generation to future generations, the need to break from the past - and hence the stewardship that is needed to ensure sustainable development 'for posterity'? To my knowledge only the Canadian Evaluation Society has included this explicitly in their guidelines. If this is so, we most certainly need updates of our competency sets across the global evaluation community.

Skill 10. Sensemaking

Sensemaking is a higher-level thinking skill that cannot be codified, 'the ability to determine the deeper meaning or significance of what is being expressed', creating unique insights critical to decision-making. It is in essence the combination of the skills of analysis, integration and synthesis - the actions at the core of evaluation. I often do quality assurance for evaluation offices. A significant portion of evaluation reports that I get to assess are descriptive, devoid of clear evaluative reasoning that shows analytical abilities, or the synthesis that is needed to arrive at valid evaluative findings and conclusions.

In my third and final post I will summarise my thoughts about evaluation competencies for this era.

There's a small cadre of agitators (I include myself) shouting out that the times are a-changin', but I don't know how far our voices carry, or if others can understand our words.

And next year's words await another voice.
- TS Eliot, Little Gidding

Perhaps this is proof, once again that we need new ways to think about - and talk about - this rapidly changing world: we will have to find another voice. Perhaps that is the 11^th skill.