How to establish a Useful Evaluation Function: It is a Balancing Act

By Caroline Heider, Director General of the Independent Evaluation Group at the World Bank (IEG)

Today, as I reflect on what makes for a useful evaluation function, I remain convinced of the critical importance of these four elements - utility, impartiality, credibility and outreach.

Ten years ago, Ray Rist, the then president of the International Development Evaluation Association, invited me to be a keynote speaker at IDEAS' biannual conference.* The conference's topic was evaluation capacity development. At the time, I had already dedicated over twenty years of my career to making evaluation work, which meant for me enhancing its utility by strengthening impartiality, credibility, and outreach. In addition, I had evaluated capacity development projects, which gave me insights into concepts and processes that were applicable to developing evaluation capacities as well.

Today, as I reflect on what makes for a useful evaluation function, I remain convinced of the critical importance of these four elements - utility, impartiality, credibility and outreach.

A useful evaluation function also requires multiple layers—from enabling environment to institutionalized systems and individual aptitudes and skills—and involves an iterative process with a long-term vision, adaptable tactics, and continuous course-corrections.

Enabling Environment

The concept of an enabling environment dates to the origins of the Independent Evaluation Group in July 1973. In the early days, a mandate (or policy, as it is often called in other institutions) was necessary to specify roles and responsibilities. Already 45 years ago, the World Bank had understood the importance of an independent voice with authority to speak truth to power, assess success and failure, and recommend ways for improvements.

Today, I would argue that such policy-backing continues to be necessary—evaluation offices typically come under pressure when reporting evidence that contradicts the norms or existing strategic directions but it also needs to go hand-in-hand with a culture that embraces learning and accountability.

In such a culture, each of the parties needs to contribute its share:

Institutionalized Systems

Independence has often been interpreted as the evaluators' freedom to do whatever they want or as means to/reason for isolation. By contrast, in my practice independence is a means to impartiality. By removing influence—through decision-powers, budget, or human resource decisions—and combined with the commitment to highest possible standards, evaluation can aspire to impartiality.**

Why is that important? Because an unbiased view provides balanced evidence on the effects of policy-choices or ways of program implementation; information that can help decision-makers and operational teams make better informed choices.

Such aspiration cannot be achieved just by personal commitment, but also requires establishing systems:

Individual Aptitudes and Skills

The job of an evaluator is hard! One has to be analytically meaningful, technically sound, humanly engaging, a strategic thinker, and an excellent communicator. Or, at least that would be the super-hero model...

Among the many advantages of IEG, we have been able to put together an academy that combines regular courses of the World Bank Group that are relevant to our work with specific evaluation training, communication workshops and others to offer continuous learning opportunities to our staff. This is the exception and not the norm. But, any evaluation function should invest in its people to perform at their best.

Less the responsibility of the evaluation function, but equally important, are decision-makers and program managers who are well versed in understanding evaluation—educated users, who recognize the value of evaluation and know how to use findings, lessons, and recommendations for better results and performance.

How then do these building blocks—enabling environment, institutionalized systems, and individual skills—come together? ... it's an iterative process

Developing a function that meets these requirements is not easy. Above all, it is not something one can plan with a blueprint and implement. Rather, it takes a process of continuous diagnostics—testing and implementation—assessment—continuous learning and adjustment as well as jumping on opportunities when they areise.

*Following my presentation, Ray Rist also invited me to contribute to IEG's publication Influencing Change.

**I have no illusions about human imperfection, including subjectivity, and that ultimate impartiality is unattainable. But, I also believe that as evaluator we need to hold ourselves to such standard.

Consult the original article