The Realm of Possibility in Evaluation

By: Mariah Brothe Gantz

Recently I was chatting at a meeting about what I do in my work. When I said I did evaluation, the person responded with: “What is evaluation? Like accountability?” The incredulous tone was there despite a good effort to hide it. 

“What is evaluation?” is not a shocking question, and I am sure most evaluators field this question often. But perhaps some of you are surprised at my colleague’s quick summation of evaluation as simply an accountability mechanism, something to keep you in line and something that might show you failed even without meaning to. I wasn’t. This is a repeated experience for me as I explain what evaluation is and how the Center for Evaluation Innovation (CEI) thinks about it (an easier feat now that I have been in my role at CEI for a year). 

Nothing in my education prior to joining CEI in 2018 taught me that evaluation was for anything more than accountability. 

In graduate school, I took the course “Program Evaluation” and there wasn’t much time for discussion of evaluation as anything more than an approach for determining if a defined program did what it said it would do. I completed two program evaluations in my graduate school curriculum, which hardly scratched the surface of what evaluation is and could be. I should probably ask my CEI colleagues what weird things I said about evaluation during my job interview. Now that I am at CEI and learning from them and from our wide network of colleagues, I realize there is a disconnect in how most people think about what evaluation is for (just accountability) and what evaluation could be (a mechanism that can help to advance social justice and equity). 

And why wouldn’t people think this way? Even though the nonprofit sector, and philanthropy in particular, recognizes that much of social change work is not linear and programmatic but instead complex and often unpredictable, and even though other evaluation approaches that better fit this way of thinking about change are being discussed and encouraged (e.g., equitable evaluationdevelopmental evaluation), many funders are stuck in the habit loop of making sense of grantees’ work through strict accountability mechanisms. They ask for detailed plans, linear representations of change, fixed definitions of outcomes and metrics, and then reward grantees for sticking to their plan, even if circumstances change and that plan no longer makes sense. 

I now spend time correcting people’s misperceptions about evaluation because I want them to understand the possibility of evaluation. 

CEI has recently engaged in the process of creating and emboldening our practice with values. We debated the value of “possibility” for a long time because it meant something different to each of us. In the end, we landed on this value statement about possibility: “While we are clear eyed about the challenges we face, we are committed to finding a way forward together through creativity and innovation.” [If you want to hear more about our values-driven thinking, read this co-written piece by Clare Nolan, Meg Long, Jara Dean-Coffey, and Julia Coffman.]

What I love most about the value of possibility, is that it offers a different future; it gives us distinct hope. 

It can be difficult to regard evaluation as “fun” or “hopeful” or “exciting” because in these post-normal times [read about post-normal times and evaluation in this presentation Rethinking the Moral Compass for Evaluation by Thomas Schwandt], those descriptors feel too celebratory. We are yearning for positive, timely change. We want more progress on social justice and equity. 

A focus on evaluation just to enforce accountability tells us to approach change in the way that it’s always been done, with some tweaks here and there. What about the “what-the-heck-let’s-try-it” approach? Why not?

Many, including us at CEI, are frustrated that evaluation isn’t doing what we think it should be able to do. Evaluation can help us to take bolder steps, achieve transformative change, and advance equity by helping us to question our assumptions, test and learn about new ideas, and examine how existing systems and structures keep us stuck in place. Evaluation can help us to explore new possibility instead of keep us rooted in place. 

Evaluation as a tool of possibility may seem obvious to some. However, I have been taking a “blame it on my youth” approach. [I consider this sound practice because I am frequently reminded by my fellow evaluators about how young I am. Why not take advantage of the existing branding?] 

I would like to “blame it on my youth” and question the way things are, boldly calling people on their lack of patience and creativity, and asking “Why not change it?” I want to challenge my fellow evaluators to renew their energy and spirit in evaluation by regarding its possibility

It is possible to get more from evaluation, and it starts by challenging our routines and habits and why we do them. 

Pretend that you, like me, are just entering the evaluation field and are not yet tired by the same arguments, the same rules, the same tools, and the same excuses. Now, ask, “Why do we have these tired habits? What function do they serve?” 

Critically examine your routines and habits and Think Like a Pterodactyl! [A replacement statement to “Think Outside the Box” offered by Vu Le in Nonprofit AF]. Break the cycle of those habits by challenging them. Sarah Stachowiak of ORS Impact models how to do this as she challenges her traditional views of expertise and nicely summarizes how to break this habit in her piece: Unlearning: Being an Expert

Lean into the possibility of evaluation. If you have access to new evaluators, encourage their questions and listen to your own answers. 

Get new evaluators into the room and listen to their craziest ideas and allow them to question big decisions. You could probably use renewed energy, vigor, and new ideas. Not all our thoughts as new evaluators will be diamonds, but you might still catch a gem if you listen hard enough. 

Mariah Brothe Gantz is an associate at the Center for Evaluation Innovation. Before joining CEI, she worked with the Leadership Conference Education Fund as the lead data analyst on the Communities Against Hate Initiative. Mariah has a graduate degree in public administration from the George Washington University with an emphasis in nonprofit management, and an undergraduate degree in cultural anthropology from the University of Northern Colorado.

Related Resources