Resolve to evaluate in 2016

Resolve to evaluate in 2016

We’ve heard loud and clear that climate advocates understand the importance of evaluating their work and feeding what’s learned back into campaign strategies. The challenge many face is how to do this well – consistently, efficiently and in support of making measurable change.

In December, the Climate Advocacy Lab hosted an online conversation featuring three people in the vanguard of evaluating for impact. Tom Novick, Jackie Mahendra and Courtenay Brown all emphasized that one of the most important considerations, nay, rules of evaluation is that measuring our work isn’t about exposing flaws but identifying where and how we can become more effective. It’s less about whether we’ve won or lost and more about how to use evaluation for continuous improvement.

“We need an alarm clock that reminds us to pause and build in a rigorous strategic review,” said Tom Novick of M&R Strategic Services, who has evaluated major policy advocacy efforts from the Waxman-Markey bill to the affordable health care act. Tom emphasized the value of checking in on a regular basis to reflect on:

  • Is the campaign running as planned – has it achieved the goals it said it would by a specified time, deployed the strategies it intended to, reached audiences, etc.? Why or why not? What has changed?
  • Was the original strategy and assumptions it was built on based on sound thinking?
  • Is the campaign in a strong/better position to address the opportunities and challenges of the upcoming year?
  • Has the campaign been responsive to previous evaluation recommendations?

Jackie Mahendra of the Citizen Engagement Lab talked about using evaluation to move beyond tallying a campaign’s outputs or outreach. For Mahendra, who advised on the report Beyond Vanity Metrics: Toward a Better Measurement of Member Engagement, the question of impact comes down to whether organizations are measuring and growing their source(s) of power.

One of the first steps for those wanting to develop a culture of evaluation is to discuss whether current metrics align with an organization’s mission and theory of change. Mahendra offered these questions to help guide a better, impact-oriented approach to measurement:

  • What do your current metrics incentivize and how often do you discuss them?
  • Where are you still reliant on vanity metrics?
  • Can you simplify your key metrics? Visualize them?

Tactic-level measurement also offers major benefits to campaigns that want to make sure their outputs turn into outcomes. Courtenay Brown of the Climate Reality Project reminded us that using quick and inexpensive tools for getting feedback on campaign collateral – such as Prolific Academic or Amazon Mechanical Turk –  allow organization’s to test effectiveness before materials go live.

Check out the full presentation, Evaluating for Impact: Are you measuring what matters?

Photo via