Each year, promising new approaches to global problems fail. Better evaluations could tell us why.
“It is perhaps no coincidence that the development community’s biggest successes--eradication initiatives like polio and smallpox--are precisely the ones that made monitoring central to their work,” wrote Ned Breslin, CEO of Water for People, in the Stanford Social Innovation Review’s recent series on impact measurement and monitoring.
According to Innovation Network’s October 2012 State of Evaluation study, fewer than half of nonprofits use measurements to adjust their programs at least annually, a figure that actually fell in the last year. Yet many promising new approaches to tough problems continue to fail in ways that surprise their creators, funders, and constituents.
But good measurements can tell us more concretely and specifically which factors lead some projects to fail and some to succeed. Take microcredit as an example:
“An evaluation of a microcredit program might investigate, ‘Does access to microcredit alleviate poverty?’ But designing a rigorous study to answer this question requires researchers to probe into deeper sub-questions: ‘For whom does microcredit alleviate poverty?’ and, ‘What kind of microcredit product?’” wrote Jenny Stefanotti, of the Hasso Plattner Institute of Design at Stanford, in the SSIR series on impact measurement
To address these questions, we need to understand specifically for whom a given project is intended and details of the intervention. In other words, we need context. We need to know the ways a microcredit intervention in Uganda will be similar to and different from one in Indonesia. For all the great data we get from randomized evaluations, they don't take into account this valuable information.
And we also need to understand our own perspective--because a development economist’s view of microcredit is different from a microfinance lender’s. If nonprofits begin to account for context in their impact measurements and monitoring efforts, they can begin to encourage the successful adoption of solutions that work within the communities they serve. Perhaps more importantly, they can model and isolate the issues that caused problems, and design solutions that can be expected to work in a different community.