Stanford Social Innovation Review : Informing and inspiring leaders of social change

SUBSCRIBE | HELP

Philanthropy

Getting Beyond Hype: Four Questions to Predict Real Impact

A simple way to think about investment and design.

Through a kind of magic I don’t fully understand, some emerging poverty solutions—products, services, technologies—become shiny new objects that trigger something akin to a feeding frenzy. Celebration at CGI and SOCAP and TED, breathless articles in Wired and Fast Company, endorsements from celebrities, awards of all kinds—all that stuff is great if the thing has real potential for impact. Too often, though, the appetizer becomes the entrée and hyperbolic celebration displaces systematic evaluation.

That’s a shame, because we really do need to sort good ideas from bad, and it ought to happen before we stoke the fires of publicity. At Mulago, we’ve spend a lot of time trying to figure out if start-ups with a new “thing”—a product, service, or technology—are likely to create real impact in the lives of the people we’re trying to serve. Over time, we’ve evolved a set of four questions that help us make better predictions. Because we’ve been thinking a lot about cookstoves and indoor pollution lately, I’ll use improved cookstoves to illustrate how those questions work.

1. Is it needed?

Given the nature of the thing, is there a profound social impact to be had? Every thing ought to have a mission (preferably in one that’s eight words or less). If you can’t figure out a clear mission that matters, all you may have is a solution in search of problem—and some things ought to be killed while they’re young.

We might reasonably ascribe various missions to better stoves, including: improve health, prevent deforestation, save time and money, and decrease carbon emissions. Since indoor air pollution is a major killer and bad stoves are the major contributor, we think that the mission to “Prevent Respiratory Disease in Poor Families” is a compelling one and rises above all the others (especially since the evidence connecting the other potential missions is considerably more sketchy). So the answer to question one is yes, absolutely, we need better stoves.

2. Does it work?

If the thing is deployed as designed, under the conditions for which it was designed, will it have real impact? In the case of stoves, this is mostly about emissions: Does a given stove reduce pollutants enough to drive a significant improvement in health? Recent research indicates that most improved cookstoves don’t. It turns out that you have to reduce more than 85-90 percent of pollutants before you see any real reduction in respiratory illness (Burnett et al, 2014). A stove that emits 50 percent less pollution doesn’t accomplish much. To date, the only stoves that make the cut are the expensive “forced-draft” models that use a built-in fan to stoke a much more efficient burn. The answer for most stoves, however admirable the effort may be, is no. Really.

3. Will people use it as designed?

Behavior, behavior, behavior. Despite all that user-centered-design, iterative-rapid-prototyping stuff, people use and abuse things in ways that leave even the best designers scratching their heads. Too many—way too many—products have made their way into wide distribution without evidence that people use them as intended. Successful things, though, usually have these qualities in common: They fit well with local customs and culture, are easy to use right and hard to use wrong, and need little maintenance (and when they do, they’re fixable and you can get parts).

Everyone has heard about stoves that people use improperly—or not at all—but field researchers have handed the stove industry another fun surprise: Even when people do use that wonderful new stove, they often use the old one too (Ruiz-Mercado et al, 2011). It’s become known as “stove-stacking,” and you see it when you visit homes—there’s the new stove, fired up and roaring, and there’s the old three-stone fire still belching smoke in the corner. Oh no! If you want to fulfill the mission—prevent respiratory disease in poor families—not only do you have to provide an expensive stove, you might have to provide two! It kind of sucks, but that may be what it takes to get to yes.

4. Will it get to those who need it most (a lot of them)?

Nothing matters if the thing doesn’t reach the people it is intended to serve. If you don’t have a reality-based idea about distribution, you should hold off on the design process until you do. Three things matter above all else:

  1. Price. If it’s too expensive, they won’t buy it. Knowing the customer’s price point is your first order of business, and you need to design the thing to hit that number.
  2. Distribution channel. Poor people live in a world of market failure, and there are a finite number of ways to get things to them—the hardest is to set up your own network.
  3. Sales. Somebody has to make the transaction happen—who are they and how are they going to do it?

And, of course, if the thing costs more than about 20 bucks, you’d better think about financing, too…

This is where stoves have always struggled: The affordable ones are inadequate, and the good ones are unaffordable. Even with the (unreliable) assistance of carbon credit subsidies, stoves that can fulfill our health mission are still too expensive. Emerging technologies will almost certainly lower the price of forced-draft stoves, but for the time being, they’re mostly out of reach for those who need them most. Financing can help, but poor people often have more urgent things to do with available credit.

One organization working on the stove problem that gets to “yes” to all four questions is Inyenyeri, a Rwandan company that leases ultra-low emission, forced-draft cookstoves to households at a nominal cost (about $7 per year), and sells customers the fuel pellets to burn in them. The pellets cost less than charcoal, and people who can’t afford to buy them can trade biomass for pellets. Inyenyeri leases families two or even three stoves to account for stacking. It’s still early days and there are a lot of moving parts, but it’s headed in the right direction.

The four questions are unforgiving; getting a no on any one of them means you won’t pass go on your way to impact. That doesn’t mean that a no answer should lead you to abandon the whole effort, but it should refocus your efforts on getting to yes. Stick with R&D, and avoid the big splash until you’re pretty sure you’re there—it’s the least we owe the people we serve.

Read more stories by Kevin Starr.

Tracker Pixel for Entry
 

COMMENTS

  • Cathy Lada's avatar

    BY Cathy Lada

    ON September 3, 2014 09:17 AM

    Kevin - excellent points, all. I agree wholeheartedly that beyond proof of concept, all of these great social innovations we’re hearing about need to pass some rigorous reality checks. I think the four you’ve enumerated above capture the essence of what I’m seeing in scholarly literature, and, more often (and thanks to people like you) in popular literature. The importance of understanding - deeply - local culture is often de-valued, and I’m so pleased to see it mentioned here. The only thing I’d add are 2 items to your 4 questions are in #4 - first, manufacturing. I’m not sure if you’re assuming the product (stoves in this case) are made abroad and brought in, or being manufactured locally (which adds jobs, etc. but adds training, CAPEX, and labor concerns, among others). When you factor in manufacturing, there are also supplier (esp. availability and quality of components) concerns. The second thing I’d add to #4 is maintenance and repair. How will replacement parts be fabricated and/or made available? Who is qualified to repair the item?

  • Debbie Hall's avatar

    BY Debbie Hall

    ON September 5, 2014 10:32 AM

    Excellent story, Kevin.  I’ve seen numerous examples of ‘improved cookstoves’ and have always wondered if they are improved enough!

    You article brought to mind a product I noticed on a trip this summer to Kenya.  At the home of a struggling farmer, a “LifeStraw” hung on the wall of the hut. I inquired about it and learned it was handed out for free by a nonprofit working in the community.  I asked them mother if she uses it and the answer was “yes, when I have time”.  But most of the time she’s in a hurry to use the water and doesn’t take the time to filter it first. The product is facing issues two points you raised. Point #1, does she perceive the need for the product? While she understands clean water is important, it appears the need is not a high enough priority to always purify the water. And point #3, can she use it as designed? She feels the rate at which water comes through the filter is not fast enough for her.

  • BY David White

    ON September 5, 2014 11:45 AM

    very well stated….makes me consider our own work.  Thank you.

  • BY Steve Hardgrave

    ON September 10, 2014 11:13 PM

    Here, here! Spot on Kevin. So much of what will make a difference is the unglamorous hard work, persistence and tiny adjustments that don’t make headlines. We definitely need the pendulum to swing from talk to action.

Leave a Comment

 
 
 
 
 

Please enter the word you see in the image below: