Stanford Social Innovation Review : Informing and inspiring leaders of social change

SUBSCRIBE | HELP

Measuring Social Impact

Amplifying Local Voices

GlobalGiving’s storytelling project turns anecdotes into useful data.

 

Beneficiaries of the Trans-Nzoia Youth Sports Association in Kenya evaluate the organization through stories. (Photo by John Hecklinger)

A couple of years back, an American visitor to the slums in Kisumu, Kenya’s third largest city, handed out bumper stickers asking an open-ended question: “What does your community need? Tell us.” That got people talking. Their stories revealed growing dissatisfaction with a community-based youth sports organization that was receiving funding through GlobalGiving, a nonprofit marketplace for matching donors with projects. Eventually, and with community approval, the donor funding stream was redirected to a new organization that enjoyed stronger local support.

That’s far from the end of the story, however. Inspired by what happened when people were given a voice in Kisumu, GlobalGiving has continued to fine-tune its strategies for soliciting and making sense of the stories people tell about the projects intended to help them. For nonprofits and potential donors, “this helps you see what you’re doing through the eyes of the beneficiaries,” explains John Hecklinger, chief program officer for GlobalGiving.

Listening to stories may seem simple, but turning this into a method for monitoring development work has meant drawing on fields as diverse as complexity theory, behavioral psychology, and technology. Although GlobalGiving’s typical partners are grassroots organizations with small budgets, the storytelling project has garnered grant support from the Rockefeller Foundation because of potential benefits across the development sector. “There are thousands of small organizations that will never be able to afford or manage typical monitoring and evaluation functions,” says Nancy MacPherson, managing director of evaluation for the Rockefeller Foundation. “This could be a way to help smaller grantees be more systematic.”

The big goal, Hecklinger adds, is helping organizations get to better results more quickly. “We hope this leads to much faster and earlier detection of successes and failures to make the marketplace work better,” he explains. “It’s a way to create an organized and convenient mass of information that helps everyone from donors to beneficiaries make better decisions about what gets funded and what gets done.”

MAKING SENSE OF STORIES

David Snowden, a Welsh cognitive scientist and founder of a UK-based firm called Cognitive Edge, acknowledges that storytelling “is kind of in fashion with a lot of organizations.” Gathering heartwarming stories for their emotional appeal is not his aim. Rather, he’s interested in analyzing what he calls “micro-narratives.” These are the snippets of conversation we exchange while waiting in line at the supermarket or talking around a village campfire. They turn out to be quite useful for providing a snapshot of what’s on people’s minds.

Over the past decade, Snowden has developed a system for gathering and making sense of large quantities of micro-narratives. Listening to soldiers’ stories can improve troop safety in combat zones. Sales representatives’ stories can yield important insights for marketing. Until the GlobalGiving project came along, however, this approach had never been applied to development work.

Central to the Cognitive Edge approach is the conviction that storytellers are best qualified to interpret what their own narratives mean. Snowden has devised a simple system that enables people to put their stories into context. For example, if people are sharing stories about justice in their community, they might be asked whether a specific example is more about retribution, restitution, or revenge. They show how their story relates to those three potentially intertwining meanings by placing a dot on a triangle.

Cognitive Edge’s proprietary software, called SenseMaker, then turns this raw information into data that can be visually represented and analyzed to reveal patterns. With large volumes of data, the result “is like a 3-D landscape,” Snowden says. “You are able to see patterns, attitudes, and belief systems,” as stories form clusters around particular topics. The data can be filtered according to the storyteller’s gender, age, or other variables.

Irene Guijt, a Dutch consultant in organizational learning and evaluation, discovered Snowden’s work just as she was finishing her doctoral dissertation. As a method of generating fast feedback in the development sector, she saw this as “the best of both worlds. You get the value of stories and the merit of statistical analysis so that you can see patterns.” She envisioned how this information could shape community decision making “and lead to more innovation. It’s not a bunch of stories sitting in a dead library.” Guijt brokered introductions between the Rockefeller Foundation, Cognitive Edge, and GlobalGiving, and was soon part of a pilot project to test the storytelling approach in Kenya.

LEARNING FROM THE FIELD

Marc Maxson directs evaluation for GlobalGiving (and was also the bumper sticker-wielding visitor to Kisumu). He’s eager to develop better evaluation and monitoring methods for the 1,000 organizations that are GlobalGiving partners. Most operate on slim budgets with small staffs. Few have ever experienced a formal program evaluation. The handful that have been evaluated, he adds, have most likely never seen a copy of their own reports.

The storytelling approach can drive down the cost of evaluation to about 5 percent of more traditional methods, Maxson estimates. It’s also a way to gather community feedback—positive or negative—and share it quickly. “You don’t have to wait years for formal evaluation. This makes it cheaper for everyone to be effective.”

During the pilot year of the storytelling project in Kenya, Maxson saw “a 180-degree turn” in how grassroots organizations view monitoring and evaluation. “Instead of thinking of this as something that happens from the outside—from above—now it comes from within the community,” he says.

Maxson worked with local nonprofits to recruit native Kenyans who would gather stories from people they already knew in the community. Paid “scribes” underwent brief training in how to gather stories and have people interpret them. What motivates people to share their stories? Story sharers were entered into a lottery for a chance to win $100. Beyond that, says Hecklinger, “they have to see a value—that something might change as a result.”

Scribes were encouraged to ask deliberately open-ended questions. For example: “Tell us about a community effort that was successful (or one that failed).” Some stories that bubbled up spoke to broad community concerns, such as crime or jobs. Others were more specific and set the stage for follow-up by local service providers.

One cluster of stories mentioned the Trans-Nzoia Youth Sports Association (TYSA), a GlobalGiving partner that provides a range of services—school fees, uniforms, nutrition, and medical care—to children in the Rift Valley of Kenya. “A lot of the stories were about violence by police against youth. The organization’s goal has been to keep kids out of trouble,” Hecklinger says, “but now they are considering new strategies to work with the police as youth advocates.” Similarly, TYSA’s efforts to protect the rights of children got little mention in most stories, even though that’s a major emphasis of programming. “It is a gap we are seeing, a gap between our service and the community’s awareness,” explains a TYSA staffer. Storytelling doesn’t necessarily present a solution, Hecklinger notes, “but it can help organizations develop a hypothesis.”

The 2,500-plus stories collected in Kenya cited the work of more than 200 organizations. Many were previously unknown to the GlobalGiving team. Through stories, Hecklinger points out, “you detect what’s going on in your network and also outside it. You find out what you didn’t even know to ask.” As a result, some new organizations have been invited to join the GlobalGiving platform. “If you want to replicate innovation in the field,” he adds, “you have to be able to find it first.”

MORE TO LEARN

The eclectic team working on this effort agrees that there are more lessons to be learned. With a second round of funding from the Rockefeller Foundation, new projects are under way in Uganda and, soon, Tanzania to gather more stories and fine-tune best practices.

To share lessons learned, GlobalGiving has published an online guidebook called the Real Book for Story Evaluation Methods, coauthored by Maxson and Guijt. Among their conclusions: Community feedback needs to include multiple perspectives—those who have received services as well as those who haven’t. It needs to happen fast, so that feedback can be applied quickly to make improvements. It needs to generate action to make story sharing worth doing. And participants must be open to surprise—and willing to learn what they didn’t expect to learn.

GlobalGiving is working on strategies to make sure that stories collected will get back to the communities as well as to potential donors. The organization’s online platform will soon be updated with maps to show story locations, along with narrative feedback on individual project pages. Meanwhile, the role of technology to accelerate story gathering is still being worked out. Unreliable Internet connections in Kenya made Web-based tools impractical during the pilot. Eventually, there may be a role for SMS texting using mobile phones.

Guijt hopes to introduce the storytelling approach to other contexts, such as tracking values across the cotton supply chain. Already, she has a sense of where storytelling is appropriate and where it’s not. “This works best in situations where there is room to adapt en route. If you’re talking about a five-year program that’s set in stone, forget it. But if you anticipate a lot of obstacles between A and B and are willing to make changes along the way, then this approach becomes interesting,” she says.

Maxson ends her book with this advice: “Keep listening and keep sending these messages back and forth so that the people with the cash hear from the people in the grass of every grassroots project.”


Suzie Boss is a journalist from Portland, Ore., who writes about social change and education. She contributes to Edutopia and is co-author of Reinventing Project-Based Learning.

 
Tracker Pixel for Entry
 

COMMENTS

  • Cristina Sette's avatar

    BY Cristina Sette

    ON May 27, 2011 03:26 AM

    Excellent article. I can see great potential for using these tools in M&E. Glad also to see familiar names in your article, people I admire and respect. Thanks.

  • BY Jochum Stienstra

    ON June 3, 2011 03:56 AM

    Excellent paper. I like the simple, efficient and elegant way you describe Sensemaker. Now I see that I make it to complecated in my explanations! Too bad I couldn’t download the pdf. It appears to be damaged.

  • BY Oscar N

    ON June 5, 2011 11:56 PM

    Good read on M&E! I will look to use this within my non-profit. thanks!

  • muhu@erols.com's avatar

    BY muhu@erols.com

    ON June 30, 2011 12:07 PM

    Congratulations, Global Giving!

  • BY Jason W Anderson

    ON June 30, 2011 12:21 PM

    Really interesting piece.  As a former communicator for a environmental non-profit stories from the field—specifically success stories from the field - were vital to our fundraising efforts.  Yet, with small staffs and small budgets getting busy team members to collect stories and report back was a constant challenge.  This approach can show a more tangible ROI.  The conclusion offers another vital piece of information - circle back to your staff and communities in the field.  Make sure they know their story was told and how it was told.  Too often this gets overlooked and that hurts you the next time you go back asking for another story.

  • pelhamhoyt@aol.com's avatar

    BY pelhamhoyt@aol.com

    ON June 30, 2011 12:38 PM

    This will work really well in the beginning.  Then it will be “gamed” wherever it is repeated periodically.  Those who want donor money can easily “buy” votes and comments from the community.  Even here in the U.S., when there is a chance to win and award or other recognition via voting, nonprofits will call on their constituency to vote. (the American Idol effect)

  • BY Dean Cycon, Deans Beans Organic Coffee

    ON June 30, 2011 01:04 PM

    A wonderful article.  I should point out that we have been using this approach in our People-Centered Development program in the countries we buy from for over eighteen years. That’s eighteen years. There are plenty of small organizations around the world who have been engaged in this work. I only hope that one day organizations like yours open your lens to see and acknowledge the great work that has been done by these small groups for decades, without Rockefeller, USAID or other mega-grantors. There is a lot of human wisdom already out there.

  • Karen Nassi's avatar

    BY Karen Nassi

    ON June 30, 2011 03:13 PM

    I’m not sure I understand how this is a substantively new way of evaluating a program.  It seems there are two innovations being extolled here: 1) the technology that allows the parsing of prose data into numerical and statistically relevant data, and 2) the practice of asking beneficiaries of a given program about their experience with and opinions of said program.  The first does sound like a powerful new tool, and I would have liked to hear a little more about how it actually works.  But the second, as it is described in the article, appears to be simply the re-naming (into “storytelling”) of a practice with which we should already be quite familiar.  Unless we are NOT already querying beneficiaries and local communities, in which case there’s a huge problem that goes well beyond the scope of ‘evaluation’.

    What I find particularly interesting about the value of listening to people’s stories that isn’t mentioned here, is its usefulness in revealing very minor but key details in a bigger picture.  These are details that are often only articulated when people are chatting away to each other, and simply won’t come out in response to standard queries.  Details that are not necessarily omitted purposefully, but rather because people don’t think to point them out.  Until, that is, they get into the guts of their ‘story’ and the details become narrative steam.

  • BY MUUNGANO DEVELOPMENT GATEWAYS :MDG

    ON July 1, 2011 02:39 AM

    This is good program .i came across it today how can we too get involved.

  • BY Kare Anderson Center for the Edge + Say it Better

    ON July 1, 2011 08:28 PM

    Suzie this was a helpful story, citing several resources. Reminds me of message in the book (do not like title) Tell to Win, re value of a purposeful narrative that is meaningful to the listener so listener can see a role for themself in the story, want to be a part of it, remake and share it w/others

  • claire lyons  pepsico foundation's avatar

    BY claire lyons pepsico foundation

    ON July 2, 2011 10:55 AM

    Global Giving is a fabulous organization facilitating connectivity and importantly funding at the grassroots. Excellent. Thank you for this article.

  • BY العاب

    ON December 26, 2011 06:58 PM

    Good posting, im subscribing to your rss. Many thanks once more

  • BY Diane Hopson

    ON February 28, 2012 08:50 PM

    OMG, do you see whats transpiring in Syria? In spite of a brutal government crackdown, the demonstrations continue

  • BY Premal Shah

    ON October 23, 2013 05:03 PM

    Helpful post on M&E possibilities at Kiva.  Thanks!

Leave a Comment

 
 
 
 
 

Please enter the word you see in the image below: