SUBSCRIBE | HELP

Measuring Social Impact

Humans: There’s No App for That

The inconvenient truth about data, technology, and social impact.

“SMS makes organizations more responsive to community needs!” “Social media brings us closer to the beneficiaries of our work!” “Mobile is changing everything!” “Social entrepreneurs everywhere are measuring and scaling social impact!”

A quick look at any social impact-related blog would lead you to think that the era of data-driven programming and results-orientation has arrived, made possible by breakthroughs in technology and the rise of social networking. Unfortunately, this rhetoric is entirely disconnected from the reality of the NGOs and social enterprises that we meet every day. What the hype obscures, is that data and technology are tools, two of many, that make up a “data system”—a human-centered process that takes into account the reasons for collecting data in the first place and the learning that might come from their analysis. It’s an inconvenient truth that most social impact organizations couldn’t accurately report the number of people they reach, let alone make claims about the significantly more complicated concept of “impact.”

There are two key reasons for this gulf between the hype around data and the current reality.

First, measuring social change is very difficult. In the for-profit world, data collected for the means of testing efficiency, researching potential markets, or tracking sales are all collected to answer one question: What is our profitability? Because the question is clear, the data system built to answer it—from conception to implementation to analysis to feedback—is also clear. In the social impact world, defining one’s intended change, and deciding on a fixed way to measure it, requires careful, strategic, and localized thinking that we unfortunately don’t see many organizations engaging in effectively. Not surprisingly, their data systems are usually a patchwork of paper forms and Excel files that contain unsystematic, low-quality information. To the extent that data collection does occur, it’s almost entirely through a process peripheral to daily operations, done to appease donors or investors; it is almost never an internally driven process of knowledge-seeking or improvement. Collecting data to answer someone else’s questions will never spark the enthusiasm and innovation necessary to build a true data system.

The second reason for the gap between rhetoric and reality is the misplaced emphasis on technology. There are hundreds of tools available to assist with the tasks that organizations perform daily—survey collection, SMS communication, buying and selling goods through mobile phones—and many are truly innovative, well-designed, and powerful. But knowing what questions to ask and having a plan for how to use the answers is a necessary prerequisite for the effective use of technology—technology alone will not conjure up answers to questions that haven’t yet been asked.

Over the last three years, our company, Vera Solutions, has been building cloud and mobile data systems for NGOs and social enterprises on Salesforce’s Force.com platform. We’ve been incredibly fortunate to work with some of the most innovative, thoughtful, and respected organizations in the world—to date, nearly 70 of them in more than 30 countries, working on health, education, sanitation, and other issues. In each case, we try to understand what the human users need data for and how they might interact with those data before we decide on the right technology solution.

Our work with Sanergy is a great example of this. Sanergy is a social enterprise located in Nairobi, Kenya, that makes hygienic sanitation affordable and accessible in urban slums. Its challenge—and ours, by extension—was to determine the driving questions for each internal department and to design a system in such a way that each department felt it was included in the process of data collection and data-driven programmatic decisions. Whereas Sanergy’s sales managers wanted to know which of their sales agents were talking to leads most likely to convert into actual Fresh Life Toilet buyers, for example, the Sanergy operations team needed data on whether Fresh Life Toilets with a higher percentage of child users were more or less profitable than other Fresh Life Toilets. The teams didn’t want to invest in unnecessary data collection, and certain technology applications were more and less useful depending on context. By focusing our technology design on people, in terms of a system’s usability, and on answers, in terms of the usefulness of data, we—Vera and Sanergy—have ensured that the resulting technology solution is a data system, rather than a database.

The social issues that organizations such as Sanergy and so many others address every day are profoundly complex. Data and technology solutions can be huge assets to these organizations in helping them manage, refine, and scale the work that they do. But the promise of both is bound to go unrealized until they are treated as components of a human-centered design process. The most important outcome of our work is not the technology solution that we come up with, but the systems of human thoughtfulness that coalesce around them. And there’s no app for that.

Tracker Pixel for Entry
 

COMMENTS

  • BY Jeremy Parrott

    ON June 12, 2013 04:24 PM

    Well said!

    On the positive side, at least the rhetoric is trending in the right direction…i.e. recognizing that performance based metrics are long overdue.

    The solution we implemented has nine performance metrics that speak to the overall impact a nonprofit has on society…none of which is enough to tell us anything significant…but together provide a unique insight to the success of a nonprofit mission.

    Defining social impact requires many variables…many of which are long term and correlate to reducing the overall demand for the services provided. Homeless shelters should not only be rewarded based on the number of beds they provide, but also by the number of beds needed.

    Again…great article!

  • Agreed! I have been working on the ground for an NGO called Pesinet that uses mobile technology to facilitate preventative health monitoring of young children in West Africa. Often funders and the public latch onto the project simply because the mobile technology component is fashionable, and we have to fight to explain what the core of our service is really about - the work of local agents in the community - and how the technological component is just facilitative and perhaps even non-essential. Moreover, in a country like Burkina Faso, where massive power outages and network failures are a frequent occurrence, and the population is 70% illiterate (and therefore unable to read SMS), technology definitely has it’s limits. In terms of reporting, we struggle every day with discrepancies in our database and information collected by hand, largely due to these infrastructure problems.

  • BY Rachel Armstrong

    ON June 14, 2013 12:49 PM

    I appreciate this conversation because I find it difficult to convert data into decisions. My organization, Farm Commons, provides legal education and services to U.S. sustainable farmers. I know how many farmers I reach, especially if I’ve created a contract or established a business. But, claiming I’ve reached “30 farmers” or what have you, isn’t important if I don’t know what impact the contract I drafted has on a specific farmer. ..and thus, if I should put more effort into more written contracts. Collecting data is cool, but it’s not always intuitive how to take action on the data. Many consultants dismiss that final step (but not Vera, which I can say from experience)!

Leave a Comment

 
 
 
 
 

Please enter the word you see in the image below: