Mobile Measurement: Lowering Costs and Engaging Beneficiaries
We should be paying attention to monitoring and evaluation innovation in developing countries, where technology leads to better programs.
Beneficiaries fail to complete monitoring surveys. Front-line staff devote weeks to chasing down responses and keying in information. The data is messy, so they spend even more time cleaning and validating the little data they’ve collected. By the time they’ve prepared reports, the window to use data for decision-making has passed, so leadership shares the numbers with funders and then casts them on the shelf.
All sound familiar?
If it does, then you should be paying attention to the considerable innovation occurring in monitoring and evaluation (M&E) in developing countries, where technology is dramatically lowering costs, accelerating data collection, improving response rates and accuracy, and ultimately leading to better programs.
Take Camfed, which supports the education and leadership development of more than one million girls and young women in rural Africa. In a pilot project last year, Camfed Zambia’s community-based data collectors traded in their pens and paper for software-equipped mobile phones that automatically transmit client data to Camfed’s central program database. Using the new platform, the collectors were able to gather data more easily, quickly, and accurately, and the organization was better able to analyze it and put the resulting insights to good use. According to Laurie Zivetz, Camfed’s director of impact, participating communities have become demonstrably more effective at tracking resources, monitoring education quality, and demanding accountability from government and schools. The pilot was such a huge success, in fact, that Camfed is rolling out the platform in two more countries by the end of this year, and in the remaining countries in which it works in 2012.
Or consider Souktel, a Middle East-based organization that has designed mobile phone-based M&E solutions for more than 20 USAID and UN projects. One of those projects is the Education Development Center’s (EDC’s) Somali Youth Livelihoods Program, which trains youth in financial literacy and pairs them with jobs. EDC adopted a Souktel-designed data collection system in 2009 to track program attendance, monitor student progress, and adapt content to learners’ needs—all in real time—and the results have been staggering. SMS surveys, sent to 200 youth in six different cities, regularly achieve an 80 percent response rate within one hour, and virtually eliminate an entire week’s worth of work for EDC—work that was previously needed to ship forms, locate youth, implement the survey, and enter the data. Not surprisingly, this approach has also cuts costs. Paper-based surveys required EDC to print 200 pages, mail packages to each city (and back), and incur wages for six dedicated staff to administer the survey and process the results—at an additional cumulative cost of $1,000 per survey relative to the mobile version. According to Kristen Roggemann, Souktel’s team leader for client outreach (and a former colleague at Bridgespan), the technology has reduced the time EDC staff devote to M&E by 75 percent, while yielding a 67 percent annual cost savings.
With mobile phone ownership outpacing web access by a factor of at least ten in most developing countries, Roggemann believes the new technology has just scratched the surface in its potential for improving M&E. She is quick to point out that mobile phone-based M&E isn’t meant to replace in-person interviews or long-form paper surveys; yet, a simple mobile handset can hold the key to dramatically better monitoring when an organization needs to collect time-sensitive data from a large number of people, and when beneficiaries are dispersed and resources are scarce.
The opportunities in developed countries appear no less fertile, particularly when considering that over 40 percent of mobile-phone users in the US and EU have already migrated to function- and feature-rich smartphones. According to Pew Research, this is particularly the case for minority youth in the US, a beneficiary group that domestic NGOs often struggle to report on with traditional M&E tools. Much like Camfed, these organizations could invest in real-time, smartphone-enabled data collection to replace the more common end-of-week, laptop-based data entry that is subject to the imperfect memories and available time of front-line staff.
So, no matter where you do your work, ask yourself: Has your organization adequately explored how technology can improve your monitoring? Could mobile-phone cameras validate self-reported beneficiary data? Could the emergence of instant speech translation programs on mobile phones dramatically reduce the need for live translators and improve your ability to learn directly from beneficiaries? Could Geographic Information Systems (GIS) mapping supplement traditional beneficiary tracking to identify physical and natural resource usage in the communities you serve?
Please share your stories of how mobile or other forms of technology are saving you time and money in your M&E.
Read more stories by Matthew Forti.