25 Jun ANDE Global Webinar: Adapting Impact Measurement and Management to the COVID-19 Crisis
What impact measurement and management changes should your organization consider in response to the COVID-19 crisis?
Last week, ANDE hosted a webinar highlighting the resources and changes organizations can implement to adapt impact measurement and management both during and potentially after the crisis. The call featured Dun Grover from ACDI/VOCA and Amreen Choda and Mishkah Jakoet from Genesis Analytics.
“As we see emerging best practices on changing IMM practices in the face of COVID, the most immediate need is to rethink how we as organizations record data.”
As detailed by Choda, the first step for organizations is to evaluate their current data collection methods, including both the options available and the requirements for each. For example, if the organization would like to engage a wide range of individuals, one of the suggested methods would be an emailed online survey; however, this would require a viable email address and the expectation that respondents have regular access to internet and a technology platform to access their email. Choda also outlined the various associated costs, response rates, and other factors that are imperative to consider when determining the appropriate method for data collection.
Data Collection Spotlight: Focus Group Discussions
Choda provided the attendees with a helpful walk through of the various factors to consider when determining a method to collect qualitative data during the COVID-19 crisis. One of the instrumental techniques of qualitative data collection for many organizations has been Focus Group Discussions (FGDs), a qualitative research method that convenes a group of people with similar backgrounds or experiences to discuss a specific topic. In this case, the purpose of the FGD may be to understand the challenges local businesses are facing due to the COVID-19 crisis. How does an FGD structure change in a virtual world? Unsurprisingly, video conference platforms such as Zoom allow for organizations to conduct similar FDGs with slight modifications:
- Ensure participants are comfortable with the technology platform in advance of the survey. Many conferencing platforms provide tutorial videos that can be easily shared
- Assign each participant a number and let them respond to the questions in the order of their numbers
- When possible, switch on your video when making introductions and thanking participants at the end
- Should participants lack internet connectivity or hardware, conduct staggered FDGs where the lead connects one-on-one with the participants and the answers are then incorporated and built upon through additional FDGs
What are some top tips for virtual surveying?
During the session, Choda recommended several helpful tips for organizations to keep in mind when planning their virtual survey program some of which are listed below:
- Send a text message to inform participants ahead of time to improve response rates
- Run a duplication on the phone numbers in your sample to ensure respondents aren’t contacted multiple times and to reduce unnecessary costs
- Keep questions short and to the point (15 questions for SMS, 20 questions for Interactive Voice Response)
- Brand your phone number to increase the likelihood of engagement
- Make sure you pilot your survey outside the study sample
- Consider social and cultural norms (e.g., if the sample is majority women, consider using a woman’s voice for recorded questions)
For many, the current situation has revealed a need to develop new IMM (Impact Measurement and Management) metrics and potential partnerships to monitor the crisis
Grover presented a monitoring system used as the backdrop for a broader lesson: the uncertainty during this climate means we, as data collectors, may not know the metrics needed to understand the situation. As a result, Grover adopted a two-step survey process. First, open-ended “discovery” questions were used to factor in uncertainty and “capabilities” questions were employed to better understand the wider system dynamics. Below are some examples of these:
- Do you think the COVID-19 crisis provided you with some key lessons learned to prepare your company for future shocks?
- How are you changing your business model to adapt to the COVID-19 crisis and its effects?
- “Are you able to” or “can you” help grasp the dynamics between the respondent and the broader market system?
- Given the current situation, do you believe you can: find alternative buyers, access necessary financing, change products/services, etc.?
Surveys were then adapted in the second round based on what was gleaned in the first.
Grover shared another lesson learned during their survey experience. For many, there is a knee-jerk reaction to begin data collection at a time like this, but so many other parties are looking to conduct surveys as well, so it’s a natural time for partnerships. He explained how collaboration with other intermediaries to collect information reduces the burden of respondents to answer multiple surveys and enables partners to streamline their engagement while respondents cope with COVID-19.
Recognize the end goal is not to report, but to foster conversation and enable data-driven decision making
Grover emphasized the importance of generating information that can be interpreted and used by decision-makers to see through the “fog of information.” One way to facilitate this type of data-based decision making is through interactive data presentation. Through interactive tools such as Power BI users can learn from the data, facilitators can understand what elements of the data users are seeking to better present information and data can be easily fed into the platform to create visuals. Finally, Grover reiterated the need to ask, “who needs this information and what for?” It is these types of key questions will help data collectors design the metrics, tools and ultimately the final presentation of this useful information.