Playing with microdata IV: response rates

The World Bank Group Country Survey FY 2014 for Myanmar which has been the subject of my last three posts has a low overall response rate of just 26%. Clearly, users may have to look beyond the survey data to make up their minds if the data reflect the opinions of the respective stakeholder groups.

That led me to look for the response rate situation of the opinion surveys in general. This paper, Understanding non response rates: insights from 600,000 opinion surveys by Germ´an Reyes, June 2016, seems to be a good one to look into. Contrary to my expectations, the paper was based on opinion surveys from Latin America, Spain, and more recently from the Caribbean and United States rather than a more global coverage. The data came from two databases arising from two complementary public opinion surveys:

The first one is conducted by Latinobar´ometro in 18 Latin American countries and Spain since 1995, interviewing about 1,200 individuals per country each year (with the exception of 2012 and 2014) about individual socioeconomic background, and preferences towards social and political issues. The second database comes from the Latin American Public Opinion Project (LAPOP). In addition to the countries covered by Latinobar´ometro, this database includes information from the Caribbean and United states, although the time span covered is shorter since the survey started in 2004, and the survey wasn’t conducted in 2011 nor 2013. LAPOP also interviews about 1,200 individuals per country each year about political issues. In both cases samples are representative at the national level for the voting-age population.

I was looking for response rates, that is, the percentage of respondents who responded from among the persons selected for interviewing. Alternatively put, I was looking for the opposite of those who didn't respond (the unit non-response). The paper however targeted at rates of non-response by each questions or group of questions (item non-response). In estimating the item non-response rates, denominator used in this paper was the total number of questions asked to an individual respondent and the numerator was the number of questions he/she chose not to answer. This ratio was then used as the dependent variable in regression analysis.

To my dismay I couldn't find any data on unit non-response rates in the paper and so I had to set a more modest objective of comparing unit non-response rates for World Bank Group Country Surveys from other countries in the ASEAN. Anyway, the figure-A2 shows that the public policies category had the lowest non-response rates compared to democracy, economy and socioeconomic categories. My guess is that it is so because we stand to lose or gain directly from public policies and our instinct, naturally, would be to let our voices heard on these matters.

If so why would we have a remarkably low response rate of 26% (a unit non-response rate of 74%)?

Some response rates for the World Bank Group Country Surveys for the ASEAN were:

Country	Number responded	Response rate	Year
Indonesia	316	29%	2015
Lao PDR	532	52%	2012
Lao PDR	164	45%	2015
Malaysia	258	30%	2016
Myanmar	173	26%	2014
Philippines	352	16%	2016
Vietnam	632	43%	2017

I am a bit more familiar with household surveys and I guess such big non-response rates would have been big problems there. Cited by Massey and Tourangeau response rate of less than 80% is officially considered to deserve serious attention in statistical surveys in the US:

The Office of Management and Budget’s Standards and Guidelines for Statistical Surveys, issued in 2006, recommend that investigators carry out studies like these to estimate the level of nonresponse bias whenever the response rate for a survey falls below 80 percent (http://www.whitehouse.gov/sites/default/files/omb/assets/omb/inforeg/statpolicy/standards_stat_surveys.pdf).

Before looking at the list of response rates for some ASEAN countries above, I thought that countries like Indonesia, Malaysia, and Philippines with more mature statistical systems and greater exposure of the public to sample surveys/opinion polls would have considerably better response rates than us. But in reality, even for the Malaysia with a higher response rate, 70% of stakeholder failed to respond! Perhaps opinion survey people have some tricks for living comfortably with such problems. Perhaps we could console ourselves that we were not that bad as far as response rates go in the ASEAN.

But can't we get a better response rate and what is hindering that? For me, I would've thought a little and write down my three development priorities straight. But think again, those stakeholders, those experts, were not answering that question only. The overall objective of the World Bank Group Country Survey in Myanmar was for “gaining a better understanding of how stakeholders in Myanmar perceive the Bank Group”. And for that matter they were to answer questions relating to:

Their overall attitudes toward the World Bank Group in Myanmar;

Overall impressions of the World Bank Group’s operations, knowledge work and activities, and communication and information sharing in Myanmar;

Perceptions of the World Bank Group’s future role in Myanmar.

I don't know what the stakeholders would have thought. But for a dumb guy like me, those questions look formidable. Despite the assurance of confidentiality, what if I couldn't come up with answers that were not intelligent enough? Like me, those stakeholders who didn't respond were worried that their reputations would be at stake in answering those questions and so they chose to play safe, I imagined.

For some time I've been laboring with this feeling of hopelessness for all our peoples trying to make a peaceful and simple living and playing our part with dignity in our planet, because there have been so many distractions within and without. I don't really know if I've cheated myself being convinced that there are all-purpose solutions to these, and slept well thereafter. Seems like we need to get more educated (get more ပညာ) to face all internal issues and look for effective “small state survival strategies” for external affairs. I guess getting more educated would mean getting better informed, getting more knowledge, and getting wiser. I'm not smart enough to say what underpins “small state survival”, but I could mention successful examples like Switzerland and Singapore, among others.

One glossary of Pali terms gives ပညာ as:

paññā:Discernment; insight; wisdom; intelligence; common sense; ingenuity. One of the ten perfections (pāramīs).

In the context of this post, the respondents don't need intelligence or ingenuity to outsmart the data collectors; they could just use their common sense and be strong enough to speak out, I would say. What is there to be afraid? After-all, we have to be masters of our own fate.