The cornerstone of effective research is the ability to gather information from representative samples. For MJB’s Fieldwork Unit, however, it is all part of a day’s work that routinely delivers an almost unheard-of 70% response rate to telephone surveys.
The high response rate, together with the high quality of the responses themselves, is a major reason that policy makers feel that they can rely on MJB’s findings as a basis for key decisions.
The task of achieving high response rates and representative samples has become more complex, as the use of the cell phone, the overabundance of surveys, and the trend toward internet surveying pose new challenges.
According to Chen Tzuk, Director of the Fieldwork Unit, a key difference between MJB and other private companies is the attention paid to the representativeness of the sample. Many companies are happy with the first five hundred responses to a survey, without worrying if those five hundred really represent the population being surveyed. In contrast, it is crucial for MJB to have a sample that fully represents the population by variables such as gender, ethnicity, age, and economic status.
“We invest a lot of time in the survey process,” said Tzuk. “How to get a good sampling base, how best to ask a question, how best to get in touch, how best to get their agreement to be interviewed and complete the interview.”
She noted the constant effort to learn from experience so as to reduce refusal rates. Fieldwork staff work closely with the researchers to phrase the questions in just the right way, to elicit quality responses. “We pre-test all of our questions to ensure that they can be understood clearly. Then, we pay close attention to the quality of the responses. Have the questions been understood? Are people answering honestly?”
A key component of quality control is an intensive training program. “We train our interviewers not only about the interview itself but we expose them to the issues that they are working on,” Tzuk explained. “This way, they understand the importance and are committed to doing their best.”
Tzuk regularly consults with colleagues in organizations such as the Central Bureau of Statistics to share experiences and discuss ways of overcoming difficulties.
Testifying to its high quality of work, MJB’s Fieldwork Unit has become the address for other organizations to learn about our best practices in obtaining quality responses. For example, MJB is a national expert in interviewing the Ethiopian population, having honed our methods with many surveys of this community over the years.
Today, broader social changes are posing challenges to the world of survey research.
The omnipresence of the cell phone has made it even more difficult to create a proper sampling base. Landlines used to be a standard household feature, and were listed in telephone directories. Today, fewer people have landlines, and phone directories do not always list cell phones. When people do answer their cell phone, they are often doing two or three other things while talking and are not giving the surveyor their full attention.
A second challenge is survey fatigue. Surveying is everywhere these days: on the phone, on the internet, on the street corner. People are tired of answering surveys, and particularly ones that are as long as MJB’s.
Such survey fatigue makes MJB’s high response rates even more impressive, especially when compared with the low rates among the larger survey firms—one of Tzuk’s colleagues from a private firm bragged at having a 20% response rate!
Finally, for professional groups, internet-based surveying has become almost the only method to reliably survey social service professionals, who prefer the flexibility to respond at a convenient time. Here as well, it is critical to employ techniques that maximize the response rates.
Tzuk is most proud of the fact that, through our work, MJB is giving a voice to respondents who come from disadvantaged backgrounds and who may be suspicious of answering personal questions to a stranger on the phone. “We work hard to explain that our surveys are dealing with important issues in their lives. We are not asking about laundry soap—we are finding out about their most pressing needs so that government, JDC, and others can create services to improve their lives.”
