What is Survey Fatigue, and How Does the Myers-JDC-Brookdale Institute Avoid It?
There was a strange pattern in the National Medical Expenditure Survey in 1987. One portion of the survey targeted the 65+ population in the US, and its questions revolved around Activities of Daily Living (ADL), the umbrella term for tasks such as dressing and bathing, which act as benchmarks for a person’s ability to function independently. The research on aging and ADL points to people having increasing difficulties with these tasks as we age, and caregivers responding on behalf of a relative or friend echoed that the person they were helping was struggling more and more as time went on. However, many of the self-respondents seemed to be doing the opposite; they reported improving over the months between survey intervals.
Sociologist Nancy Mathiowetz set out to understand this miraculous trend. Did the self-respondents possess some special gene? Had they created the perfect lifestyle for staving off the effects of aging? Mathiowetz examined several possible explanations, and she ultimately suggested a different trait that the self-respondents had in common. They had realized that answering “yes” to having difficulties with any specific ADL opened up a series of follow-up questions about the exact details of the complications they were experiencing, and they merely wanted the survey to be over as quickly as possible.
Just One More Question
Consumers today are constantly bombarded with surveys. Every company justifiably wants feedback on its services, as do schools, health care professionals, community organizations, and governments. Survey-takers, however, are finding themselves growing reluctant to answer yet another set of questions appearing in their inbox, or losing focus during a survey and resorting to answering in whichever way is most efficient.
This rising lack of engagement is known as survey fatigue, and many organizations view it as an epidemic. In fact, an International Workshop on Household Survey Nonresponse has been held annually since 1990, with professionals meeting in a different country each year to share strategies of counteracting their continually decreasing response rates.
A Survey of Survey Improvement
What advice do the experts have? Survey length pops up in several guides, as does over-surveying. The key is to design concise questionnaires that ask only questions directly answering your topic, and send them out as rarely as possible. National Research Center, Inc. recommends surveying only once per year; by this standard, the ADL survey arguably earned the errors in its results, following up once every five months.
Yet is unclear how much of a difference these combined prescriptions can make. SurveyGizmo, a popular data collection platform, noted in 2015 that a response rate of 10-15% is average for external surveys (distributed to an audience, rather than internal surveys shared within an organization). For most of us, the tide of survey fatigue appears unstoppable.
The Myers-JDC-Brookdale Institute is proud to maintain a 70% response rate.
“The main way we make ourselves different from others,” says Ben Schlomi von Strauss, Brookdale’s Data Collection Team Leader, “is that we are not just asking about the TV; ‘what are you watching,’ ‘what do you buy.’ We ask people about the things that are close to them and matter to them. We present ourselves as people who are trying to make those things better.”
Nevertheless, subject matter alone is insufficient to adapt to the modern landscape, and Ben employs several practical tactics to increase survey effectiveness.
The Right People Asking…
One vital piece is training interviewers the right way.
“To everyone we tell as much as we can about the specific research. [The callers] come from a place of knowledge about what they’re doing and why they’re doing it.” Beyond empowering callers to better keep subjects engaged, Ben says, “it also gives them a sense of connectivity and belonging to the research itself,” improving the quality of their calls.
Ben also teaches interviewers to make an important distinction between types of refusals, and tailoring the next few lines toward bouncing respondents back from an initial rejection.
“If the person says they don’t have time, we say, ‘no problem; we’ll call you at a better time; which time is best for you?’ We’re kind of making a meeting with her. The next time, she knows we’re going to call, she knows who we are, and she kind of committed and made that next step.” For someone who instead does not want to talk at all, the caller will sit on the introduction longer, stressing why the respondent’s time is important, the privacy of his answers, and what is being done with the information.
The Right Question… Order
Brookdale has also learned that the presentation can make as large of a difference as the questions themselves. Shaping the surveys requires balancing what questions the researchers need to ask with the Data Collection team’s understanding of respondents’ attitudes.
For a recent survey on loneliness among elderly populations, one simple strategy started to create a secondary problem, and this in turn led to an unexpected innovation.
“In that questionnaire,” Ben recalls, “we had a chart that asked, ‘how often do you feel lonely,’ ‘do you have any friends,’ ‘do you have any close family that visit,’ and it made some of the people feel very uneasy and sometimes uncooperative, because they were sad.”
Typically, Ben advises pushing questions of this type to the end of a survey, so that respondents who decide to leave have already answered the majority of the questions. In this case, however, “we didn’t want to ask all the sad questions and then just say, ‘thank you, goodbye.’” This abrupt ending felt inconsiderate and could have made respondents less willing to pick up the phone again.
One of the researchers came up with an unconventional solution to maintain the order of the survey.
“We asked people to think about a happy event that had happened, saying, ‘take a few minutes to think and then tell me about it.’ First, we didn’t look at it in just a cold manner that we had gotten an answer to our questions. We were able to send off the people we interviewed in a better mood. By doing that, we also gave our callers the chance to feel better themselves.”
For Ben, “it’s a win-win situation… As a manager, it’s important to me to take care of their welfare, and it gives them a higher motivation to keep going. The structure itself contributes a lot.”
Seeking Information in the Information Age
A vital aspect of Ben’s role is to monitor and learn from how people engage with surveys differently from generation to generation. As one might expect, smartphones pose the largest challenge. Just a few years ago, Ben notes, callers would hope to catch respondents at home by the phone. “Today, catching them by the phone is irrelevant,” and people are only expecting, willing, and able to give their attention for shorter periods of time.
The most effective strategy to combat this trend is one few companies are willing to spend resources on, as it requires doing the opposite: putting more time and attention toward each respondent.
Ben often presents different forms of the survey to different groups; streamlined online questionnaires will raise response rates among executives, but the elderly population prefers to be called or approached face-to-face. “Sometimes half of the survey can be performed in one way, and the other in another,” Ben says. “It demands us to look very closely and see how we can change and bring new tools into that environment.”
Questions to Ask When Building Your Next Fatigue-Free Survey
The main obstacle for Brookdale—and everyone else collecting data—is that everyone else is collecting data. New platforms will only continue to increase the amount of surveys people receive every day.
As Ben summarizes it, “each person gets a lot of calls, and we need them to see us in a different light.”
To ensure that your organization shines under a different light, escapes the tide of survey fatigue, and does not find itself in North Carolina for this year’s International Workshop on Household Survey Nonresponse, you may want to survey your own surveys under these guidelines:
- Is it clear to my respondents how answering this survey will benefit them?
- If I am conducting interviews, are my callers informed and invested in the work?
- Are the tedious/unpleasant/problematic questions placed in a way that will not disengage respondents early on?
- Is this survey tailored to its target audience?
- Will my respondents feel heard in this survey, and do I have an opportunity to make them feel more positive about my organization as a whole?