What Works With Respondents?
By Patrick Glaser, CMOR Director of Respondent Cooperation
November 2007

CMOR has launched a new research initiative into understanding how to boost respondent cooperation in Internet panel research. Already in its second run, the “Engineering Optimal Survey Introductions” study randomizes the components of survey invitations in order to identify the messages that are most likely to result in respondents agreeing to participate in a survey. Researchers will be able to examine the results from this research effort to assist them in tailoring messages to their respondents.

Background
Howard Moskowitz (Moskowitz-Jacobs, Inc.) initially approached CMOR with the idea of creating a “Respondent Cooperation Playbook.” The concept is simple; create a series of phrases that can be randomly combined with each other to make a host of survey introductions. Respondents are then asked to rate these introductions based on their appeal and strength as a motivator to participate in research. The study can then be replicated numerous times, and the results combined into a reference book for selecting survey introduction phrases and messages.

This study is similar to CMOR’s Tracking Study, which will give insight into the impact variables such as survey introduction language has on refusal rates/response rates (for more information about that study see http://www.cmor.org/rc/rcArticle8_07.cfm.).

Howard has successfully completed an initial run of the online introduction study in 2006, and invited CMOR to collaborate on the 2007 iteration of the study. Survey Sampling International and Greenfield Online graciously supported the effort by donating panel samples to the study.

Study Design
Six categories were developed that represent common concepts found in survey introductions: basic request to participate, importance of participation, survey topic, promise of confidentiality, time it takes to complete survey, and the direct benefit/incentive to participate. Within each of these categories, six phrases were also developed.

The following example shows two phrases from each of the six categories:

Each respondent was asked to rate 48 survey introductions. Each survey introduction was composed of four phrases selected at random.* An example of a survey introduction:

Your Opinion Counts!
Your participation in this study will help improve products and services
We need to ask you about 10 questions
Participate and earn a free gift card from Home Depot

After all 48 of the combinations were rated by the respondent, they were then asked to answer simple demographic questions.

Results & Next Steps
Through statistical analysis, each phrase was assigned a unique value which denoted its power in eliciting respondent cooperation. The results were combined in a single reference for researchers to use when designing survey introductions for panel studies. Going forward, CMOR will continue this research initiative in order to expand the “playbook for respondent cooperation.”

To learn more about this study, email Patrick Glaser (pglaser@cmor.org).

* Two categories were left out of each survey introduction at random, leaving a phrase to be selected from each of the remaining 4. This was done to ensure statistical independence.