Unless you have a practically limitless number of potential survey respondents at your disposal, you probably spend a lot of time thinking about how you can maximize your response rates — that is, the percentage of people that you invite to take your survey who actually take it. Several of the professors at the UCR Sloan Center for Internet Retailing have put some time and effort into researching the problem and have recently published a working paper in which they discuss their findings to date.
The paper is entitled "An Optimal Contact Model for Maximizing Online Panel Response Rates," was written by Scott A. Nelsin, Thomas P. Novak, Kenneth R. Baker, and Donna L. Hoffman. Here are a few of the points I found interesting in the background section of the paper:
- Response rates from online surveys tend to be lower than from traditional survey methods. Some of the theories that have been put forth to explain this include respondent privacy concerns, technical problems, respondent confusion, and poor design.
- Standard methods used to increase response rates in the real world may not work online.
- Nearly 80% of consumer goods and 74% of B2B companies use online panels.
- Newly formed online panels often experience high response, although these levels drop quickly without proper management, which includes attractive incentives, pruning of non-responders, recruitment of new panelists, personalization of messages, and creating a sense of "community" among panelists.
- Online panels are very inexpensive: telephone surveys cost anywhere from $15 to $20 per respondent; mall intercepts cost around $10 per complete, while online surveys tend to cost $1 to $2 per response (for the panel owner).
The "meat" of the paper discusses a predictive model for categorizing respondents into different classes based on their expected response rate and then using those classifications develop an optimization model for determining how many invitations to send to each group in order to maximize a broad-based response.
The abstract of the paper:
We develop and test an optimization model for maximizing response rates for online marketing research survey panels. The model consists of: (1) a decision tree predictive model that classifies panelists into “states” and forecasts the response rate for panelists in each state, and (2) a linear program that derives a plan specifying how many panelists should be solicited from each state in order to maximize response rates. The linear program is forward looking in that it optimizes over a finite horizon during which S studies are to be fielded. It takes into account the desired number of responses for each study, the likely migration pattern of panelists between states as they are invited and respond or don’t respond, as well as demographic requirements. The model is implemented using a rolling horizon, whereby the optimal solution for S successive studies is derived and implemented for the first study; then, as results are observed, an optimal solution is derived for the next S studies, and the solution is implemented for the first of these studies, etc. The procedure is field tested and shown to increase response rates significantly, compared to random selection and the heuristic currently being used by panel management. Implications are discussed for further model development, implementation issues for online panel managers, and for the broader area of optimal contact models in customer relationship management.
Why you wouldn’t just send all of your invitations to the most responsive group? Well, if you send out surveys infrequently and if you feel confident that you high responders are representative of your entire market then there is not reason not to do that. However, if you do send out a lot of surveys and you want to capture the broadest sample, then you need to find a way to maximize your entire panel without relying too heavily on a few dedicated panelists. That is where this model can come in handly (if you can figure out how to implement it, of course).
