How long should your online survey be? I’ve seen and participated in a number of discussions on this topics over the last several years. After all, you want to collect as much information as possible from your respondents but at the same time you don’t want to annoy them to the point where they quit the survey half-way through. I’ve compiled a list of four factors/questions which determine how long your online survey can reasonably be to avoid dropouts. (more…)

Most survey software systems offer the ability to send out invitations to your potential respondents on your behalf. But how many of the e-mails actually get through, and how many end up in the spam folder? If there is one thing I’ve learned over the years, you can’t assume that your survey software provider will offer the same level of e-mail deliverability management as an e-mail vendor. (more…)

Have you ever gone to analyze the results of a survey in which you’ve asked for open-ended numeric information (that is, number based fields where respondents type in a number instead of choosing from a drop down list) only to find that many/some of your respondents have entered things like “10-20″ or “about 15?” You’re not alone. While many survey systems work to defeat these “alphanumeric elaborations,” Marak Fuchs published a paper last year in which he discussed the “optimal visual design of open-ended frequency questions in web surveys in order to reduce the percentage of alphanumeric elaborations and explicit extremes to frequency questions.” (more…)

So let’s say that you need to get 500 survey responses. Which is going to be more efficient: sending a list of potential respondents a $5 gift cards along with a request to take a survey or them the promise of a $10 gift certificate if they take your survey? In 2002 Alhoscha Kaplan and Glenn White of Ernst & Young published a paper in which they did such a test and their results were a little surprising. (more…)

It is probably fair to say that most online survey software companies get many (if not most, or even all) of their customers view the web. Many of them count on search engines, and those that do try to optimize their sites to drive as much traffic as possible. My only problem is when they “cheat” by posting fake reviews by fake people, comparing themselves to other products when the fact of the matter is that these comparisons aren’t really fair or valid.

(more…)

Conducting an online survey yourself is fairly easy, but it does require a few ingredients (other than the survey itself). If you can come up with these three things you can probably execute your own customer survey online:

1. You need to select some a survey system. This part is fairly easy to come by as there are all sorts of options out there to choose from. If you don’t expect to do a lot of surveying then you are probably OK to use a fairly low cost option. If you’re doing it for a real life professional business I would be careful not to do it on a free system that doesn’t let you remove their branding (there is nothing more tacky then having your survey end on the surveymonkey signup page – your survey should end on your home page or something similar). If you’re going to show images, make sure that you choose a system that makes it easy for you to upload and insert images – don’t choose a system that makes you upload the images to your own web server since that will make things too complicated (especially if you don’t have a web site).

2. You need respondents. This is often the most tough part of conducting a survey online – you need to find people, preferably customers, to take your survey. If you have a web site you can definitely solicit your web site visitors to take your survey, keeping in mind that you don’t want to do anything that will stop them from making purchases. One option is to invite past customers to take your survey – hopefully you have plenty of e-mail address from past customers, and a carefully worded, friendly e-mail to them may get enough of them to take your survey – especially if you make it clear to them that you’ll be using their feedback in the development of your new products. If you don’t have any e-mail addresses and you want to get your feedback from a group of non-customers, then what you may want to do is rent an outside survey panel. You’ll pay upwards of $5 or more per response, but if the feedback is important enough, it may be worth it to you.

3. You need to offer an incentive. There are probably many people who will take your survey for free. Maybe they like you, maybe they like your product, or maybe they just like to take surveys. In most cases, there won’t be a enough people like this to give you a fleshed out sample. By introducing a simple incentive – the chance to win a drawing or even some kind of an exclusive discount – you can greatly increase the number of people who will take your survey without significantly increasing your costs. For example, one of my favorite incentives is to offer a chance to win a $200 gift certificate to giftcertificates.com. You’ll double or even triple your response rate which means that the quality of your responses will be significantly higher without spending significantly more money overall.

Julie Lemaster, an MBA student at the University of California- Riverside, has written an interesting paper (which is posted to the Sloan Center for Internet Retailing web site) entitled "Online Panel Management and Strategies: An Introduction for Managers." It is an introductory guide to managers who have been asked or have decided it is time to start looking into online market research for their companies.

Lemaster contrasts full-service providers, such as M/A/R/C and SSI against complex and potentially expensive "self-serve" packages from Confirmit, Globalpark, GMI, and SurveyZ to low cost providers such as QuestionPro, Survey.com, SurveyMoney, and Zoomerang.

The abstract of the paper summarizes it as providing…

"…an introductory guide to managers who have been asked or have decided it is time to start looking into online market research for their company.  The size of the company you work for does not matter, as we will discuss several methods that can be used for any size company or investment level.  This paper is for managers who want to quickly learn the basic issues of online market research panels.  It will also be useful to managers who need to become familiar with some of the major providers of online panel management."

In addition to reviewing the providers and the various strategies for organizing your panel, Lemaster discusses a variety incentives that can be used to motivate and retain participants, such as lotteries, bonus points, and raffles.

• Online Panel Management and Strategies: An Introduction for Manager

Unless you have a practically limitless number of potential survey respondents at your disposal, you probably spend a lot of time thinking about how you can maximize your response rates — that is, the percentage of people that you invite to take your survey who actually take it. Several of the professors at the UCR Sloan Center for Internet Retailing have put some time and effort into researching the problem and have recently published a working paper in which they discuss their findings to date.

The paper is entitled "An Optimal Contact Model for Maximizing Online Panel Response Rates," was written by Scott A. Nelsin, Thomas P. Novak, Kenneth R. Baker, and Donna L. Hoffman. Here are a few of the points I found interesting in the background section of the paper:

• Response rates from online surveys tend to be lower than from traditional survey methods. Some of the theories that have been put forth to explain this include respondent privacy concerns, technical problems, respondent confusion, and poor design.
• Standard methods used to increase response rates in the real world may not work online.
• Nearly 80% of consumer goods and 74% of B2B companies use online panels.
• Newly formed online panels often experience high response, although these levels drop quickly without proper management, which includes attractive incentives, pruning of non-responders, recruitment of new panelists, personalization of messages, and creating a sense of "community" among panelists.
• Online panels are very inexpensive: telephone surveys cost anywhere from $15 to $20 per respondent; mall intercepts cost around $10 per complete, while online surveys tend to cost $1 to $2 per response (for the panel owner).

The "meat" of the paper discusses a predictive model for categorizing respondents into different classes based on their expected response rate and then using those classifications develop an optimization model for determining how many invitations to send to each group in order to maximize a broad-based response.

The abstract of the paper:

We develop and test an optimization model for maximizing response rates for online marketing research survey panels.  The model consists of:  (1) a decision tree predictive model that classifies panelists into “states” and forecasts the response rate for panelists in each state, and (2) a linear program that derives a plan specifying how many panelists should be solicited from each state in order to maximize response rates.  The linear program is forward looking in that it optimizes over a finite horizon during which S studies are to be fielded.  It takes into account the desired number of responses for each study, the likely migration pattern of panelists between states as they are invited and respond or don’t respond, as well as demographic requirements.  The model is implemented using a rolling horizon, whereby the optimal solution for S successive studies is derived and implemented for the first study; then, as results are observed, an optimal solution is derived for the next S studies, and the solution is implemented for the first of these studies, etc.  The procedure is field tested and shown to increase response rates significantly, compared to random selection and the heuristic currently being used by panel management.  Implications are discussed for further model development, implementation issues for online panel managers, and for the broader area of optimal contact models in customer relationship management. 

Why you wouldn’t just send all of your invitations to the most responsive group? Well, if you send out surveys infrequently and if you feel confident that you high responders are representative of your entire market then there is not reason not to do that. However, if you do send out a lot of surveys and you want to capture the broadest sample, then you need to find a way to maximize your entire panel without relying too heavily on a few dedicated panelists. That is where this model can come in handly (if you can figure out how to implement it, of course).

• Optimal Contact Model for Maximizing Online Panel Response Rates" at UCR eLab and Sloan Center

Phil and Ken Berry of BayaSoft recently gave a presentation at the 2006 American Marketing Association Research Conference designed to "equip market researchers with the tools and knowledge to design questionnaire documents that can be easily converted into online surveys, with minimal pitfalls." If you’re working on surveys and would like a little guideance to help you improve your communications with your programmers, it’s worth a read.

The presentation is available at the BayaSoft web site.