Research provider Knowledge Networks (Menlo Park, CA) has published an interesting nine page white paper entitled The Decisions Maker’s Guide to Online Research. Although the document is clearly geared towards illustrating how the Knowledge Networks panel methodology is better than others, the pamphlet does provide a thoughtful framework for deciding between web-based surveys with self-selected panels, web-based surveys with probability-based internet panels, mall intercepts, and mail surveys.

It identifies two types of online panels:

• Self selecting panels, which anyone can join — a "convenience sample" of the internet, and one that is likely to contain "professional respondents" and possibly even competitors trying to get insights into your secret plans;
• Probability-based Internet panels, which are (painstakingly) built by randomly selecting people, calling them, and then inviting them to join the survey panel.

One of the primary sources of data for the Knowledge Networks publication was a study conducted by the Stanford Institute for the Quantitative Study of Society entitled "Comparing the Results of Probability and Non-Probability Sample Surveys." The study begins by acknowledging that in general, the same survey conducted by two firms with the same methodology will usually yield comparative findings. However, most studies that led to this conclusion focused on surveys conducted in the same mode with comparable sampling methods. The folks at Stanford wanted to find out what would happen if the mode and sampling methods differed.

Nine data collection firms participated in the study: seven of whom use a self-selecting, volunteer sample (self selecting panel) and the other two who used probability-based panels (one that used a probability based telephone sample, the other (Knowledge Networks) that used an Internet-based probability panel). Each data collection firm asked their respondents the same set of questions, and the results were compared against benchmark probability-based responses.

The findings of the folks at Stanford were that the results were "remarkably comparable" across the board. Knowledge Networks had the most accurate findings, followinged by SRBI (telephone survey) and Harris Interactive (volunteer sample) who tied for second place (all of the others were about equally as accurate).

A few questions led to bigger differences between the self-selecting and the probability based methods: for example, volunteer respondents tended to be more comfortable using computers than probability-based respondents. Otherwise, however, it would appear based on the results of the study that a volunteer sample base will ultimately lead to results that are closely comparable to the more expensive probability-based sample.

Read the Knowledge Networks publication.
Read the press release announcing the publication.
Read the results of the 2005 Stanford study.

SurveyMania.com certainly isn’t the first web site to collect information about different companies that pay/reward people for taking surveys, but their recent press release and their relatively full-feature web site make really makes you stop and think about online survey panels and whether or not it is the same set of people who join all of them.

The site presently provides instructions for joining over 30 different panels, broken into categories such as "Consumers," "International," "Technies," "Parents," and "Specialty."

The site makes money from selling ad space to survey panels and through banner ads.

In addition to maintaining the list of online research panels you can join, the site also promises to e-mail members (at no cost to join) new opportunities as they arise. The site claims to have over 750,000 members (only 65,000 of which are in the United States).

Read the SurveyMania.com press release at PRWeb.
Visit SurveyMania.com