Roy Poynter of The Future Place has posted a fascinating, thoughtful note about a session he led at a recent meeting of the World Association of Research Professionals (ESOMAR) in which the topic of discussion was the degree to which marketing research is being changed by the Internet. I won’t rehash the entire post here (it is eloquently written and worth the few minutes it will take to read) but I think it is worth identifying a few highlights.

First point: the fact that surveys are easier to conduct and that people are participating in more surveys doesn’t necessarily mean that the resulting data is less valid. I agree. In my experience, while it is of course necessary to think about the questions you are asking in the context of the respondent base you are using (whether a rented panel or your own in-house list), there is no reason to think that the results you get from an online survey will be any more biased than the results you get from calling those same people on the phone or visiting them at their homes.

Second point: Shorter time lines caused to some degree by the availability of quick data collection capabilities, easy to use tools, and smaller research budgets may be leading to less time spent on thoughtful analysis. While this is certainly possible in some environments, I would argue that this doesn’t always have to be the case and that the same tools that are making things so quick and easy may be eliminating much of the grunt work and leaving more time for analysis. Plus, as we move forward, better analysis tools will do even more of the work leaving more time for thinking.

There is more: such as a discussion about the role of online panels, the growth of outsourcing and what marketing research will look like in the "Web 2.0 era" (Research 2.0).

One last point that I thought was especially thought-provoking:

"A key conclusion of the session was that Research 2.0 will need new skills and a greater understanding of how people are taking over the Internet. It was also clearly understood that any researcher who wishes to be competent at Research 2.0 will need to master conventional research."

Read Ray’s full post and view the slide show.

Read a summary of the event at mrweb.com

Jared Heyman, president of Infosurv, recently published a white paper to Knowledgestorm.com entitled "The Benefits of Hiring a Full-Service Online Research Firm — Software vs. Online Research Firms," in which he pretty much "proves" that survey software should only be used for very simple, unimportant polls that don’t require a thoughtful methodology or unbiased execution. He says:

Online survey software is well-suited for "quick and dirty" type surveys where  directional quantitative or qualitative is needed. These surveys often run very short, with 20 or less items, and do not require the statistical accuracy that a full-service research firm can assure. An example may include a survey of twenty PTA parents to see what they would like the school to accomplish that year for their children, where only a rough tally of program preferences is required. Such surveys also do not require complex response validation, piping, or branching logic.

Although I agree with Jared on some points, I do think some of his assumptions don’t necessarily reflect the reality of today’s online survey software (many packages make it quite easy to validate responses or branch from question to question) and doesn’t take into consideration the access that many companies now have to thousands if not hundreds of thousands of customer e-mail addresses, along with the demographic data necessary to pull a meaningful, relevant sample. Sure — if you have no idea what to ask or if you don’t feel comfortable formulating questions — or even if you don’t have access to a good respondent pool — then it definitely makes sense to work with a professional agency.

I wonder if there isn’t some middle ground for companies and organizations who have the in-house know-how to write acceptable questions but need some help analyzing the data; or maybe the analytical capabilities are in-house but a broad-based response panel is still needed because the in-house list isn’t adequate. In other words, given the current economic environment where companies can’t afford to do a lot of research if there isn’t a sizable market out there for research firms that are willing so sell their surveys a-la-carte at reasonable prices…as far as I know there are a few companies who will do this, and then there is always www.guru.com.

Jared’s article is an interesting (albeit a little biased) read and covers many of the issues that you will have to think about when you conduct research yourself instead of hiring a firm.

Read the white paper at KnowledgeStorm (registration required).