Conducting an online survey yourself is fairly easy, but it does require a few ingredients (other than the survey itself). If you can come up with these three things you can probably execute your own customer survey online:

1. You need to select some a survey system. This part is fairly easy to come by as there are all sorts of options out there to choose from. If you don’t expect to do a lot of surveying then you are probably OK to use a fairly low cost option. If you’re doing it for a real life professional business I would be careful not to do it on a free system that doesn’t let you remove their branding (there is nothing more tacky then having your survey end on the surveymonkey signup page – your survey should end on your home page or something similar). If you’re going to show images, make sure that you choose a system that makes it easy for you to upload and insert images – don’t choose a system that makes you upload the images to your own web server since that will make things too complicated (especially if you don’t have a web site).

2. You need respondents. This is often the most tough part of conducting a survey online – you need to find people, preferably customers, to take your survey. If you have a web site you can definitely solicit your web site visitors to take your survey, keeping in mind that you don’t want to do anything that will stop them from making purchases. One option is to invite past customers to take your survey – hopefully you have plenty of e-mail address from past customers, and a carefully worded, friendly e-mail to them may get enough of them to take your survey – especially if you make it clear to them that you’ll be using their feedback in the development of your new products. If you don’t have any e-mail addresses and you want to get your feedback from a group of non-customers, then what you may want to do is rent an outside survey panel. You’ll pay upwards of $5 or more per response, but if the feedback is important enough, it may be worth it to you.

3. You need to offer an incentive. There are probably many people who will take your survey for free. Maybe they like you, maybe they like your product, or maybe they just like to take surveys. In most cases, there won’t be a enough people like this to give you a fleshed out sample. By introducing a simple incentive – the chance to win a drawing or even some kind of an exclusive discount – you can greatly increase the number of people who will take your survey without significantly increasing your costs. For example, one of my favorite incentives is to offer a chance to win a $200 gift certificate to giftcertificates.com. You’ll double or even triple your response rate which means that the quality of your responses will be significantly higher without spending significantly more money overall.

It would appear that SurveySoftwareReviews.com is published — or at least sponsored by — Prezza Technologies to promote Prezza Checkbox Survey software using fake reviews of Checkbox and competitors.

I have a number of Google alerts set up to help me keep up to date with new research related stuff on the web. I was going through it today when I noticed a survey software review on a web site called “SurveySoftwareReviews.com” that I had never remembered seeing before. Upon visiting the site, it appeared to contain reviews of Perseus SurveySolutions 6, Prezza Technologies’ Checkbox Survey Server, classapps SelectSurvey ASP, SNAP Surveys, Apian SurveyPro, ConfirmIt, Inquisite, KeySurvey, Quask, Raosoft, SPSS and Vovici. (more…)

SPSS recently put out a news release (which has been picked up by at least two news sites CRM Today and TMCnet) whose sole purpose appears to be to scare companies away from using the plethora of survey research tools in favor of their multi-modal survey system. How can you begin to know what your customers are thinking, reasons the release, if you only ask those who are online when hardly anyone is even using the Internet these days?

"Web-based surveys may appear to be less intrusive and easier to conduct, but without pen and paper or a good ‘old-fashioned’ telephone, organizations miss the opinions of many, including those without a computer, the forever and selectively computer illiterate and a large part of the senior population that simply missed the tech revolution."

The release goes on to remind everyone that paper and phone survey are "in many cases essential, if organizations expect to present clients with the most accurate and complete view of customer attitutes and opinions." 

It then proves its point by referring to a recent Pew report:

"In fact, the Pew Internet & American Life Project recently found 49 percent of Americans only occasionally use modern gadgetry and many others bristle at electronic connectivity — the Internet."

Wow. Who would have thought that in 2007 more than half of the US population either doesn’t use and/or extremely dislikes the Internet. We all may need to rethink our online programs and go back to the phone banks, door-to-door solicitors and shopping malls many of us have mostly abandoned.

…but before we do, here are some stats not included in the SPSS release:

• Total US population is about 300 million people with 225 million of them over the age of 17. (US Census)
• There are 178.8 million web users in the US (comScore, June 2007)
• 71% of all adults are online (Pew)
• 87% of 18-24 year olds, 83% of 30-49 year olds, 65% of those 50-64 and 32% of those over 65 are online. (Pew)
• 73% of white, 62% of black, and 78% of English-speaking Hispanic are online. (Pew)
• 73% of people living in Urban/Surburban Environments and 60% living in rural areas are online. (Pew)
• 93% of those earning $75K+, 82% of those earning $50K-$74K, 69% of those earning $30K-$49K and 55% of those earning less than $30K are online. (Pew)
• Total number of households is 105.4 million (US Census)
• Almost 70% of US households have Internet access at home.  (Leichtman Research Group Q1 2007)
• 53% of US households have high-speed access (Leichtman Research Group Q1 2007)

The Pew study that SPSS refers to in their release is called "A Typology of Information and Communication Technology User." The study measures not whether or not people have internet access (as implied by SPSS) but instead tries to categorize people by the degree to which information and communication technologies are utilized and enjoyed.

According to the report, only 15% of the population can be characterized as "Off the Network" — that is, individuals with neither cell phones nor internet connectivity. They tend to be in their mid-60s, nearly three-fifths are women. Only 7% have college degrees (vs. the US average of 27%) and only 4% earn over $75K a year (vs. the US average of 22%). They are the heaviest users of "old media" such as radio and TV but do not have the inclination to try new information and communication technology.

Obviously, not everyone is online and if you’re looking for a particularly special group you may want to revert to paper, pencils and phones. However, I’m thinking that for most purposes you’re going to be able to find who you’re looking for online.

But the implication of the SPSS release is that unless you use (expensive) multi-channel research techniques (provided by them?) you will be collecting bad information and misleading your clients. This isn’t true and is in fact extremely misleading.

• Read the SPSS news release
• Read Pew Report "Typology of Information and Communication Technology Users"

Response I recently posted to a question about survey incentives on LinkedIn:

I’ve fielded dozens of online surveys — some with incentives, some without incentives. There is a huge difference in the results that you get — even if your incentive is a sweepstakes-type and the prize isn’t really that significant.

It is important to think about the incentive that you are offering in the context of what it is you are trying to get from the research. For example, offering chocolate chip cookies as an incentive for a survey about how much you like chocolate chip cookies is only going to get you people who want chocolate chip cookies — which is going to obviously skew your results.

That isn’t to say that there isn’t a situation where you couldn’t offer chocolate chip cookies as an incentive for a survey about chocolate chip cookies — if you don’t care whether the respondents are chocolate chip cookie lovers or not — for example, if you had a new kind of chocolate chip cookie and you wanted to compare it to another kind of chocolate chip cookie and all you had available was a panel of chocolate chip cookie lovers…well, chances are you’ll be OK.

I once did a survey once about what *kind* of sweepstakes. For example, would people prefer one big prize or a lot of little prizes (so that the odds of winning would be better). The result was that they actually preferred a middle option — offer a couple of big prizes and a lot of little ones.

The best incentive I’ve ever offered? The chance to win back the value of your order (this was a satisfaction survey for an online store). The response rate was huge. And based on the average transaction value of what we were selling, the risks were minimal.

With all that said, you do need to be careful if you are going to do a sweepstakes type of thing. There are rules. In fact, your sweepstakes has to have rules — type "sweepstakes rules" into Google and you’ll find numerous templates to work with. You’ll note that if you offer anything substantial as a prize you’ll probably want to consult with a lawyer to ensure that you’re not opening yourself up for a massive class action law suit by accidentally breaking one of the rules (it doesn’t happen frequently, but it happen).

Another especially popular incentive that isn’t as frightening and might even drive some more business to your door is to offer some kind of coupon to everyone that completes the survey. A coupon for one of your products — perhaps even a product you’re trying to get rid of. Perhaps they even have to come into your store to claim their reward. That way, everyone is happy.

One final note: know that you don’t always have to offer an incentive. It depends on your subject matter and how much your pool of potential respondents care about you, your business, and potential impact that the results of the survey could have on their lives. If the topic is interesting and/or people feel that the results will have a positive impact on their lives then you probably don’t have to offer an incentive. If the results are mostly going to benefit you and everyone knows it then an incentive is probably in order.

A couple of days ago Richard Kassiasieh of Kassblog posted an interesting commentary about Markettools’ Zoomerang product in which he expressed some concern not about the functionality of the service (which he thought was pretty straightforward) but about the branding, pricing, and control one loses over data by using an outside resource.

In terms of the branding, Richard was concerned because the free version of Zoomerang ends with a message inviting respondents to use the system to create their own services, and one could argue that this would be somewhat unprofessional-looking way for a "real" company to end an exchange with a customer (although to be fair, a company that at least goes to the trouble to collect feedback from customers — even free online software — is much better than all of those companies that don’t collect any feedback at all). And to be completely fair, the paid version of Zoomerang does allow you to remove the branding.

But Richard raises a good point — if you are going to go to the trouble of collecting invaluable data from your customers, you at least owe it to yourself to use a process where you control the look and feel of the experience from beginning to end (which, I should again point out, is possible using the paid version of Zoomerang. But even there you need to make sure you change the settings appropriately and test your survey to make sure it looks exactly as you want it to look).

Richard’s second point is that services such as Zoomerang store your data on their servers. This is, of course, true for all web-based survey systems and there is no getting around it unless you only use survey software installed on your own computers that are maintained in your own facilities. You can to some degree mitigate the risk by making sure you understand the level of security and regular backup procedures of the company that you choose to use for your survey research.

Zoomerang doesn’t say precisely how your data is stored other than to indicate that information on members and panelists are stored in "secure databases" protected by passwords, and that data from surveys is stored at "a secure hosting facility with both physical and software-based security systems." For most people, this probably sounds secure enough but if you want to know more and if security really is a concern you may want to contact the provider before you start your survey.

Data retention policies are also important to consider. For example, if you use the free version of Zoomerang they say that your data will only be saved for 10 days following the start of the survey. The impression I get is that they do retain it longer than that and it can be accessed again if you sign up for a paid account, but it is always important to understand the data retention policies before you get started as well as whether or not it will be easy for you to download your data in a managable format when you are finished. Richard expressed concern that Zoomerang only allows a flat file format — well, I think for most people that is probably enough. There are a few services that will allow you to download your survey into a relational file format (such as an Access data file), but these services tend to cost more and frankly I’m not sure what most researchers would ever do with the data in that format anyways. SPSS takes only flat files.

Finally Richard raises the question of cost and why there aren’t more open-source alternatives. He speaks the truth when he notes that survey software isn’t really that hard to write and to that I really don’t have that much of an answer. I imagine it will happen eventually.

• Read Richard’s article about Zoomerang at Kassblog

Martin Day, a Director of SurveyGalaxy, wrote an interesting article entitled "20 Top Tips to Writing Effective Surveys." The article is simply laid out, easy to read, and offers some pretty simple straightforward tips for writing online surveys (or offline surveys for that matter). My favorite tip (of the 20) is "Ensure that the questionnaire flows: whenasking questions group the questions into clear categories as this makes the task of completing the survey easier for the participants."

Sometimes it seems that many of the folks writing surveys — even the professionals — don’t seem to get it that the people taking the survey are for the most part volunteers (unless you’re paying everyone who takes your survey and not doing some kind of a drawing, almost all of your respondents are effectively volunteers) and if you don’t make the experience interesting and perhaps even fun then it is unlikely that anyone is going to go to the trouble of finishing the survey.

Not every survey can be fun. Some surveys are on boring topics. Some surveys use complex methodologies that make it difficult to create any kind of positive user experience. But it seems to me that it is important for us to at least try. If we’re going to ask our volunteers to give us their time and their opinions, it seems that the least we can do is try to make the experience at least somewhat entertaining and interesting.

To that end, in addition to Martin’s article below, I also present you with a link to a page on SurveyGalaxy which offers a list of the most highly rated (i.e., most interesting) surveys available on SurveyGalaxy as rated by respondents. Note that these aren’t always the prettiest survey in the world or the most interactive — but something about them has made respondents give them high ratings.

• Article: 20 Top Tips to Writing Effective Surveys
• SurveyGalaxy List of Best Public Surveys

Peter Leppik posted an insightful story on the VocaLabs blog about a recent trip to his local Home Depot where an apparently store sponsored signed informed customers "9 or 10 = PASS, 1 through 8 = FAIL." The sign was apparently in reference to the register tape survey customers were being asked to take when they got home.

Peter’s point is that "any time you give someone an incentive to hit a goal, you’re also creating an incentive to manipulate the metric."

What struck me as well, though, was the that it is so easy to get respondents to change their ratings based on such simple instructions. It is a good reminder that, in the end, respondents don’t care very much about the ratings they give and are happy to manipulate their responses to meet the needs of an anonymous sign posted in a store.

Read Peter’s post at VocaLabs.

Ray Poytner of The Future Place Blog recently posted an article entitled "The Push/Push Dichotomy - or why most reportals fail." It considers the value of "research portals" — which, by the way, refers to web sites set up by research firms so that clients can, on their own, download and review the results of ongoing or recently completed surveys, usually before the research firm goes in and adds their insight/analysis to the data. The question is whether or not setting up such systems is as effective as research vendors send the results to clients directly.

My own experience is consistent with Ray’s — when I have in the past set up research portals at my company — or even uploaded the research to a central web site or other location where the user has to go and download it for themselves — I think the result has generally been that the research is ignored or receives spotty dissemination. With all of the work everyone has to do, very few people really have the time to veer away from their day-to-day activity to download — or even worse, generate — marketing research reports.

That is, unless they’re looking for something in particular. Ray makes a good point that there are certain types of research that do lend themselves to reportals. The key, then, is to know when a reportal is appropriate — and when it will really be used — and when it makes more sense to disseminate results using more active (rather than passive) methods.

Read Ray’s article at the Future Place Blog.

Danielle Rodgers of Small Business Branding posted an interesting and thoughtful commentary about the potential benefits and pitfalls of using marketing research to make decisions in the new product development process. Her observations, which she follows up on with detailed explanations and suggested countermoves (so you should read her article) are as follows:

• Very few people will really understand the vision of your product when you describe it to them.
• If you ask people to find a problem they will (even if they never would have noticed it in real life)
• Talk is cheap — people don’t always (often? ever?) do what they say are are going to do (such as buy your product).
• Sometimes respondents don’t answer for themselves, but instead try to guess what the market is thinking.
• People don’t always have words to describe what they really want or need.
• Some people may have strong negative reactions to specific elements of your product.
• Just because one type of person doesn’t like your product doesn’t mean that others won’t.

Read Danielle’s full article at Small Business Branding.

Customer research, taken out of its appropriate marketplace context, can be extremely misleading. Consider the scenario presented by Lior Arussy of the Strativity Group who in a recent DestinationCRM article where a research firm, after conducting a study to help a client indentify key loyalty factors to build greater customer relationships, came back with a finding that the most important thing the company could to to retain customers was to "excel in invoicing."

Lior argued (and I agree with him) that from the customer’s perspective accurate, on-time invoices — like clean bathrooms or safe rides at a theme park — aren’t reasons that most customers are going to do business with you. Sure, they’re important to maintain and ultimately speak to the gestault of how people perceive your business (nobody wants inaccurate invoices) but nobody is really going to choose you over your competitor if your greatest claim to fame is that you have the most accurate billing system in the business.

Says Lior:

"The goal of customer experience is not simply to stop upsetting people, it is to delight them and maximize revenues and loyalty. It is essential that market research surveys–and the client companies they purport to help–target and measure true experiences that help competitive differentiation."

To derive insight from research takes more than just good methodology and execution — it also requires an understanding of the business that your in and enough knowledge about your customers to be able to interpret the results in such a way leads to meaningful, actionable findings. In other words, you can’t simply leave it up to your research firm to go out, do a survey, and report back with results that you can immediately integrate into your business. You also need to bring to the tables your own experience and your own knowledge of the business at every stage of the research in order to ensure that the results that you get make sense in the context of your work.

Read Lior’s article at DestinationCRM.

David Tebbut of IWRBlog (Information World Review) recently posted some interesting observations about Confirmit’s dashboard application, in which an online survey system is used to track customer attitudes in real time and report the results in an automatically updated "dashboard" application. The idea is to be able to provide useful research results instantly — as soon as they are relevant — instead of having to wait hours, weeks or days for results.

In my own experience, the greatest challenge to this type of a dashboard — which in some ways speaks to the potential to integrate customer satisfaction directly into a balanced scorecard type system in a meaningful way — is the ability to collect enough data on a regular basis to cause the "dials" on the dashboard to reflect something meaningful. 10 or 15 responses a day are simply not enough for a system that is meant  to be continuously updated.

On the other hand, there are applications where such a system might be somewhat useful and relatively easy to "keep fed." For example, if on the way out of the store there was a single question that customers could answer — either as they walked out the door or as they checked out — and if there were enough registers in the store — it might be possible to collect enough data to make the dashboard meaningful. Or maybe if there were a way to ask the question on customer cell phones as they walk out of the store (perhaps a little less realistic).

Read more about this article at IWRBlog.

Mario Menti of msurveys.com recently posted a note on his blog describing how easy it was to create a survey, solicit responses, and then compensate the respondents in SecondLife. A very interesting, unique and possibily even useful experiment to be sure.

Second Life is a virtual community much like a massively-multiplayer online role-playing game (MMORPG) such as World of Warcraft, Everquest or Star Wars Galaxies except that instead of fighting monsters and completing quests your objective is…well…there isn’t really one. You buy property, meet people, simply fly around or create things. In fact, using the basic building blocks provided by the software you can create just about anything if you have enough patience and skill.

While most people build digital representations of physical things (like buildings, sculptures or stargates). Mario used his time and skill to create an online survey. In fact, he even set it up so that individuals who finish the survey are rewarded with in-game money.

Think about this in context with the recent findings of the Pew Internet & American Life project, where 52% of a broad-based, national sample agreed with the following:

"By the year 2020, virtual reality on the internet will come to allow more productivity from most people in technologically-savvy communities than working in the "real world." But the attractive nature of virtual-reality worlds will also lead to serious addiction problems for many, as we lose people to alternative realities."

The entire report (entitled "The Future of the Internet II") is fascinating and contains a number of interesting predictions for the future (along with what people thinking of them). Definitely worth a read.

I love Mario’s experiment. And while it is probably true that it is just a little too early to jump on this technological bandwagon (unless you’re trying to do a survey about virtual communities and the people who use them, it is probably going to be difficult to come up with a sample that is relevant to your research questions) I think the time won’t be so far off when data collected in environments such as Second Life is the norm.

Read about Mario’s Second Life survey experience on his blog.
Watch a video of someone completing the Second Life survey.

Roy Poynter of The Future Place has posted a fascinating, thoughtful note about a session he led at a recent meeting of the World Association of Research Professionals (ESOMAR) in which the topic of discussion was the degree to which marketing research is being changed by the Internet. I won’t rehash the entire post here (it is eloquently written and worth the few minutes it will take to read) but I think it is worth identifying a few highlights.

First point: the fact that surveys are easier to conduct and that people are participating in more surveys doesn’t necessarily mean that the resulting data is less valid. I agree. In my experience, while it is of course necessary to think about the questions you are asking in the context of the respondent base you are using (whether a rented panel or your own in-house list), there is no reason to think that the results you get from an online survey will be any more biased than the results you get from calling those same people on the phone or visiting them at their homes.

Second point: Shorter time lines caused to some degree by the availability of quick data collection capabilities, easy to use tools, and smaller research budgets may be leading to less time spent on thoughtful analysis. While this is certainly possible in some environments, I would argue that this doesn’t always have to be the case and that the same tools that are making things so quick and easy may be eliminating much of the grunt work and leaving more time for analysis. Plus, as we move forward, better analysis tools will do even more of the work leaving more time for thinking.

There is more: such as a discussion about the role of online panels, the growth of outsourcing and what marketing research will look like in the "Web 2.0 era" (Research 2.0).

One last point that I thought was especially thought-provoking:

"A key conclusion of the session was that Research 2.0 will need new skills and a greater understanding of how people are taking over the Internet. It was also clearly understood that any researcher who wishes to be competent at Research 2.0 will need to master conventional research."

Read Ray’s full post and view the slide show.

Read a summary of the event at mrweb.com

Jared Heyman, president of Infosurv, recently published a white paper to Knowledgestorm.com entitled "The Benefits of Hiring a Full-Service Online Research Firm — Software vs. Online Research Firms," in which he pretty much "proves" that survey software should only be used for very simple, unimportant polls that don’t require a thoughtful methodology or unbiased execution. He says:

Online survey software is well-suited for "quick and dirty" type surveys where  directional quantitative or qualitative is needed. These surveys often run very short, with 20 or less items, and do not require the statistical accuracy that a full-service research firm can assure. An example may include a survey of twenty PTA parents to see what they would like the school to accomplish that year for their children, where only a rough tally of program preferences is required. Such surveys also do not require complex response validation, piping, or branching logic.

Although I agree with Jared on some points, I do think some of his assumptions don’t necessarily reflect the reality of today’s online survey software (many packages make it quite easy to validate responses or branch from question to question) and doesn’t take into consideration the access that many companies now have to thousands if not hundreds of thousands of customer e-mail addresses, along with the demographic data necessary to pull a meaningful, relevant sample. Sure — if you have no idea what to ask or if you don’t feel comfortable formulating questions — or even if you don’t have access to a good respondent pool — then it definitely makes sense to work with a professional agency.

I wonder if there isn’t some middle ground for companies and organizations who have the in-house know-how to write acceptable questions but need some help analyzing the data; or maybe the analytical capabilities are in-house but a broad-based response panel is still needed because the in-house list isn’t adequate. In other words, given the current economic environment where companies can’t afford to do a lot of research if there isn’t a sizable market out there for research firms that are willing so sell their surveys a-la-carte at reasonable prices…as far as I know there are a few companies who will do this, and then there is always www.guru.com.

Jared’s article is an interesting (albeit a little biased) read and covers many of the issues that you will have to think about when you conduct research yourself instead of hiring a firm.

Read the white paper at KnowledgeStorm (registration required).