I finally received my copy of SPSS Statistics 17 a couple of days ago and determined that I would try to push past my frustration with the “new and improved” Java-based interface and use it to analyze some data for a report I’m working on. But I can’t do it. There aren’t enough compelling features in the new version of SPSS to help me get past my frustration with the Java interface. So I’m sticking with SPSS 15. (more…)
I spend a lot of time working in SPSS, and occasionally I need answers about various techniques and methods that aren’t readily available in the included documentation. Fortunately, there is a tremendous amount of free SPSS information and training materials scattered all over the web. Here are a few of my favorites. (more…)
It is probably fair to say that most online survey software companies get many (if not most, or even all) of their customers view the web. Many of them count on search engines, and those that do try to optimize their sites to drive as much traffic as possible. My only problem is when they “cheat” by posting fake reviews by fake people, comparing themselves to other products when the fact of the matter is that these comparisons aren’t really fair or valid.
Conducting an online survey yourself is fairly easy, but it does require a few ingredients (other than the survey itself). If you can come up with these three things you can probably execute your own customer survey online:
1. You need to select some a survey system. This part is fairly easy to come by as there are all sorts of options out there to choose from. If you don’t expect to do a lot of surveying then you are probably OK to use a fairly low cost option. If you’re doing it for a real life professional business I would be careful not to do it on a free system that doesn’t let you remove their branding (there is nothing more tacky then having your survey end on the surveymonkey signup page – your survey should end on your home page or something similar). If you’re going to show images, make sure that you choose a system that makes it easy for you to upload and insert images – don’t choose a system that makes you upload the images to your own web server since that will make things too complicated (especially if you don’t have a web site).
2. You need respondents. This is often the most tough part of conducting a survey online – you need to find people, preferably customers, to take your survey. If you have a web site you can definitely solicit your web site visitors to take your survey, keeping in mind that you don’t want to do anything that will stop them from making purchases. One option is to invite past customers to take your survey – hopefully you have plenty of e-mail address from past customers, and a carefully worded, friendly e-mail to them may get enough of them to take your survey – especially if you make it clear to them that you’ll be using their feedback in the development of your new products. If you don’t have any e-mail addresses and you want to get your feedback from a group of non-customers, then what you may want to do is rent an outside survey panel. You’ll pay upwards of $5 or more per response, but if the feedback is important enough, it may be worth it to you.
3. You need to offer an incentive. There are probably many people who will take your survey for free. Maybe they like you, maybe they like your product, or maybe they just like to take surveys. In most cases, there won’t be a enough people like this to give you a fleshed out sample. By introducing a simple incentive – the chance to win a drawing or even some kind of an exclusive discount – you can greatly increase the number of people who will take your survey without significantly increasing your costs. For example, one of my favorite incentives is to offer a chance to win a $200 gift certificate to giftcertificates.com. You’ll double or even triple your response rate which means that the quality of your responses will be significantly higher without spending significantly more money overall.
It would appear that SurveySoftwareReviews.com is published — or at least sponsored by — Prezza Technologies to promote Prezza Checkbox Survey software using fake reviews of Checkbox and competitors.
I have a number of Google alerts set up to help me keep up to date with new research related stuff on the web. I was going through it today when I noticed a survey software review on a web site called “SurveySoftwareReviews.com” that I had never remembered seeing before. Upon visiting the site, it appeared to contain reviews of Perseus SurveySolutions 6, Prezza Technologies’ Checkbox Survey Server, classapps SelectSurvey ASP, SNAP Surveys, Apian SurveyPro, ConfirmIt, Inquisite, KeySurvey, Quask, Raosoft, SPSS and Vovici. (more…)
SPSS recently put out a news release (which has been picked up by at least two news sites CRM Today and TMCnet) whose sole purpose appears to be to scare companies away from using the plethora of survey research tools in favor of their multi-modal survey system. How can you begin to know what your customers are thinking, reasons the release, if you only ask those who are online when hardly anyone is even using the Internet these days?
"Web-based surveys may appear to be less intrusive and easier to conduct, but without pen and paper or a good ‘old-fashioned’ telephone, organizations miss the opinions of many, including those without a computer, the forever and selectively computer illiterate and a large part of the senior population that simply missed the tech revolution."
The release goes on to remind everyone that paper and phone survey are "in many cases essential, if organizations expect to present clients with the most accurate and complete view of customer attitutes and opinions."
It then proves its point by referring to a recent Pew report:
"In fact, the Pew Internet & American Life Project recently found 49 percent of Americans only occasionally use modern gadgetry and many others bristle at electronic connectivity — the Internet."
Wow. Who would have thought that in 2007 more than half of the US population either doesn’t use and/or extremely dislikes the Internet. We all may need to rethink our online programs and go back to the phone banks, door-to-door solicitors and shopping malls many of us have mostly abandoned.
…but before we do, here are some stats not included in the SPSS release:
The Pew study that SPSS refers to in their release is called "A Typology of Information and Communication Technology User." The study measures not whether or not people have internet access (as implied by SPSS) but instead tries to categorize people by the degree to which information and communication technologies are utilized and enjoyed.
According to the report, only 15% of the population can be characterized as "Off the Network" — that is, individuals with neither cell phones nor internet connectivity. They tend to be in their mid-60s, nearly three-fifths are women. Only 7% have college degrees (vs. the US average of 27%) and only 4% earn over $75K a year (vs. the US average of 22%). They are the heaviest users of "old media" such as radio and TV but do not have the inclination to try new information and communication technology.
Obviously, not everyone is online and if you’re looking for a particularly special group you may want to revert to paper, pencils and phones. However, I’m thinking that for most purposes you’re going to be able to find who you’re looking for online.
But the implication of the SPSS release is that unless you use (expensive) multi-channel research techniques (provided by them?) you will be collecting bad information and misleading your clients. This isn’t true and is in fact extremely misleading.
Response I recently posted to a question about survey incentives on LinkedIn:
I’ve fielded dozens of online surveys — some with incentives, some without incentives. There is a huge difference in the results that you get — even if your incentive is a sweepstakes-type and the prize isn’t really that significant.
It is important to think about the incentive that you are offering in the context of what it is you are trying to get from the research. For example, offering chocolate chip cookies as an incentive for a survey about how much you like chocolate chip cookies is only going to get you people who want chocolate chip cookies — which is going to obviously skew your results.
That isn’t to say that there isn’t a situation where you couldn’t offer chocolate chip cookies as an incentive for a survey about chocolate chip cookies — if you don’t care whether the respondents are chocolate chip cookie lovers or not — for example, if you had a new kind of chocolate chip cookie and you wanted to compare it to another kind of chocolate chip cookie and all you had available was a panel of chocolate chip cookie lovers…well, chances are you’ll be OK.
I once did a survey once about what *kind* of sweepstakes. For example, would people prefer one big prize or a lot of little prizes (so that the odds of winning would be better). The result was that they actually preferred a middle option — offer a couple of big prizes and a lot of little ones.
The best incentive I’ve ever offered? The chance to win back the value of your order (this was a satisfaction survey for an online store). The response rate was huge. And based on the average transaction value of what we were selling, the risks were minimal.
With all that said, you do need to be careful if you are going to do a sweepstakes type of thing. There are rules. In fact, your sweepstakes has to have rules — type "sweepstakes rules" into Google and you’ll find numerous templates to work with. You’ll note that if you offer anything substantial as a prize you’ll probably want to consult with a lawyer to ensure that you’re not opening yourself up for a massive class action law suit by accidentally breaking one of the rules (it doesn’t happen frequently, but it happen).
Another especially popular incentive that isn’t as frightening and might even drive some more business to your door is to offer some kind of coupon to everyone that completes the survey. A coupon for one of your products — perhaps even a product you’re trying to get rid of. Perhaps they even have to come into your store to claim their reward. That way, everyone is happy.
One final note: know that you don’t always have to offer an incentive. It depends on your subject matter and how much your pool of potential respondents care about you, your business, and potential impact that the results of the survey could have on their lives. If the topic is interesting and/or people feel that the results will have a positive impact on their lives then you probably don’t have to offer an incentive. If the results are mostly going to benefit you and everyone knows it then an incentive is probably in order.
A couple of days ago Richard Kassiasieh of Kassblog posted an interesting commentary about Markettools’ Zoomerang product in which he expressed some concern not about the functionality of the service (which he thought was pretty straightforward) but about the branding, pricing, and control one loses over data by using an outside resource.
In terms of the branding, Richard was concerned because the free version of Zoomerang ends with a message inviting respondents to use the system to create their own services, and one could argue that this would be somewhat unprofessional-looking way for a "real" company to end an exchange with a customer (although to be fair, a company that at least goes to the trouble to collect feedback from customers — even free online software — is much better than all of those companies that don’t collect any feedback at all). And to be completely fair, the paid version of Zoomerang does allow you to remove the branding.
But Richard raises a good point — if you are going to go to the trouble of collecting invaluable data from your customers, you at least owe it to yourself to use a process where you control the look and feel of the experience from beginning to end (which, I should again point out, is possible using the paid version of Zoomerang. But even there you need to make sure you change the settings appropriately and test your survey to make sure it looks exactly as you want it to look).
Richard’s second point is that services such as Zoomerang store your data on their servers. This is, of course, true for all web-based survey systems and there is no getting around it unless you only use survey software installed on your own computers that are maintained in your own facilities. You can to some degree mitigate the risk by making sure you understand the level of security and regular backup procedures of the company that you choose to use for your survey research.
Zoomerang doesn’t say precisely how your data is stored other than to indicate that information on members and panelists are stored in "secure databases" protected by passwords, and that data from surveys is stored at "a secure hosting facility with both physical and software-based security systems." For most people, this probably sounds secure enough but if you want to know more and if security really is a concern you may want to contact the provider before you start your survey.
Data retention policies are also important to consider. For example, if you use the free version of Zoomerang they say that your data will only be saved for 10 days following the start of the survey. The impression I get is that they do retain it longer than that and it can be accessed again if you sign up for a paid account, but it is always important to understand the data retention policies before you get started as well as whether or not it will be easy for you to download your data in a managable format when you are finished. Richard expressed concern that Zoomerang only allows a flat file format — well, I think for most people that is probably enough. There are a few services that will allow you to download your survey into a relational file format (such as an Access data file), but these services tend to cost more and frankly I’m not sure what most researchers would ever do with the data in that format anyways. SPSS takes only flat files.
Finally Richard raises the question of cost and why there aren’t more open-source alternatives. He speaks the truth when he notes that survey software isn’t really that hard to write and to that I really don’t have that much of an answer. I imagine it will happen eventually.
Martin Day, a Director of SurveyGalaxy, wrote an interesting article entitled "20 Top Tips to Writing Effective Surveys." The article is simply laid out, easy to read, and offers some pretty simple straightforward tips for writing online surveys (or offline surveys for that matter). My favorite tip (of the 20) is "Ensure that the questionnaire flows: whenasking questions group the questions into clear categories as this makes the task of completing the survey easier for the participants."
Sometimes it seems that many of the folks writing surveys — even the professionals — don’t seem to get it that the people taking the survey are for the most part volunteers (unless you’re paying everyone who takes your survey and not doing some kind of a drawing, almost all of your respondents are effectively volunteers) and if you don’t make the experience interesting and perhaps even fun then it is unlikely that anyone is going to go to the trouble of finishing the survey.
Not every survey can be fun. Some surveys are on boring topics. Some surveys use complex methodologies that make it difficult to create any kind of positive user experience. But it seems to me that it is important for us to at least try. If we’re going to ask our volunteers to give us their time and their opinions, it seems that the least we can do is try to make the experience at least somewhat entertaining and interesting.
To that end, in addition to Martin’s article below, I also present you with a link to a page on SurveyGalaxy which offers a list of the most highly rated (i.e., most interesting) surveys available on SurveyGalaxy as rated by respondents. Note that these aren’t always the prettiest survey in the world or the most interactive — but something about them has made respondents give them high ratings.
Peter Leppik posted an insightful story on the VocaLabs blog about a recent trip to his local Home Depot where an apparently store sponsored signed informed customers "9 or 10 = PASS, 1 through 8 = FAIL." The sign was apparently in reference to the register tape survey customers were being asked to take when they got home.
Peter’s point is that "any time you give someone an incentive to hit a goal, you’re also creating an incentive to manipulate the metric."
What struck me as well, though, was the that it is so easy to get respondents to change their ratings based on such simple instructions. It is a good reminder that, in the end, respondents don’t care very much about the ratings they give and are happy to manipulate their responses to meet the needs of an anonymous sign posted in a store.
Ray Poytner of The Future Place Blog recently posted an article entitled "The Push/Push Dichotomy - or why most reportals fail." It considers the value of "research portals" — which, by the way, refers to web sites set up by research firms so that clients can, on their own, download and review the results of ongoing or recently completed surveys, usually before the research firm goes in and adds their insight/analysis to the data. The question is whether or not setting up such systems is as effective as research vendors send the results to clients directly.
My own experience is consistent with Ray’s — when I have in the past set up research portals at my company — or even uploaded the research to a central web site or other location where the user has to go and download it for themselves — I think the result has generally been that the research is ignored or receives spotty dissemination. With all of the work everyone has to do, very few people really have the time to veer away from their day-to-day activity to download — or even worse, generate — marketing research reports.
That is, unless they’re looking for something in particular. Ray makes a good point that there are certain types of research that do lend themselves to reportals. The key, then, is to know when a reportal is appropriate — and when it will really be used — and when it makes more sense to disseminate results using more active (rather than passive) methods.
Danielle Rodgers of Small Business Branding posted an interesting and thoughtful commentary about the potential benefits and pitfalls of using marketing research to make decisions in the new product development process. Her observations, which she follows up on with detailed explanations and suggested countermoves (so you should read her article) are as follows:
Customer research, taken out of its appropriate marketplace context, can be extremely misleading. Consider the scenario presented by Lior Arussy of the Strativity Group who in a recent DestinationCRM article where a research firm, after conducting a study to help a client indentify key loyalty factors to build greater customer relationships, came back with a finding that the most important thing the company could to to retain customers was to "excel in invoicing."
Lior argued (and I agree with him) that from the customer’s perspective accurate, on-time invoices — like clean bathrooms or safe rides at a theme park — aren’t reasons that most customers are going to do business with you. Sure, they’re important to maintain and ultimately speak to the gestault of how people perceive your business (nobody wants inaccurate invoices) but nobody is really going to choose you over your competitor if your greatest claim to fame is that you have the most accurate billing system in the business.
Says Lior:
"The goal of customer experience is not simply to stop upsetting people, it is to delight them and maximize revenues and loyalty. It is essential that market research surveys–and the client companies they purport to help–target and measure true experiences that help competitive differentiation."
To derive insight from research takes more than just good methodology and execution — it also requires an understanding of the business that your in and enough knowledge about your customers to be able to interpret the results in such a way leads to meaningful, actionable findings. In other words, you can’t simply leave it up to your research firm to go out, do a survey, and report back with results that you can immediately integrate into your business. You also need to bring to the tables your own experience and your own knowledge of the business at every stage of the research in order to ensure that the results that you get make sense in the context of your work.
David Tebbut of IWRBlog (Information World Review) recently posted some interesting observations about Confirmit’s dashboard application, in which an online survey system is used to track customer attitudes in real time and report the results in an automatically updated "dashboard" application. The idea is to be able to provide useful research results instantly — as soon as they are relevant — instead of having to wait hours, weeks or days for results.
In my own experience, the greatest challenge to this type of a dashboard — which in some ways speaks to the potential to integrate customer satisfaction directly into a balanced scorecard type system in a meaningful way — is the ability to collect enough data on a regular basis to cause the "dials" on the dashboard to reflect something meaningful. 10 or 15 responses a day are simply not enough for a system that is meant to be continuously updated.
On the other hand, there are applications where such a system might be somewhat useful and relatively easy to "keep fed." For example, if on the way out of the store there was a single question that customers could answer — either as they walked out the door or as they checked out — and if there were enough registers in the store — it might be possible to collect enough data to make the dashboard meaningful. Or maybe if there were a way to ask the question on customer cell phones as they walk out of the store (perhaps a little less realistic).
Mario Menti of msurveys.com recently posted a note on his blog describing how easy it was to create a survey, solicit responses, and then compensate the respondents in SecondLife. A very interesting, unique and possibily even useful experiment to be sure.
Second Life is a virtual community much like a massively-multiplayer online role-playing game (MMORPG) such as World of Warcraft, Everquest or Star Wars Galaxies except that instead of fighting monsters and completing quests your objective is…well…there isn’t really one. You buy property, meet people, simply fly around or create things. In fact, using the basic building blocks provided by the software you can create just about anything if you have enough patience and skill.
While most people build digital representations of physical things (like buildings, sculptures or stargates). Mario used his time and skill to create an online survey. In fact, he even set it up so that individuals who finish the survey are rewarded with in-game money.
Think about this in context with the recent findings of the Pew Internet & American Life project, where 52% of a broad-based, national sample agreed with the following:
"By the year 2020, virtual reality on the internet will come to allow more productivity from most people in technologically-savvy communities than working in the "real world." But the attractive nature of virtual-reality worlds will also lead to serious addiction problems for many, as we lose people to alternative realities."
The entire report (entitled "The Future of the Internet II") is fascinating and contains a number of interesting predictions for the future (along with what people thinking of them). Definitely worth a read.
I love Mario’s experiment. And while it is probably true that it is just a little too early to jump on this technological bandwagon (unless you’re trying to do a survey about virtual communities and the people who use them, it is probably going to be difficult to come up with a sample that is relevant to your research questions) I think the time won’t be so far off when data collected in environments such as Second Life is the norm.
Read about Mario’s Second Life survey experience on his blog.
Watch a video of someone completing the Second Life survey.
MRT is maintained by Mark Kupferman and
features reviews, news and trends in marketing research technology. "I'm trying to find the best marketing
research software and hardware packages for my company and clients and I'm sharing my findings with you. I welcome your comments and suggestions!"
You are currently browsing the archives for the commentary category.
