Not to go too much into detail yet, but we’ve been spending some time with SPSS Desktop Reporter 4.0. Wow. Despite the price, it’s a great software package with a lot of extremely useful features and worth every penny.

What it allows you to do is to take an SPSS file (or any variety of other data files), create crosstab tables and charts (doing all sorts of fancy analysis along the way) and then save structure of what you’ve created so that you can then rerun all of the tables and charts against an expanded dataset later on. Makes it great for surveys that feature multiple waves. But it’s also great for ad-hocs. If you’re a fan of the SPSS Tables module you’ll absolutely love SPSS Destop Reporter, which is kind of like what you’d get if you built an entire application around SPSS Tables.

Like most SPSS products, you definitely pay for what you get and you’re probably looking at spending around $2,800 for one copy (the price goes down for additional copies). If you have the budget available, it’s definitely worth taking a look at.

Learn more at the SPSS web site.

The N, MTV/Nickelodeon’s evening network for teens, has announced an alliance with mobile community provider mbuzzy.com to create a teen wireless research panel using the mbuzzy technology platform beginning in January. The program, to be called "Teens Everywhere" will allow the alliance to conduct research with 10,000 teen panel members via their mobile phones on a variety of topics, ranging from network programming to advertising to events and other information.

Read the full press release at Yahoo.

Is it enough to survey 100 people or are you only going to get useful results if you survey 1,000 people? The answer, unfortunately, really depends on the questions you are asking, the likely results, and your preferred "margin of error" (the +/- 3% or +/- 4% you see posted with most survey results). You basically need this information so you can reliably know whether that 4% difference between the two bars on your graph mean anything or not.

Personally, when I’m conducting an online survey I tend to prefer a sample of 1,000. In an overall sense it is usually overkill, but it usually allows me to segment the results in a number of different ways — I can break the results down by age group, gender, income, etc — something I couldn’t necessarily do if I started from a much smaller sample. I suppose it is the luxury of having access to a large respondent base — I can afford to oversample. Believe me, if I were paying $10 a response (like what I sometimes have to do when I rent a panel of people in another country) I am much more conservative in my sample sizes and pay really close attention to my margin of error and the needs of the study.

There are some web sites out there that make margin of error more understandable. The Red River College Marketing Research blog recently pointed to an article at www.isixsigma.com entitled "Margins of Error Made Easy!" which I found to be worth reading.

There are also several sample size calculators out there (you can find them by typing "sample size calculator" into Google). One that appears especially handy is at dssResearch.com. Grapentime.com offers not just a sample size calculator, but also a sample size calculator for attribute ratings (in other words, it tells you the minimum sample sizes you need for different type of metric mesasurement scales).

Margins of Error Made Easy at Isixsigma.com
Sample size calculator at dssResearch.com
Sample Size Calculator for Attribute Ratings at Grapentine.com

Is it bad to offer incentives to people who complete your surveys? On more than one occasion I’ve had to answer this question for clients and others who were concerned that by offering a potential respondent some kind of perk for participating, you might somehow corrupt the data: either as a result of skewing the response base or by somehow influencing the respondent (perhaps to give you a better score, because they like you so much for giving them a free cup of coffee or an entry in a drawing to win a free Ipod).

My personal opinion is that there is nothing wrong with incentives — in fact, that they can be quite beneficial — as long as you are thoughtful about what you are offering and to who. For example, if I want to find out how interested people are in visiting a particular theme park, I probably should offer as my prize two tickets to the park in question — the only people who will take my survey are those who want the theme park tickets, and those will only be people who like theme parks — and so my results will, in fact, be pretty skewed. On the other hand, if I offer something more neutral — such as cash — then I should get a pretty broad based response since, as far as I can tell, liking cash has no bearing on whether or not you like theme parks.

If you need a more credible source than my personal opinion, a variety of studies have been conducted on the subject of incentives and the impact they have on research. I recently came across a literature review by Eleanor Simmons and Amanda Wilmont ("Incentive payments on social surveys: a literature review") in which they looked at incentives from many different angles, including the impact on response rates; monetary v. non-monetary incentives; value of incentives; differential incentives (which is when you only offer to pay incentives to people after they refuse to take your survey); the effect of incentives on interviewers; incentives and data quality; effects of incentives on sample composition; and the effects of incentives on long-term expectation effect.

Read "Incentive Payments and Social Surveys" here.

Triexa, a Toronto-based company formerly know as Sensation Designs, is presently beta testing a new web-based survey software system called AskItOnline. Although I haven’t had a chance to play with it yet (the software is in "private" beta with only some screenshots posted), what makes it fascinating is 1) it is a "Web 2.0" type program that utilizes a drag-and-drop web-based interface and a particularly reasonable pricing structure (four levels: "basic" which gives you very little functionaliy but it allows you to collect 100 surveys for free; "standard" which adds themes, some additional survey/design features and a cost of $7 a month for up to 5 surveys with 750 responses each; "professional" which gives you up to 10 surveys and 2500 responses per survey for $15 a month and "enterprise" which allows you to do an unlimited number of surveys with up to 15,000 responses each for $25 a month.

If you’re looking for an easy-to-use package that has a very low cost, this pacakge may be for you. Although each level contains different features, the enterprise version allows multi-page surveys, an unlimited number of questions per survey; "star ratings"; randomized responses; the ability to insert images and video; response validation; question piping; skipping/branch logic; and up to a terabyte of data storage. You can also review your results online, download your results, and receive e-mail notification whenever someone completes your survey.

The software is still in "private" beta, but it appears that they are still accepting new testers and it’s always fun to participate in the development of a new program.

Learn more about AskItOnline

Peter Leppik posted an insightful story on the VocaLabs blog about a recent trip to his local Home Depot where an apparently store sponsored signed informed customers "9 or 10 = PASS, 1 through 8 = FAIL." The sign was apparently in reference to the register tape survey customers were being asked to take when they got home.

Peter’s point is that "any time you give someone an incentive to hit a goal, you’re also creating an incentive to manipulate the metric."

What struck me as well, though, was the that it is so easy to get respondents to change their ratings based on such simple instructions. It is a good reminder that, in the end, respondents don’t care very much about the ratings they give and are happy to manipulate their responses to meet the needs of an anonymous sign posted in a store.

Read Peter’s post at VocaLabs.

SurveyGold from Golden Hills Software is an easy to use Internet-based survey software system that has been around for some time and has to some degree mastered the art of making online research a simple, don’t-have-to-think-to-hard-about-it process. Although the software is limited to a basic set of question types, it does allow for some basic skipping based on responses and if you’re looking for a program that will get you the results quickly with very little training, this package may be worth exploring.

I’m not going to go into an entire review of the software package here at this moment (although I did just download it — Golden Hills offers a free 30-day trial), what I did want to say is that version 8 of the software was just released a few weeks ago and it offers a variety of features such as:

• Opportunity to organize surveys into folders.
• Basic weighting.
• Multi-page web survey form (takes SurveyGold to the next level!)
• Basic validation
• Ability to import/export surveys for other SurveyGold users.
• Memorized filters (you can set up data filters for viewing your results and it will remember them)
• Wave reporting (view mdata collected over time against prior periods)
• Updated user interface (very XP-like in my opinion)
• Faster database
• Improved graphing in the reports module

Several bugs from the previous version of the software have also been fixed.

Learn more about SurveyGold 8

I recently received a request for information about the available data tabbing (cross-tab, table generating) software that is available out there and thought I might share what I know. Personally I was a little surprised that there aren’t more software packages out there that do this kind of work — the ones that are available are fairly expensive.

The idea of this type of software is that you can take a data set (perhaps in SPSS format or some other standard or not so standard format) and almost automatically generate tables for each of the questions in the dataset. Usually each table would feature a number of columns representing different market segments (the "banner") while the various responses to each question would be shown in each row.

Such software is usually capable of automatically doing various statistical tests, sometimes including but not necessarily limited to t-tests, z-test, ANOVA and Chi-Square. Some programs can also weight and sort the data, as well as merge data sets, provide sample balancing, generate charts to go with each table, and provide output into a number of different formats ranging from text files to Excel spreadsheets to Powerpoint templates.

WinCross is perhaps the most well known of the bunch, and has been around a long time (it is presently at version 7). It is entirely compatible with SPSS 15.0, allows for 32,000 data columns and up to 3,000 rows per table; offers a number of different statistical tests; and can important export in a number of formats. My biggest complaint is that the user interface seems like it was developed about 10 years ago, and that it costs $1,995 per copy (the more copies you buy, the less it costs). Learn more at the WinCross web site.

Version 2 of GMI Research Analyzer was recently released by Global Market Insite. It will also take your data set, crosstab it, create reports, and export it into a number of different formats — but it also allows you to do drag-and-drop analysis of you data and has a much friendlier interface. It can be purchased from the GMI web site for $995 per copy.

MarketSight is a completely online tool for analyzing your data and is sold as a subscription service for $995 per year. It offers much of the same data analysis and reporting capabilities as the WinCross or GMI tool, and it has the added advantage of being accessible from anywhere you havea web connection. There are some limits to the size of the files you can upload yourself (only 50,000 records) but if you’re looking for a fairly easy to use program that doesn’t require a lot of effort to learn, visit the MarketSight web site.

SPSS Desktop Reporter was recently released as an integrated member of the SPSS Dimensions suite of products. Based on my limited experience with the program it appears to be a very user-friendly tool that can be used independently of the other Dimensions products. If you’ve ever used mrTables — well, this is like mrTables, except that you have all of the data available on your desktop and there are many more options. SPSS Desktop Reporter has all of the features of the other software packages, plus especially easy integration with other SPSS products. It sells for about $2,800 per copy. More information is available at SPSS.com.

If you are on a "budget" (and I use that term loosely) and already have SPSS, you may be able to get SPSS Tables (a module available for SPSS) to do many of the things you need, albeit with the requirement that you do a lot of it manually. It sells for only $800 a copy and can be found on the SPSS web site.

Another package worth considering if you are on a budget is Memphis Market Intelligence’s Survey Explorer. Different versions of the software are available which offer the ability to work with differently sized datasets — at different prices. So if you are dealing with a medium sized data set (50 questions, with no more than 2500 respondents) you could get the "Personal" edition of the software for only $519, the "Professional" edition for $579, or the "Enterprise" edition for $749. There most advanced version of the software allows for an unlimited number of questions, an unlimited number of waves and up to 20,000 respondents and can be purchased for $1,439. Although I haven’t had a chance to play with it, it appears to offer many of the features of the more expensive packages and is worth looking into at the Memphis Software web site.

If there are any other packages that you would recommend, please don’t hesitate to send them to me.

• WinCross
• GMI Research Analyzer
• MarketSight
• SPSS Desktop Reporter
• SPSS Custom Tables Module for SPSS
• Memphis Software’s Survey Explorer

Although because of the nature of this blog it is somewhat hard to say whether anyone noticed, but I haven’t updated the site much in he last few weeks. Just about one month ago I started a new position in charge of Universal Orlando’s Consumer Insights program — and as a result I’ve been focused on getting things lined up for 2007. Now that things have "settled down" a bit, I hope to get back to posting regularly.

Just for the record, I should say that all of the opinions expressed on this blog are 100% my own and do not in any way reflect the views of Universal Orlando or any of the vendors I work with.