Visit the Salt Room Orlando web site.

One example of the new technology is the ability of Confirmit users to have their survey automatically rendered “outside the HTML/browser paradigm.” For example, Confirmit Flex makes it easy to program your survey for regular web users and then have that survey also appear on iPhones, or in Flash, or as a Widget on a blog.

The software makes it possible for Confirmit to launch new elements of it’s system quickly, without disrupting the regular “main” Confirmit codebase which is so important to so many research agencies. As they describe it:

Effectively, Confirmit Flex has allowed us to create a system that at its heart is a super-tanker (Confirmit) but which can be improved by speed boats (Flex components) enabling us, our partners, and clients to respond rapidly to market changes and take advantageof emerging technologies.”

Read more about Confirmit Flex (including more technical details)

Confirmit Flex Press Release

As Tim Macer reports, it does have a bit of a learning curve, but once you work your way through the various manuals (one manual for data processing and setting up surveys; another manual for doing actual data analysis) you’ll find that the software is exceptionally powerful and very useful. It has a variety of data important options, a number of different ways you can process the data, and it integrates very well with Microsoft Excel and Powerpoint.

I won’t rewrite all of Tim’s article (you can read it by following the link below), but I will say that if you do a lot of tracking surveys and want to easily be able to cut your data every which way in a fairly effortless manner (once you get the survey set up right!) then this is the software package for you.

Read Tim Macer’s review of Ruby Tabulation Software

Visit the Red Centre Software website

• Improved research and reporting tools
• New bootstrapping module for more stable models
• New non-parametrics tests procedure
• Automated data preparation for easier model building
• New Direct Marketing add-on module
• More customizability in the base PASW Statistics
• Easier enterprise integration, deployment and management

PASW Bootstrapping Module

As I understand it, this new module lets you automatically run statistical tests on thousands of randomly drawn sub-samples of your dataset, which lets you minimize impact of outliers and anomalies on your results. If this is easy to use it could be exceptionally useful and time saving. It can be applied to the following SPSS/PASW procedures:

• Descriptive Procedures
  • Frequencies (PASW Statistics Base)
  • Descriptives (PASW Statistics Base)
  • Examine (PASW Statistics Base)
  • Means (PASW Statistics Base)
  • Crosstabs (PASW Statistics Base)
  • T-tests (PASW Statistics Base)
  • Correlations (PASW Statistics Base)
  • Partial Correlations(PASW Statistics Base)
• Modeling Procedures
  • One-Way (PASW Statistics Base)
  • UniAnova (PASW Statistics Base)
  • GLM (PASW Advanced Statistics)
  • Regression (PASW Regression)
  • Nominal Regression (PASW Regression)
  • Discriminant (PASW Statistics Base)
  • Logisitic Regression (PASW Regression)
  • Binary Multi-nominal Logistic Ordinal Regression (PASW Statistics Base)
  • GENLIN (PASW Advanced Statistics)
  • Linear Mixed Models (PASW Advanced Statistics)
  • Cox Regressions (PASW Advanced Statistics)

PASW Direct Marketing

SPSS PASW Direct Marketing is a new model specifically designed to make statistical procedures that can be used to better understand customers more accessible and easier to use.  This is accomplished by cleanly packaging six different procedures/techniques that might be accomplished through other means but that are unlikely to ever be done simply because they are so complex. It appears here that SPSS has made it fairly easy. These procedures are:

• RFM Analysis allows users to score and rank customers by analyzing a combination of how recently they made their last purchase, how frequently they purchase, and how much they spend. The results can then be analyzed and output in a variety of ways.
• Cluster Analysis allows you to group your customers/data into different market segments which can be used to improved your targeting. It includes an interactive model viewer that allows the user to visualize and play with the results in real time.
• Control Package Test allows you to utilize the results of marketing campaigns to figure out which of your customers are most likely to respond to similar campaigns.
• Propensity to Purchase Analysis lets you determine which of your customers are the “lowest hanging fruit” so you can minimize your marketing costs.
• Postal Code Analysis helps you determine best locations for brick-and-mortar stores and other geographic sensitive decisions.

PASW Statistics Developer

PASW Statistical Developer is a new module that gives PASW Statistics users access to thousands of free R algorithms and packages using an interact that is “indistinguishable” from PASW Statistic’s built-in dialogs.

Non-Parametric Testing

Offers more non-parametric testing, which is the ability to make multiple comparisons with non-normal (that is, non-bell shaped) data.

A few links with more information:

• Press Release for SPSS PASW Statistics 18.0
• PDF Detailing PASW Bootstrapping Module
• PDF Detailing PASW Direct Marketing Module
• Details of PASW Statistics Developer (R & Python)

PSAW Statistics 18 will be released on August 17, 2009. SPSS will be hosting an online event in which they intend to go through all of the new features. Sign up here.

In other news, I came across a particularly good article on the web site of the National Council on Public Polls by Sheldon Gawiser and G. Evans Witt entitled, “20 Questions a Journalist Should Ask About Poll Results.” Im not a journalist — but what I find particularly helpful about this article is that it helps me understand the way other people are viewing (or should be viewing) my research and gives me some great things to think about when I’m putting together my own projects.

Some of the key questions include:

• Who did the poll?
• Who paid for the poll and why was it done?
• How many people were interviewed?
• How were those people chosen?
• What group were those people chosen from?
• Are the results based on everyone who was interviewed?
• How were the interviews conducted
• What is the sampling error?

Etc, etc. There are 20 questions in all, and each question is followed by a detailed answer. To read the article, go to the NCPP web site.

Before sending me the note telling me just how wrong I am, record the following:

As I figure it, there are four different kinds of security as it applies to online surveys. The first kind of security is what we’ll refer to here as back-end security. This basically means that strangers can’t simply log into the survey software server and start making changes to your survey, or download your data, or do all sorts of other things which really ought to be protected by a good password. Most survey systems do a pretty good job with this, assigning separate accounts, usernames, and passwords to each survey respondent. Some even take it to the next level, by allowing account administrators to set different permission levels for each user on the account, thereby ensuring that people who are meant to have access to your survey can’t “accidentally” go and make changes.

The second kind of survey security is where only authorized people are allowed to take your survey. Some implement this feature by giving you one respondent password that you can share will all of your respondents, or some systems even allow you to give every respondent their own password (or access key) which not only keeps out the riff raff, but also makes sure that each respondent can take the survey only once. This type of security not only ensure that only the “right” people take your survey, but also that the wrong people can’t get access to the content of the survey.

I’ve seen this kind of survey software security implemented well, but I’ve also seen it implemented poorly. Some systems, instead of assigning each survey an ID made up of a random collection of letters and numbers, use a sequential, easy to guess series. Which means that it really isn’t all that hard to view (and possibly edit!) surveys written by other clients of the survey system. So watch out for that.

The third kind of security about the connection between the respondent’s computer and the survey software server. Is it secure (look for the https: in the URL). Frankly, unless you’re doing super secret stuff and your concerned that hackers and government agents are trying to listen in on your respondents, I’m not sure it is something you need to worry too much about. But if you are afraid of industrial espionage or that someone is going to tap into your wireless signal, then you may want to take the precaution of choosing a system that allows for encrypted connections.

It’s easy enough for a well thought out survey software program to provide good security surrounding these first three measures. Where they get into trouble — and where you get into trouble if you believe them — is when they start to protect your survey content.

The fourth kind of survey software security is content protection, which basically means keeping your untrustworthy respondents from copying the top secret images and product descriptions that you include in your survey onto public web sites or your competitor’s email account.

I know, I know…there are survey systems that feature technologies that make it harder to copy images or capture video. This can be done using javascript, or by doing some crazy encryption to the feed. One could even go so far as taking over the computer so that nothing else works except the survey.

But I have a camera. And I have a video camera. And if you don’t completely lock down my computer, I have some really good screen capture software that lets me capture both audio, images and pictures. If I want to capture your survey content, believe you me, I will capture your survey content.

And then I can do pretty much anything I want with it, can’t I? No doubt 600-1000 people took that confidential survey of yours, and how are you going to know which one of them posted it to the anonymous message posting board using an anonymous IP address?

I suppose it is theoreticaly possible to embed a visual identifier into each image (a watermark of sorts) so you can trace the image back to whoever posted it online, but I’ve never heard of technologies built into survey software (if it does exist please let me know — I’ll report it here!).

There are also ways you can reduce the risk. You can, for example, use a pre-screened panel made up of people you trust. Employees, for example. Another approach is to threaten your respondents with legal action (although most will figure out pretty quickly that you have no way of identifying them). You could also threaten to stop doing online surveys if they leak your secret.

But the best way to ensure that your super secret new business idea doesn’t leak out onto the Internet is not to do online research (there, I said it). If your entire business model is based around keeping something confidential, do not put it in an online environment in front of strangers. Period. And don’t trust any research firm that says they have a foolproof method of keeping your images, video and other content safe. If they do, just pull out your camera or video camera and press record.

I’ll give you an example: We once conducted a survey about a concert series we host each spring to try to find out what kind of music our guests wanted to hear. When we go the results back, we were surprised to find that, more than anything else, our visitors wanted to listen to 80s music. By a big margin too.

Fine by me — there is a lot of 80s music that I like. But it seemed a little funny. So we ran a few cross-tabs, looked at some other variables, and found that a disproportionate number of our respondents were between 35-44 years old. Given that research has shown that people tend to be attached to the music they listened to as teenagers, is it so surprising that 80s music was so popular among a bunch of 35-44 year olds? After weighting the data to match attendance at the event, other forms of music, including pop, rock, and a couple of other genres jumped to the front.

Imagine if we had just sent our survey to a bunch of 15-18 year olds. Or if we had sent it to women over 65 years old. Or if we had just sent it to respondents in Mexico.

First rule of survey research: make sure your survey people who represent your customers.

Second role of survey research: make sure you survey them in the right proportion to your customer base.

I’m guessing the the reason for these two rules is fairly straightforward. If you don’t survey the right people in the right proportion, how will you know if your results reflect the behavior and attitudes of all of your customers (or the specific group of customers you are trying to understand)? If the people you survey aren’t representative of the people you’re making decisions about, there is very much the possibility that in conducting your survey you could be doing more harm than good. Its as if, in the example above, we got the results of our survey and then went about featuring 80s bands at all of our concerts. Nobody would come. We’d lose tons of money. And we’d assume that the research must have been flawed.

There are three important steps to having the right survey respondents take your survey. Step #1 is figuring out who the right survey respondents are. Step #2 is finding those people. Step #3 is actually convincing the right people to take your survey. The remainder of this post is about Step #1.

If you want to be technical about it, step #1 could also be called “setting your quotas.” It is the process of making sure in advance that you select people in the right proportions (that is, 50% male, 50% female; or 60% male and 40% female) to best represent the people you want to understand.

If You Want To Survey The World

Maybe you just want to learn more about people living in a particular city, or in a particular state, or even in a particular country.

This information is actually easier to come by than information about your customers, and it may even be possible that the information is available for free. The census bureau offers a great deal of information which can be broken down in a number of different ways, and can be narrowed down to very specific regions. You can find information about age, income, how many cars people in a particular region have, and lot more. Other countries offer similar information.

You’ll find that you have a lot of data to choose from. What you need to do at this point is determine which of it is relevant. For example, if you’ve created a new kind of dental implant, you may not want to spend much time collecting demographic information about people under the age of 40. Conversely, it may turn out that it doesn’t matter how old your respondent is (because you have a product that is equally useful to everyone of every age) but it is really important to survey only women. Make these decisions as you are pulling the data so that you can make sure your “quotas” are in the right proportions.

If You’re Trying To Survey YOUR Customers

If you don’t know the demographic and behavioral makeup of your response base, you may find yourself a little challenged with step #1. After all, if you’ve never surveyed your customers before, how are you going to know who you should survey?

When I used to work in the marketing department of a summer-stock theater, we used to go into the parking lot and write down the states on everyone’s license plates. Pretty crude. But if a substantial number of your customers pay cash and you don’t have a way to know where they are from, it can be a reasonably effective method for understanding where your customers come from.

If you’re an online business, or a business that has invoices that contain content information for your customers, then you can probably get some useful information right from your database. You can examine the breakdown by gender and figure out where they live. You can tell, for example, whether or not all of your customers come from a specific geographic area (indicating that you should only survey people from that area) and even how frequently they visit you (if you’re trying to find out about your repeat customers, it may not be appropriate to survey people who haven’t been visiting you for long).

Theme parks and other out-of-home entertainment destinations periodically conduct something called a “Point of Origin” survey. This is a survey, conducted of randomly selected guests as they enter the facility, is used to establish a baseline understanding of where people are coming from, how old they are, gender, ethnicity, and other information.  Because it is a random survey of customers, and theoretically reflects all of your customers, it is a valid way of understanding who your customers are — which means that you can use it to make such statements as “25% of my customers are between 25-35 years old” and then use it to make sure that approximately 25% of your respondents fall within that age range.

A Point of Origin survey doesn’t have to be long. And it isn’t hard to conduct. All you need to do is have an employee stand just inside the front door of your store and ask people to take a short, anonymous survey before they start shopping. Alternatively, you could conduct the survey at the cash register, which would give you the added capability of know who purchased and who didn’t.

Remember — keep your Point of Origin survey short so that it won’t create any hardship for your paying customers. Let them know why you’re doing the survey (you’re trying to learn just a little bit more about the people who are your customers) and perhaps offer an incentive for participating (chance to win a pair of stereo speakers, or something that you actually sell).

Once you’ve conducted your Point of Origin survey, you’ll have a good basis for determining who should participate (and in what proportion) in your other surveys.

What If There Is No Easy Way To Determine Your Quotas

Sometimes you want to survey someone other than you customers. Maybe you’re creating a business plan and don’t yet have any customers. Or maybe there is no easy to way to survey your customers. Or maybe there isn’t time to survey your customers.

When you’re in a situation like this, sometimes the best option is to make an educated guess. That is, watch the customers walking in the front door. If it looks like there are more women than men coming in, then maybe you survey 70% women and 30% men. If it looks like you have a lot of customers with kids, maybe you say that you’re going to survey 60% families with kids and 40% families without kids.

What is important, when you’re doing things this way, is that you make sure that you’re surveying enough people from each group that you want to be able to describe to make your results meaningful. For example, your customers may be mostly women with just a few men and you might decide to survey 95% women and only 5% men. While this will probably give you good overall results, you need to ask yourself up front whether or not you’re going to want to look at the results from the men separately from the women (perhaps you want to know why the men are coming into your store).

To effectively do this, you need to collect at least enough male responses so that you can analyze them properly (do you really want to launch a new men’s product line based on what 10 men had to say?).

In my experience, as a rule of thumb, any time I want to be able to summarize the results from a particular subgroup of respondents, I try to make sure that I have a minimum of 50 responses in that group (although, to be fair, I should say that my average sample size for most projects is about 1,000 responses so collecting 50 from a particular subgroup isn’t much of a challenge for me). If you’re only collecting a few hundred responses you may feel comfortable going as low as 25, but you probably don’t want to go lower than that (and even then you’re going to have to watch your margin of error).

The point of all this is that even if you guess, as long as you have enough of a sample from each of the segments (that is, as long as you have enough male and female respondents) you should be able to compensate if you later find that you’ve collected too many responses from one group or too few responses from another group.

Guessing isn’t optimal, but when it comes down to it, its better than not having any quotas at all, and 90% of the time it is going to get you into the right ball park.

Establish the Right Quotas or Get the Wrong Answers

Remarkably, none of this has anything to do with writing the questions, programming the survey, or analyzing the results. But it is extremely important. If you don’t survey the right group of people you are, 6 times out of 10, going to find that you don’t get the right answer — not because you didn’t ask the right questions, but because you didn’t ask the right people for the responses.

• List of the new features in SPSS Statistics 17
• Home page of SPSS Statistics 17.0 (on the SPSS web site)

SPSS, I really did try. I wanted to like it. I’m one of those people who can’t stand to be using old versions of software — I always want to have the latest features (admittedly, I’m still using Office 2003, but that’s mostly a work thing), and so I always upgrade as fast as I possibly can. And when I received SPSS 16 in the mail last fall I did rush to install it — but after only a few hours of trying to use it, I gave up and for the rest of 2007 and most of 2008 went back to using SPSS 15 (as far as I could tell there weren’t that many differences between the two versions — at least for the features I used).

But I couldn’t very well not upgrade after two “major” version releases of SPSS. I couldn’t suffer the embarrassment of having to tell people that I was still using a version of the software that was now almost 3 years old. So I sat down, installed the software, and even went so far as to adjust the options so that the settings would be just the way I liked them.

I should admit right off the bat that I’m not much of a statistician. There are a lot of features in SPSS that I don’t use all that regularly. I mean, I use just about everything on the “Data” menu and the “Transform” menu on a regular basis, but I don’t use a lot of the fancy options on the “Analyze” menu (incidentally, did you notice I keep saying “Menu” — I’m not much of a syntax geek — VB.NET, SQL, and Actionscript are enough for me on the computer language front). So if all of the magic and wonder of SSPS Statistics 17 is in the SPSS syntax, or in the Python, or in whatever fancy programming options they’ve introduced, I won’t notice them.

What I do notice is the fact that the Java interface feels funny. When I try to select SPSS 17 menu items on my admittedly high resolution screen, I miss. When I select cells, they don’t react in the same way as all of my regular Windows applications. When I click drop down menus, they look weird. Frankly, nothing is really in the right place, or the right size, or organized in a way I like it.

And yes, I did customize the SPSS 17 in the “Options” menu to look as Windowsy as I could. I was hopeful for a little bit when I hit “Accept” and the window changed colors to look less Java-like and more like the regular SPSS windows I’m used to seeing.

But then I tried to run Custom Tables 17. Which, besides looking weird, and besides not offering ANY new features from what I’m used to seeing in SPSS Custom Tables 15, has funny looking output.

Actually, I should take that back. SPSS Custom Tables 17 does have one new feature that I am absolutely not crazy about. After you make your table and try to copy the results into Excel (which is where I make all of my charts — call me old-school) the results now retain their SPSS Output formatting. That is, in the past, when I would copy data from SPSS to Excel, only the data itself would be copied. But now when I copy the data, it brings along all of the formatting, including the percentage signs (which tends to mess with how Excel deals with the data).

So to use SPSS Statistics 17 I either need to change the way I work with data in Excel, or I need convert it all back to what I’m used to.

The thing is, I can’t really find any new features in this new version of SPSS that I actually need. For example, for my simple marketing research purposes I don’t care about the new algorithms, or the speed improvements. I don’t have the Missing Values module, so I don’t care about the new multiple imputation. I don’t use syntax, so I don’t care about the updates to the syntax editor, and I don’t care all too much about the new EZ RFM module. I’m also never going to integrate third party applications.

Improved SPSS 17 integration with Microsoft Office ought to be a plus, but if that simply means that the software is going to format my data against my will (as described above) I’m not impressed.

So for the meantime I am sticking with my beat up old copy of SPSS 15. SPSS Statistics 17 can stay in its box, right next to the box for SPSS 16, and maybe I can look forward to either (a) exciting new features in SPSS Statistics 18.0 that will make me feel compelled to shift to the new version (give me folders to help me organize my variables and I’ll shift for sure!) or (b) maybe SPSS will give up on this Java interface and go back to the Windows interface me and my colleagues at work have come to love.

Qualtrics: (internet survey software) has a fantastic variety of question types with a really easy to use survey development system. Also features advanced survey branching capabilities including looping, random blocks of questions and more.

SurveyGizmo: (internet survey software) Also extremely easy to use survey package with a lot of features. Probably the most “Web 2.0″ survey system out their. Friendly, irreverant, with plenty of features for both the advanced user who wants to control every element of the survey experience and the analyst who just wants to get the project done. Has good e-mail invite capabilities, basic analysis tools. Plus, the free account has more features than any other “free” accounts I’ve seen.

SurveyZ: (internet survey software) This is the “light” version of Qualtrics, great for academic surveys. The survey development system isn’t anywhere near as pretty as the Qualtrics engine, but it offers a lot of advanced capabilities and question types. If you’re not doing a ton of surveys and you don’t want to spend a crazy amount of money, take a look at this package.

Digivey Survey Suite: (kiosk survey software) If you’re looking to do a survey in your store on a computer or kiosk, this software is absolutely for you. It is super easy to program, allows you to create really great looking surveys, and is fairly indestructable in the field (that is, respondents have a hard time breaking it). Great company too willing to work with you to make sure you have what you need. I’ve been a customer for quite a while now.

Techneos Entryware Software: (handheld survey software) If you’re trying to do surveys on handhelds, then I absolutely have to recommend Techneos Survey Software. I’ll admit I don’t love their pricing model, but I’m stuck because no handheld based survey platform offers as many features and is as easy to use — and as customizable — as the Techneos Entryware platform. I’ve been a customer of this group for over 5 years.

Detailed reviews of all these packages can be found on this site, including everything I liked and didn’t like about each of them. Many more reviews are coming shortly. Thanks for reading!

• Sources of Data (in-person surveys, online survey, etc)
• Import Data Formats (SPSS data files, Excel, CSV, database)
• Average number of variables in the file
• Average number of rows in the file
• Other SPSS products used in conjunction with SPSS Statistics 17
• Menu/Dialog use vs. Syntax
• Use of advance programming features (Python, R, .Net)

Last question offers a list of possible new features for SPSS Statistics 18.0:

• Additional enhancements to programmability
• Improvements to charts and graphs
• Better tools for automatic data preparation (auto transformations)
• Continued development of the customer dialog builder
• Additional enhancements to the Syntax Editor
• Campaign/Direct Marketing Tools (e.g., customer lifetime value)
• Enhanced integration with other SPSS applications
• Updates to existing procedures (e.g., add updated analytics & options)
• Continued enhancements to multiple imputation procedure
• Interactive tools for viewing, exploring, and manipulating output from predictive models

You are also given the opportunity to sign up to become a beta tester of SPSS Statistics 18, as well as the opportunity to discuss your answers with SPSS.

Personally, as someone who deals with literally hundreds of variables at a time, I would really like it if SPSS 18 allows you to (a) color code your variables and (b) put variables into folders (preferably with the ability to put folders inside folders).

I would also love SPSS Statistics 18 to have the ability to easily make a copy of a variable complete with all of the data. You know — just copy a variable on the “Variable View” page and paste it to the bottom as a new variable with everything exactly the same (except maybe that the variable name now has the word “Copy” appended to the end.

I would also like SPSS 18 to get out of Java and go back to the traditional Windows interface. I think it is fair to say that while everyone in my department has SPSS 16 installed, we all prefer to use SPSS 15 because the interface isn’t as weird feeling (we’re still waiting for SPSS 17 to arrive).

If you’d like to participate in the development of SPSS 18, take the survey.

• http://spss-voc.com/mrIWeb/mrIWeb.dll?I.Project=STATISTICS18CUST

Karen is the founder and president of The Analysis Factor, a professional statistical consulting firm. Karen holds advanced degrees in both applied statistics and social psychology. She was a professional statistical consultant at Cornell University for seven years before founding her own company, and has taught statistics at the University of California Santa Barbara and Santa Barbara City College. She has also co-written an introductory statistics textbook called Data Analysis with SPSS.

http://www.analysisfactor.com/statchat/

SPSS 17.0 new features will include:

• Various new algorithms, plus improved speed and performance with additional multithreaded procedures.
• New multiple imputation procedure in SPSS Missing Values that helps you more easily compelte datasets for more reliable analysis
• Updated syntax editor that makes it easier to create, test and correct syntax
• Improved integration with Microsoft Office
• New SPSS EZ RFM (Recency, Frequency, Monetary value) module to help analysts identify top customers and clients
• Ability to integrate third-party applications, procedures, and graphics packages created in R (through plug-ins)
• Enhanced administrative tools to improve IT configuration of software.

Also to be released is a new SPSS EZ RFM module, designed to help marketers use recency, frequency, and monetary value analysis to find their most valuable customers.

Key reasons offered by “Discover Secrets Your Spreadsheet Can’t Tell You”:

• Easy access to descriptive statistics and frequencies: True. while you can do descriptives in Excel using some of the built-in functions and the data analysis add-in, it is a lot easier and faster in SPSS.
• Wider variety of charts & graphs: True, although I tend to find Excel easier to manage
• Better, more flexible pivot tables Sortof true. That is, true if you have SPSS Tables. If you don’t, then in my opinion Excel pivot tables are easier to work with. SPSS Tables, on the other hand, is extremely easy to use and lets you do a lot more things that you can’t do with Excel Pivot Tables.
• Full set of statistical tests: True. While it is definitely possible to run statistical tests in Microsoft Excel, they’re much harder to find and work with compared to SPSS, where they pretty much come “free” with every function you run.
• Easy to run similar reports and graphics for subsets: True. Using the “Split” function in SPSS, it is relatively easy to create tables and charts for subsets without doing any extra work. Or you can create syntax (SPSS’s macro language) that lets you reuse your tables and codes over and over again.
• Labels instead of codes in your reports: I love this feature. Just because your survey software makes Male=1 and Female=2 doesn’t mean you want to see lots of 1s and 2s in your reports. And while it isn’t difficult to use search/replace in Excel to change all of your 1s to Male and your 2s to Female, SPSS lets you keep your values intact.
• Accurate results when some data is missing: Sortof true. For this item, they point to the benefits you get from using the SPSS Missing Value Analysis add-on module (an extra $800 or so). This tells you whether the questions that were skipped by your respondents will impact your analysis, and will even estimate what these values should have been. Obviously Excel can’t do anything like that, but keep in mind you need to buy the extra module to get it to work.
• Helps you spot data-entry errors or unusual data points: Certainly SPSS can help with this one, but I think you can get these types of results pretty easily in Excel.
• Easy import functions: I’m not sure that I completely agree with this one. It is true that it is easy to bring in text files. And they do provide functionality to bring in ODBC databases, including Excel spreadhsheets, Access tables and SPSS databases. But the interface for doing so is a little funky and the experience isn’t as clean or smooth as it is with Excel.
• Unlimited rows: This point describes how SPSS can handle an unlimited number of rows while Excel can only handle 65,000. Microsoft Excel 2007 can handle unlimited rows too, but SPSS’s assertion may have been true when the article was published.
• Using SPSS saves time and increases productivity: I suppose that really depends on what it is that you’re trying to do. There are a lot of analysis that I find easier to do in Excel. But certainly if you’re doing statistical analysis it is easier and faster in SPSS.
• SPSS makes it easy to understand statistical results. SPSS has added a lot of extra help files and tutorials that explain how you can/should interpret a lot of the statistical jargon that the software spits out. Excel obviously does not.

A few reasons why I still do a lot of stuff in Excel:

• For most people, the learning curve is much less steep with Excel: Learning SPSS was initially an unpleasant experience. It has a lot of options that don’t make sense until after you’ve spent a lot of time with the program. Once you’ve learned the software you’ll be amazed that you ever lived without it (or some other data analysis package) but until then you’ll spend a lot of time cursing it.
• It’s expensive. Especially if you already have Excel. Expect to spend over $1,700 for a copy.
• Charts are easier to manage/control in Excel: In my opinion. While SPSS has a lot of neat charting features, they aren’t as dynamic as Excel’s chart functionality — that is, when creating a presentation, I often need to go back in and tweak the charts or rearrange the data or rearrange the bars. In Excel, this is as easy as editing the underlying spreadsheet, which would automatically update the Powerpoint. In SPSS, you have to recreate the chart and recopy it into the presentation.
• More flexible use of functions: Excel has a lot more functions than SPSS and gives you more flexibility in how you use them.

Read “Discover Secrets your Spreadsheet Can’t Tell You”

Let’s first acknowledge that when it comes to conducting any survey, the most important element — the most value adding component is the respondent (not the survey software). Most people only have access to a limited number of respondents AND chances are they are only going to get one opportunity to collect the information they need.

Which is why we as researchers tend to want to ask as many questions as we can of each respondent who we manage to convince to participate in our survey. Ask now, or you may never get another chance. Or you’ll have to pay for a whole new panel. Or you’ll have to squeeze, squeeze, squeeze the information out of the data you were able to collect the first time. And squeezed data is never pretty.

The trouble is, when you make the survey too long respondents drop out.  I remember a couple of years ago participating in a phone survey. The survey went on, and on, and on and every time we got to the end of topic, I thought it would be over. But it never ended. I eventually hung up.

So it is a balance. And capturing that balance can be challenging. I’ve personally come up with four key considerations that more than anything else dictate how long a survey can be.

1) What does the respondent get for taking the survey? The bigger the incentive, the longer the survey can be. But keep in mind, unless it is a guaranteed incentive (finish the survey and we’ll send you a T-Shirt) you’re only going to get so far. For example, a sweepstakes in which the respondent can win $10,000 will do better than a sweepstakes for $1,000 but not necessarily 10 times better and not nearly as well as a survey in which the respondent automatically receives $20.

2) How interesting in the survey subject to the respondent? I will get the same dropout rate for a long survey about new rollercoasters than I will for a relatively short survey about toothpaste. People like to take surveys about subjects that they find interesting and will stay in a survey a lot longer if it is one that lets them think or discuss matters that are of interest to them.

3) How much fun is it to complete the survey? Believe it or not, a survey doesn’t have to be a boring, unpleasant experience. Questions can be written in an interesting or witty way, and response options can be configured in such a way that that the survey is fun for the respondent to take. For example, it is much more fun to click on pictures of things than on words — and it is more fun to drag a list of items into a specific order than it is to rank items using drop down boxes. The more fun your survey is to take, the longer it can be.

4) What impact will completing the survey have on the respondent? If I’m conducing a survey in my neighborhood to decide whether or not the lot on the corner is turned into a new park or a junk yard, I can feel confident that I’ll get a good response rate from the people living on my street no matter how long the survey is. People will spend more time on a survey that they know impacts them in some way — much more than they’ll spend on a survey that they know only benefits you.

By thinking about how these four factors come together within the context of your survey, you can make a determination regarding how you want to present your survey and whether or not you need to offer a better incentive or — if all else fails — reconsider the length of your survey.

Three main

If you’re like most people, you probably only have a limited number of potential respondents for your survey. You may have a list of your club members, or a list of fellow students, or even a list of customers you’ve painstakenly collected over a period of months if not years. You’re all set to send them a survey, and figure you’re all set to go — after all, your survey software provider has made it a snap to upload your e-mail addresses and send out personalized invitations on your behalf. Right?

Stop right there! How do you know that e-mails sent by your survey software company aren’t going to end up in the bulk mail folder or spam folder of your recipients?

If there is one thing I’ve become painfully aware of as I’ve looked at the various survey software packages out there, it is that very few survey systems put much effort into how they manage the e-mail they are sending. And since these systems send out such a high volume of e-mail, many of them are blacklisted by internet service providers or blocked using standard spam filters.

Here are some things to ask about if you plan to use your survey software provider to send e-mail on your behalf or manage your e-mail list.

See if your survey software provider uses certified e-mail

There are several companies that offer something called e-mail certification. These companies work closely with ISPs and companies to guarantee that anyone using their service will automatically be “trusted” by the e-mail providers. Getting certified is somewhat of a complicated process, but in my experience using SenderScore Certified, it is worth it since just about every e-mail you send out makes it to its destination and doesn’t end up in the spam box.

There are four big companies that offer e-mail certification. These are Habeas, SenderScore Certified, Goodmail, and SuretyMail.

Ordinarily, you as a survey system client wouldn’t need to work directly with any of the companies I just named — optimally, it would be the survey system itself who bonds their e-mail. If your company is bonded separately, then it would behoove you to send your e-mail through your own e-mail system and not through your survey software provider.

Incidentally, the way these certified e-mail providers work is that the client pays an annual fee or a per-email fee for each message that is sent (per recipient). If any of the e-mails are reported as spam (in certain ways) the sender is charged a penalty. Back in the day when I was using Sender Score Certified, every e-mail that was reported as spam cost us $20.

Now, to be fair — survey software providers don’t have a lot of control over what e-mails their clients are uploaded and sending to so it might be very risky for them to participate in such a program. Plus these programs cost several thousand dollars a year to belong to. However, if you’re serious about your e-mail surveys, or if you make your living this way, you may want to investigate sending out your invitations through a separate e-mail service that does offer certified e-mail.

What relationships does the survey vendor have with the ISPs?

Even if a survey software provider doesn’t certify their mail, they can at least develop relationships with the major ISPs such as Google, Hotmail, Yahoo, etc. to help ensure that as many e-mails get through as possible. Getting “white listed” among the major e-mail systems isn’t a terribly complicated thing to do, and it doesn’t cost very much. It will also help to ensure that the vast majority of e-mails make it to their destination.

Is the survey software provider on any blacklists? And is there a way to tell how likely my invitations are to go through?

There are tools on the internet that make it possible for you to get a sense of whether or not your invitations sent through an survey software provider are likely to go through. These tools can also help you know whether or not your survey software provider is on any blacklists. 

An e-mail blacklist is a service that compiles lists of domains and IP addresses that regularly send out spam. These lists are used by companies and ISPs to filter mail before it even makes it to your recipient’s e-mail software. There are dozens of these lists, and once a company is on one it is very difficult to get off.

Here’s what I would do to check on the deliverability of e-mails from a survey software provider. I use a free tool called SenderScore.

1. Go to http://www.senderscore.org

2. Punch in the domain name of the survey system. (Try “zoomerang.com”). You’ll be shown some basic information about how the provider sends out e-mail, such as whether or not they even send out their own e-mail (if they have an MX record), whether or not they have an SPF record (good for deliverability), and whether or not they have a Sender ID record (also good for deliverability).

 You can see on the list above that Zoomerang sends out their e-mail using two different domains — “mailsender.zoomerang.com” and “mailsender2.zoomerang.com.’ You’ll also note that these are both clickable links — so try clicking them. The first link, you’ll notice, indicates absolute no volume of mail being sent, while the second link (“mailsender2.zoomerang.com”) has a very high volume of e-mail. Note the clickable IP address on the left side. Click it.

The page that appears (specifically, the SenderScore information for 63.251.58.111) contains a lot of information that you can use to decide whether or not this survey software provider can get your e-mails through.

Starting from the top: the Sender Score is a summary number that you can use as a measure of this IP address’s “reputation” on the Internet. This number can range from 0 to 100. Let’s just say that if you sent out good e-mail from your own server all of the time you shouldn’t have a problem keeping this number above 90. If you send out to a lot of big e-mail lists where people come and go a lot, you could be at around 60-70. If you send out a lot of e-mail from clients who upload their own lists (like a survey software company, a 40 isn’t surprising.

Some other survey companies and their ratings:

• InsightExpress.com: 98
• ConfirmIt: 80
• SurveyGizmo: 80 [not 25]
• SurveyMonkey: 75
• Qualtrics: 50

Working down the page, you’ll see that next to “blacklists” there is one listing for blackholes.five-ten-sg.com, a fairly aggressive blacklist that lists e-mail senders for many reasons (actually, if you do a search on google you’ll see that this particular blacklists lists so many people that most e-mail servers are not encouraged to takes the list seriously).

Next to “Sender Score Certified” it says no, indicating that Zoomerang does nog subscribe to this particular e-mail certification program.

Under deliverability it says that about 91% of the e-mail sent through this system gets through, and that e-mail sent through this system has a “medium” risk of deliverability issues. Medium isn’t bad and isn’t something you should be worried about. You should definitely be worried if the system you are using says “High.”

Reputiation measures cover a variety of facotrs that ultimatley contribute to the overall score. Higher numbers are always better.

What if I want to improve my deliverability?

Even if you don’t want to switch survey software providers in order to send out your e-mails, you can still get good deliverability by sending your e-mails through a separate service that focuses on sending out e-mails such as CheetahMail, Constant Contact, ExactTarget, Lyris and several others.