Review of Qualtric’s SurveyZ
SurveyZ is an entirely web-based, self-service internet survey system developed and operated by Utah-based Qualtrics. It offers a variety of advanced capabilities, including unique survey questions, branching, looping, piping, trigger-based responses and form grading as well as an unparalleled collection of pre-written surveys that you can borrow questions from or use as the basis for your own surveys. The design and organization of the SurveyZ survey development environment isn’t particularly impressive (it looks like something out of 1998), and the design options for the surveys themselves are fairly limited — yet the design process is fairly smooth and even the most complex surveys can be created by individuals with very little experience. SurveyZ has a built in e-mail invitation system and a very limited reporting engine. All in all, if you’re looking for a survey system that can do just about anything and you don’t care so much about design or reporting, SurveyZ may have everything you need.
Survey Capabilities
SurveyZ includes all of the basic question types and adds a few extra "combo" question types (questions in which multiple basic question types are combined to form more complex elements) such as a "multiple line text" question (which allows you to ask several open-ended questions together through the use of one question, a "spreadsheet" question (which creates a grid of open-ended questions that look a little like a spreadsheet) and semantic differential questions (very much like traditional rating questions, except that you can use different bi-polar adjectives for each scale (see the sample below). Actually, SurveyZ gets a lot of credit for offering semantic differential questions at all — most other packages only offer basic Likert scales (SurveyZ does too) but there are times when a semantic differential question can be extremely useful.
Sample of SurveyZ semantic differential questions
SurveyZ also offers the ability to include matrixes where you can group together multiple questions into one grid. The functionality of these tables are somewhat limited in that you can only include one type of question in each matrix. SurveyZ also supports ranking questions and constant sum questions, which gives SurveyZ about as much variety in the question types that it makes available as most of the other packages that I’ve looked at.
SurveyZ supports some validation of fields: for example, you can require that respondents answer certain questions, and for open-ended questions you can require that responses are either textual, numeric, or even an e-mail address.
SurveyZ makes it easy to execute conjoint studies.
SurveyZ includes a few very special question wizards that help you execute some pretty advanced research studies. For example, the "Random" wizard allows you to randomize any group of questions in your survey (you can have multiple groups in each survey). The "Conjoint" wizard will create and manage a conjoint question set (either full profile, discrete-choice, or self-explicated) which helps you understand the relative utility of various attributes of products and packages. The "Parallel" wizard makes it fairly easy to have two questions appear next to each other.
Conjoin exercises appear as single questions — easy to edit.
The value of the conjoint wizard shouldn’t be underappreciated — it took me just a few moments to create a basic conjoint exercise. When I was finished, the entire exercise (which consists of several questions) were all rolled up into a neat package that I could edit or move around my survey at will.
SurveyZ supports a variety of advanced survey logic features including skip patterns, branching, piping, and looping. SurveyZ can also randomize questions, groups of questions, and answers within questions. All of these features are fairly straightforward to implement and function as well (if not better) than similarly priced packages offered by competitors.
SurveyZ’s branching setup is easy to use
I think the SurveyZ branching system is about as flexible as any I’ve seen so far — it allows you not only to skip to specific questions or groups of questions based on specific responses, but also to skip to specific questions based on pre-populated data (such as whether the respondent made a particular purchase). SurveyZ can also branch based on whether quotas for specific responses have been met — so, if you’ve already had 100 males fill out certain portions of your survey you can skip male respondent #101 to a different set of questions or even right out of the survey altogether.
SurveyZ also allows for trigger-based responses, which is a special feature I’ve seen in only a couple of packages that will allow the system to send an e-mail or redirect the respondent to another web site (or survey) based on their responses to questions. This might be particularly useful if you wanted to receive an e-mail immediately if a respondent gave you a particularly poor rating or if they indicated they wanted information about a particular type of product.
SurveyZ also includes something called "form grading" which allows you to assign point values to survey questions which can then be totaled at the end of the survey to yield a grade that is reported to the user.
Survey Design/Development
If you’re a non-researcher or simply someone who doesn’t want to spend a lot of time writing survey questions, SurveyZ might just be for you. It offers one of the largest pre-written survey libraries I’ve ever seen and features literally hundreds of surveys with thousands of pre-tested questions on topics that include customer satisfaction, human resources, product development, business to business, public/non-profit associations, academics, human interest ("pop psychology" and "personal interest") and numerous industry specific surveys. SurveyZ makes it easy to either copy and build upon an entire pre-written survey, or you can pick out and use individual questions.
With that said, I really don’t care too much for the SurveyZ design environment. It seems kind of ugly, kind of clunky, and very much like it just stepped out of a 1998 web site. Buttons and boxes are all over the place, the menus aren’t particularly well organized, the text boxes are too small, and the icons look like they came from one of those free icon web sites. I’ve seen much better, which is a shame because there are a lot of less capable survey packages out there that, simply because they look and feel more professional, might steal business away from SurveyZ.
It’ also a shame because after you get past the look, feel, and organization it really isn’t all that hard to create a survey. In fact, the process is pretty fast. All of the options for each new question you create are for the most part right where you would expect them to be (although, once again, it isn’t pretty).
SurveyZ doesn’t offer a lot of options for individuals who care about how their surveys look. You can choose from one of sixteen different color palettes, but otherwise you have relatively little control over fonts, sizes and color unless you use HTML - and even then, you have to change each question separately (so if you wanted to change your entire survey from Arial to Times Roman you would have to individually go in and edit each question and response. Ouch!
Users of the Enterprise edition can add their own style sheets which will allow you to change the overall look and feel of the survey all at once. However, other users are stuck with one basic design template (which in my opinion isn’t a very good one — there are issues with spacing, default fonts, and the overall organization of the page that I just don’t like very much.
Frankly, if I were the boss of SurveyZ, I would say that for their next major release they should hire a designer and a usability expert to revamp the overall look and feel of the program to bring it into line with what most users expect.
And if it sounds like I’m ranting, it is simply because I care — there are so many positive things to say about SurveyZ it disappoints me that the interface is so unnecessarily…unpleasant.
SurveyZ does allow you to produce both single page surveys and multi-page surveys (you can control where the page breaks go). Surveys can’t be imported from text files or MS Word documents, but you can build and maintain your own question and survey library.
As noted above, SurveyZ does allow you to embed your own HTML just about anywhere (headers, questions, answers) which means it is possible to add images and other forms of multimedia to your surveys. One note though: SurveyZ does not come with an interface that lets you upload images and other multimedia to include in your survey — so if you want to add a picture (or a video) you will need to host the images on your own web server (the Enterprise version of SurveyZ does allow for construction of a library of images).
SurveyZ will allow you to preview individual questions as you create them, or at any time you can click one button to see your entire survey previewed in a separate window.
User Experience/Response Management
The SurveyZ respondent interface is straightforward enough and users shouldn’t run into any issues. As the survey designer you have the option of including a progress meter on each page, and you can also allow your users to save their surveys at any time and then come back and finish them later (and here’s an interesting twist: the system will also let you e-mail a link to another user to finish the survey for you, which may be necessary for some kinds of complex corporate surveys where different people need to complete different sections).
The SurveyZ survey completion interface does not appear to allow for a "Previous" button that respondents can use to check/change their answers.
One nice feature is that SurveyZ can be configured to e-mail a confirmation note to the respondent after they complete the survey.
SurveyZ includes a built-in invitation system which you can use to invite respondents to take your survey, or it also provides you with a URL that you can embed on your website or in a newsletter. Even better, SurveyZ provides you with code you can install on your web site so that the survey will pop up in a separate window (either every time or once for each visitor).
SurveyZ also provides you with code you can use to embed your survey as a form in a web page on your site, or in an e-mail newsletter (using this option would undoubtedly give you more control over the look and feel of the survey, although I still think it is a lot of unnecessary work).
The trial account I used to test SurveyZ didn’t give me access to the e-mail system — however, as far as I could tell (based on the description), you can use this system to upload and manage recipient lists; upload and manage custom data fields for tracking purposes; create personalized messages for e-mailing; monitor and manage delivery status; and repeat mailings to individuals who have not yet responded to the survey.
Although you can’t set predetermined start and stop dates for your survey, you can start and stop your survey manually at any time. SurveyZ will allow you to monitor responses as they are collected (you can see how many times your survey has been viewed, started and completed) and the system will allow you to go in and view/edit/delete individual responses.
SurveyZ also includes a cookie-based "anti-ballot box stuffing" feature that allows you to deter individuals from completing your survey multiple times.
Reporting Capabilities
SurveyZ has a very basic built-in reporting engine. It allows you to generate charts and graphs, crosstabs, and frequency counts. It also allows you to filter the data that appears in your report based on complex criteria, and it allows you to export your data into a CSV/TXT file which can then be analyzed in a separate program such as Excel or SPSS.
SurveyZ has the unique ability to generate SPSS command/syntax files that can be used to help you import your data more easily into SPSS. While this is truly more assistance than most other programs provide, it does make you wonder why the publishers wouldn’t just include an SPSS export feature (which would allow you to generate and save SPSS .SAV files).
Reports, such as the sample above, can be made publicly accessible via a URL, and you have the option to protect your results from public consumption by assigning a password.
If you are looking for a system that can generate custom, automatic reports that look nice and feature a variety of different types of analysis, then SurveyZ probably is not for you.
Extensibility
SurveyZ operates as a web-based service provider only — in other words, you can’t buy the software and install it on your own servers. While this makes it difficult to directly integrate SurveyZ into your existing system, it does let you integrate your own data into the survey results (which means that you can include a customer ID in the URL which will ultimately allow you to associate specific surveys with customer accounts).
According to the web site, the SurveyZ architecture is designed to accommodate the "quick and easy" addition of new question types to meet your needs, but you should probably find out about costs before you commit.
Account Management
SurveyZ allows each account to have multiple users, and the account administrator can manage every element of of each user’s access, ranging from their permission to view/edit specific surveys to library access.
SurveyZ also allows users to share resources (such as surveys, questions, images), which is particularly useful in workgroup environments.
System Requirements/Security
From a security standpoint, one element of SurveyZ I don’t like is the fact that all of the surveys in the system (yours and everyone else’s) are sequentially identified (for example, your survey might be #54805 and the next person who creates a survey gets #54806). While internally the system is certainly free to number/identify surveys any way that make sense, using the survey "key" as an external identifier means that other users and respondents can change the number in the URL and thereby view other customer’s surveys.
Other survey systems avoid this problem by giving each survey a unique, alphanumeric public ID that is extremely difficult (if not impossible) to guess.
SurveyZ makes some effort to mitigate this problem by making it possible to password protect your survey so that only those who enter the right code can view/take your survey. However, this is not the default.
The same issue applies if you make the results of your survey publicly viewable. Unless you password protect the results, anyone with enough time on their hands can see your results by simply changing the ID in the URL.
Pricing Structure
SurveyZ offers separate pricing for companies and for academic organizations. Corporate packages range from $49 for 400 responses (12¢ per survey) to $99 for 1000 responses (10¢ per survey) to $7500 for 37,500 responses (20¢ per response). Yes, that seems a little weird — the best price is for 1000 surveys — but there are other differences between the packages that may explain the variance.
For academic users, prices range from $49 for a 500 response "student project" account to a $199, 2000 response "professor research" account to a $5000, 30000 response "department-university" license.
SurveyZ’s corporate pricing structure
Privately branded solutions are also available, and if you’re a non-profit you can even request a "survey research grant" which will give you free use of the service provided that you promote SurveyZ.
Additional Resources
The following resources provide more information about SurveyZ:
- SurveyZ Web Site
- General Product Information
- SurveyZ Enterprise Features
- SurveyZ’s Survey University
- Survey Survey Library
- Corporate Pricing and Academic Pricing
Related Categories & Tags
Internet Survey Software, Reviews Tags: advanced survey software, branching, embeddable-surveys, form-grading, free trial, Internet Survey Software, looping, piping, pricing-academic, pricing-medium-cost, Qualtrics, question-library, Reviews, sample-surveys, spss-export, SurveyZ, triggers, web-based
