Peter Leppik posted an insightful story on the VocaLabs blog about a recent trip to his local Home Depot where an apparently store sponsored signed informed customers "9 or 10 = PASS, 1 through 8 = FAIL." The sign was apparently in reference to the register tape survey customers were being asked to take when they got home.

Peter’s point is that "any time you give someone an incentive to hit a goal, you’re also creating an incentive to manipulate the metric."

What struck me as well, though, was the that it is so easy to get respondents to change their ratings based on such simple instructions. It is a good reminder that, in the end, respondents don’t care very much about the ratings they give and are happy to manipulate their responses to meet the needs of an anonymous sign posted in a store.

Read Peter’s post at VocaLabs.

I was poking around in Google today and came upon an interesting article in the October, 2000 issue of hte CustomerSat.com monthly newsletter (appropriately entitled "CustomerSat.com Connections" about the relevance of using top box ratings, bottom box ratings and mean scores.

The article argues that it is impossible to get a true understanding of your customer’s satisfaction by simply using one of the three metrics, and that the best way to really understand what is going on is to use some combination of at least two, one of which will be the mean score.

In terms of determining whether it makes more sense to look at the top box or the bottom box, it suggests examining how changes in either box correlate with the overall outcome.

What is most fascinating and useful about the article is actually its description of "Non-Linear Effect Analysis" in which it describes how customer loyalty tends to rise not linearly, but exponentially. In other words, customers who give a "9" out of 10 instead of an "8" out of 10 are exponentially more loyal.

Read the full article at CustomerSat.com
See all of the online resources offered by CustomerSat.com

eDigitalResearch today released a press release highlighting their relationship with U.K. based department store John Lewis, who is using their system to collect and report on real-time customer satisfaction related issues.

eDigitalResearch’s Satisfaction Suite collects customer purchase information from the John Lewis database and emails surveys out to customers, tying in all the purchase information with customer responses. Monthly reports are produced which can be cross-tabbed and filtered in terms of product groups or particular divisions.

John Lewis’s particular satisfaction solution uses elements of both eDigitalResearch’s eMystery Shopper program and their eCustomerOpinion program. eMysteryShopper specializes in the in-depth and structured studfy of website usablity, functionality, and customer service using a eDigitalResearch’s panel. eCustomerOpinions also provides feedback, but from directly from randomly selected web site visitors.

View the press release at PRWeb.
Learn more about John Lewis.
Learn more about eMystery Shopper.
Learn more about eCustomer Opinions.

Katie Fehrenbacher of GigaOM posted a note today about Embrace, a UK-based mobile survey startup that has recently introduced MSurvey, a service for running questionnaires via cell phones. Embrace’s mobile solution is particularly useful for conducting very short surveys such as opinion polls, customer satisfaction surveys, employee satisfaction surveys, and ad hoc market research.

To maximize response rates, Embrace can collect data through three different method:

• SMS surveys: universally accessible from all types of mobile phones, allow for reverse billing (so your users don’t have to pay to take your surveys) but can only accomodate a limited number of question types;
• Phone Web Browser: Most new cell phones have this capability; surveys can be branded, and a range of question types are available.
• Downloadable application: Survey can be downloaded to the user’s phone and the survey can be completed when the user is out of signal area (like downloading a game to your cell phone).

Data can then downloadable in the Survey Interchange Standard format or XML.

Read Katie’s article at GigaOM.
Read more at the Embrace web site.

Customer research, taken out of its appropriate marketplace context, can be extremely misleading. Consider the scenario presented by Lior Arussy of the Strativity Group who in a recent DestinationCRM article where a research firm, after conducting a study to help a client indentify key loyalty factors to build greater customer relationships, came back with a finding that the most important thing the company could to to retain customers was to "excel in invoicing."

Lior argued (and I agree with him) that from the customer’s perspective accurate, on-time invoices — like clean bathrooms or safe rides at a theme park — aren’t reasons that most customers are going to do business with you. Sure, they’re important to maintain and ultimately speak to the gestault of how people perceive your business (nobody wants inaccurate invoices) but nobody is really going to choose you over your competitor if your greatest claim to fame is that you have the most accurate billing system in the business.

Says Lior:

"The goal of customer experience is not simply to stop upsetting people, it is to delight them and maximize revenues and loyalty. It is essential that market research surveys–and the client companies they purport to help–target and measure true experiences that help competitive differentiation."

To derive insight from research takes more than just good methodology and execution — it also requires an understanding of the business that your in and enough knowledge about your customers to be able to interpret the results in such a way leads to meaningful, actionable findings. In other words, you can’t simply leave it up to your research firm to go out, do a survey, and report back with results that you can immediately integrate into your business. You also need to bring to the tables your own experience and your own knowledge of the business at every stage of the research in order to ensure that the results that you get make sense in the context of your work.

Read Lior’s article at DestinationCRM.

David Tebbut of IWRBlog (Information World Review) recently posted some interesting observations about Confirmit’s dashboard application, in which an online survey system is used to track customer attitudes in real time and report the results in an automatically updated "dashboard" application. The idea is to be able to provide useful research results instantly — as soon as they are relevant — instead of having to wait hours, weeks or days for results.

In my own experience, the greatest challenge to this type of a dashboard — which in some ways speaks to the potential to integrate customer satisfaction directly into a balanced scorecard type system in a meaningful way — is the ability to collect enough data on a regular basis to cause the "dials" on the dashboard to reflect something meaningful. 10 or 15 responses a day are simply not enough for a system that is meant  to be continuously updated.

On the other hand, there are applications where such a system might be somewhat useful and relatively easy to "keep fed." For example, if on the way out of the store there was a single question that customers could answer — either as they walked out the door or as they checked out — and if there were enough registers in the store — it might be possible to collect enough data to make the dashboard meaningful. Or maybe if there were a way to ask the question on customer cell phones as they walk out of the store (perhaps a little less realistic).

Read more about this article at IWRBlog.

Lesie Pegel, senior consultant for Walker Information, recently posted the results of a study they conducted of over 1,500 users of customer feedback representing 37 of their clients. Although Walker found that 75% of respondents thought that the collection of customer feedback was having a bottom line impact on their company’s financial performance, they went the extra step of comparing those individuals againt the companies that said they didn’t see any bottom line improvement.

Companies are more likely to see a bottom line improvement in financial results if they:

• Use the customer feedback they collect to motivate employees;
• Use the customer feedback in strategic planning;
• Seek to continuously expand the use of customer feedback;
• Dedicate resources to understanding the feedback they collect.

Other key findings from the survey:

• Customer loyalty improves when feedback is acted upon;
• A formalized follow-up process driven by the account team maximizes the value of the feedback to the organization;
• In reports, focus on key learning points instead of data points.

Read the full article at CustomerLoyalty.