OX2 Web Analytics blog

Quantifying emotions, particularly those associated with a brand for marketing purposes, has always been a challenging task. If you’re considering adding qualitative survey data to your web analytics reporting, whether in dashboards or other periodic reports to management, this should help you evaluate the relative value you might expect to receive from using the Net Promoter Score, 4Q from iPerceptions and Avinash Kaushik, Foresee’s ACSI, or Gallup’s CE11.

At its best, qualitative research can provide deep insight into customer motives and behavior far beyond what the raw web analytics data can tell you. Open-ended questions allow site visitors to speak freely about their likes, dislikes, frustrations, and successes in navigating your site to achieve their session goals. However, this is often difficult to condense and summarize, and the few who speak up may not be representative of the target audience you are attempting to reach. In any case, you want to collect the data in a way that makes analysis possible with limited staff resources (who has time to read or process a thousand comments a day?), and moreover, create insights that are actionable by management.

For a first tool, consider that your goal should be to show immediate value to management that collecting a small amount of qualitative survey data can yield non-obvious insights. In terms of return on investment, which free or inexpensive tools will give not a useless number, but actionable leads? This may be the first step down a path towards the use of a more powerful tool, one which can gather greater numbers of responses from customized questions, focused on specific campaigns, products, or services, and used to supplement a robust web analytics process already in place.

Having said that, let’s take a look at the four tools, keeping in mind that some flavor of “roll your own” may be the most flexible, if you have the IT resources to design and implement it on your site, and the statistical and research skills within your team to effectively process and understand the data you gather.

One overly simplified measure of customer satisfaction is called the Net Promoter Score, or NPS, from Fredrich Reichheld’s research published in the Harvard Business Review back in 2003. Simply finding the difference in percentages of people who would recommend your brand, and those who would not, is seductively compelling for those organizations too frenzied to listen to a complex explanation of their customers’ true behavior. While nobody believes the complexity of their decisions can be reduced to a simple number, some marketers apparently believe they can reduce their entire customer base to a single number. Even for a management dashboard, this would only serve to disguise complex issues which defy reduction, and encourage a glossing over of serious analysis.

The strongest benefit this has is that it is easy to measure, and easy to explain to management. However, as there is little true value to be derived, it isn’t even so useful as a “gateway drug” to encourage investment in a more sophisticated measurement tool.

The most recent entry into the field is by our friends at iPerceptions and Avinash Kaushik, widely known as “4Q”. The simple and free service works as a permission-based pop-up on your web site, prompting visitors to think about whether they are satisfied, their purpose for visiting, if their task was accomplished, and if they have other feedback to offer you. These really are the fundamentals, and it’s good to see such a clear distillation of what makes a web site visit successful, without oversimplification or sacrificing open-ended feedback.

One goal of 4Q is to show the tremendous value potential in survey analysis of your web site visitors, and guide you into unlocking it further with the iPSI, the iPerceptions Satisfaction Index, where you can first ask more detailed custom questions, and also compare results against benchmarks for your industry.

By comparison, the ACSI, American Customer Satisfaction Index has served as an established benchmark for over 14 years. Wall Street analysts sometimes use ACSI benchmarks as a predictor of future revenue and growth, reasoning that if your customers are dissatisfied today, then your brand value is headed downhill. Trending in some of these metrics is thus closely watched, and some publicly held firms tout their ASCI scores to attempt to drive share price upwards. Foresee Results applies this methodology both online and offline, helping you get a better picture of your brand health among pre- and post-purchase customers, and in your target audience market overall.

While there may be some value in your firm’s ACSI scores, having them is not necessarily going to provide any actionable information. (Of course, the same is true for any metrics — analysis and insight do not magically generate themselves from even the clearest of automated reports.) Chances are good that, if you’re aware of the challenges your customer support organization is facing, then you already know how well you’re doing, and how your customers perceive you, so the primary value of this data is as a tool to convince others within your organization to justify the allocation of scarce resources.

A marketer’s goal is often not to convince customers to buy more, but rather to convince people who are not yet your customers that you have something of value for them. In other words, the bulk of marketing efforts are focused on “upper funnel” activity, driving unaided and aided awareness towards consideration and purchase. It’s often considered a customer support function to drive purchasers towards repeat purchase and customer loyalty by ensuring customer satisfaction, though anyone with a more holistic view of the customer can see this is a short-sighted mistake caused by siloing. That is, treating marketing budgets as a place to invest resources to draw new business, while treating customer support as an expense, and a cost to be minimized, rather than allowing the two efforts to reinforce a common message of customer value.

Back in 2001, Alec Appelbaum wrote in the Gallup Management Journal about the then-new 11-question “customer engagement” metric, or “CE11,” which Gallup believed to be “the most powerful predictor of customer loyalty” at the time. As many web analysts are less familiar with these, I will restate them here.

First are three questions about customer loyalty:

* Overall, how satisfied are you with [brand]?
* How likely are you to continue to choose/repurchase [brand]?
* How likely are you to recommend [brand] to a friend/associate?

Then eight more, including two about brand confidence:

* [Brand] is a name I can always trust.
* [Brand] always delivers on what they promise.

Two for integrity.

* [Brand] always treats me fairly.
* If a problem arises, I can always count on [brand] to reach a fair and satisfactory resolution.

Two on pride.

* I feel proud to be a [brand] customer.
* [Brand] always treats me with respect.

And finally, two about passion.

* [Brand] is the perfect company for people like me.
* I can’t imagine a world without [brand].

As with most survey questions, reviewing results in isolation is relatively useless, so a one-time benchmarking exercise can only be considered just that — establishing a benchmark, against which regularly repeated measurements will demonstrate relative improvement. Comparison against other firms in your industry can also be useful, but keep in mind that this data is often anonymized to the point where large and small organizations with radically different profiles must be lumped into the same category, such as “automotive” to apply both to utility trucks, sports cars, and family vehicles. Clearly, the value of fine-grained segmentation is lost in this type of analysis, so one must exercise care when drawing conclusions (or allowing your management team to draw their own conclusions without the benefit of your insight).

Interestingly enough, a later Gallup report cites customer satisfaction as a flawed measure, pointing out that emotional satisfaction is much stronger than mere rational satisfaction. For example, an emotionally satisfied customer will purchase ancillary items because they are from your brand, while a rationally satisfied customer will make distinct purchase decisions for each item, discount your brand’s halo effect on your other product lines.

As a social media strategist, I see this bridge from brand loyalty to brand evangelism often not recognized, even by firms in the enviable position of being able to cross it. They may have wonderful products and customers who would buy no other brand — yet they failed to convert them into evangelists for the brand. Granted, as a consumer, there’s a limit as to what I buy based on the recommendations of my friends, especially if I haven’t asked for their opinions. But in most circumstances, there is a tremendous opportunity for someone who has just purchased a new product to tell their friends about their experience. And if that’s a positive story, yet it’s not being told, then you as a marketer have failed to transition your brand from its natural niche into the category of lifestyle icon.

This isn’t to suggest that every brand has the potential to be a Coke, Nike, or Apple — after all, medical equipment simply doesn’t have the same cachet as a consumer product. Yet among the cognoscenti in any industry, your products and services can become better appreciated, and talked about more, if you are careful to survey your customers politely and act on the valuable feedback they share with you.

Whatever your brand, products, or services, your web site effectiveness can almost certainly be improved if you invest in listening to your visitors, and giving them a chance to voice their opinions. Are you ready to listen? We’re happy to help you evaluate these, and several other possible qualitative research tools, to fit them to your specific needs and audience. Please contact us to get started today!