4/262010

The danger of self-service surveys

Imagine walking into a store and seeing a box on the counter with a sign that reads, “Surveys – take one.”  Most of us would shake our head, knowing that the typical customer won’t participate.  The ones that would are most likely to be friends of the owner or very upset, looking for a place to vent.  Virtually none of us would assume that the results would reflect the store’s entire customer base.

Yet, every day, many intelligent people with company websites use an electronic version of that box on the counter.  It has become so easy to generate a survey and link it to one’s website that we aren’t even surprised to see a pop-up or a request to share our opinions.

Recently, a financial institution hired our company to study customers living near their branches.  This client had used web-based surveys in the past and had made decisions based on the findings.   We gathered data on the phone, and did not let people self-select themselves into the study.   Having access to their web studies, we compared our phone responses with responses taken from the website.

The key is that self-selected website surveys reached only a subset of the financial institution’s customers and was NOT a reflection of all their customers.  Those who participated through the internet had key differences compared to those of the carefully generated random sample contacted by phone.  While website surveys may generate some interesting results, marketing decisions should be based on sound methodologies that produce an accurate picture of all customers!

Some might say their situation is different since their company doesn’t have brick-and-mortar locations.  But it is still like that box on the counter.  Those who grab the survey likely aren’t the same as those who would ignore the box.

In our business we sometimes hear, “It’s not perfect, but it’s better than no feedback at all.”  Think again.  Making decisions based on bad information is rarely a good idea.

Care to comment?