How to Get the Survey Right

Without a doubt, there is an art and science to executing meaningful surveys that belie conventional wisdom. For example, most people assume that the higher the response rate, the more reliable the data. But beware: a shallow survey that draws exorbitantly high customer response rates is like scuba diving with full gear in 3 feet of water.

Why not go deeper while you have customers' attention rather than waste this opportunity just to claim more surveys?

Recently, builders have reported getting solicitations from research vendors claiming to provide 80–92 percent response rates on their customer surveys.

To help guide you in sorting through various research programs, I have provided the following tips to consider as you evaluate the merits of each:

Tip 1: Check the Number of Questions. When shopping around for a survey company, verify the number of questions on a survey before comparing response rates. Who wants an 80 percent response from a shallow survey that creates more questions than it answers? Consider the math: an 80 percent response rate for a 25-question survey yields an astonishing 70 percent less data than a 50 percent response rate for a typical size home buyer survey of 75 questions.

Tip 2: Check how response rates are calculated. One unethical trick to increase response rates is to subtract those who were unreachable.

Imagine, for example, that you have 100 closings in a given month, but 50 of them don't answer the phone call from the survey company. Some survey companies will subtract these home buyers from the total possible.

If 40 of the remaining 50 customers actually take the survey, they call this an 80 percent response rate, even though it's really only 40 percent. Reputable companies will only remove surveys that are verifiably undeliverable or unreachable.

Tip 3: Avoid lengthy phone surveys. How many of you have had a telemarketer call you during dinner? Now envision that they are representing your company and require 25 minutes. Need I say more? Phone surveys are only good for short surveys. If you try to capture more than 20 questions, you get fatigue and false positive responses, meaning people just start say "yeah" to get off the phone.

Tip 4: Remember that you get what you pay for. If someone is claiming to get more data and charging heavily discounted rates, best to look elsewhere. With referrals being the cornerstone of the home building industry, it's a serious misstep to risk your relationship with your most valuable asset just to save a buck.

Customer research is much more than just sending a survey. The quality of the survey experience, number of questions, reporting software and staff are critical to success.