Too Much Power?

It should be a noble venture that raises the bar on customer satisfaction to the ultimate benefit of both builders and housing consumers. Why, then, do critics and seething builder resentment seem to follow wherever J.D. Power goes?

This is not a new phenomenon. Stop in at any watering hole where housing types gather at happy hour and just mention J.D. Power. Chances are, you'll get an earful questioning the survey itself and how builders use it. The only segment of the industry embracing J.D. Power's builder rankings in most markets are the largest national builders. They've learned how to play in this game, and they've got the chips to stay in.

Even critics acknowledge that the housing industry is probably better today than 11 years ago because of J.D. Power's presence in the marketplace. "There's no question they've changed the industry," says a competitor in the customer satisfaction measurement and consulting business. "The public builders especially weren't very interested in customer satisfaction before J.D. Power came along. They've been embarrassed into working at it, and a lot of them have gotten much, much better."

The largest builders were able to see the marketing potential of J.D. Power's awards, especially if they could win them in multiple markets. After all, a local builder can only win one; a national can win 34. First Pulte, then more recently Centex, became dominant leaders in this competitive arena.

"They've been good for the industry," says the chairman of a large, Texas-based builder with operations in several other states. "We bitch with the rest of the builders about some of the things they do, but the entire industry is better off for the focus on the customer that J.D. Power brings."

While firms such as Madison, Wis.-based Avid Ratings (which has a contractual relationship with Professional Builder that includes a monthly magazine column by Avid and co-sponsorship of the Avid Awards); St. Paul, Minn.'s Woodland, O'Brien & Associates; and Eliant of Irvine, Calif., all measure customer satisfaction and sponsor award programs, none can match the marketing clout of J.D. Power, which carried huge consumer brand recognition from other industries — particularly auto — when it entered the housing arena.

J.D. Power also benefits from the moral high ground it claims as an unbiased advocate of the consumer, even though its entire revenue stream is generated from builders who buy its syndicated reports and consulting services and pay licensing fees. It charges those who win its awards for the privilege of using the J.D. Power name, logo and award image in their advertising.

"We survey all home buyers in every market," says Paula Sonkin, the J.D. Power vice president who has led the housing industry program from its inception. "We get their names from independent sources in the public record. We do that because those sources are unbiased. If we got names from builders, we could never be sure that people critical of the builder were not being removed from the list."

Every person who closes on the purchase of a home in one of the 34 surveyed metropolitan markets receives a J.D. Power survey questionnaire between March and May of the following year. Sonkin reports the survey this year expanded from four to eight pages, which respondents had until the end of July to complete. "For the most part," she says, "we ask a lot of the same questions each year, so we can track performance year over year and trends as they emerge. But we always add a few questions to address hot topics in the industry."

This year's survey, for example, added a question about the impact of inventory homes on the market. It also included a question on how many negative recommendations buyers gave for their builder. The firm has now increased its awards fourfold by publishing studies on builder quality, design and mortgage origination processes in addition to customer satisfaction.

"We added those because, especially for quality and design, these are things many builders tout in their advertising to distinguish themselves from competitors," says Sonkin. "New home shoppers want to hear the unbiased, independent voice of past customers on these issues. And builders want more data on their points of differentiation so they can improve their businesses. When builders improve, that ultimately benefits consumers as well."

One thing that J.D. Power does to benefit all players in a market is measure against a fixed constant that allows everyone to see how much customer satisfaction is increasing (or not) across each market and the country as a whole from year to year. "We set the index at 100 in 2001," Sonkin reports. "In 2002, scores went up across the country on average one percent, so the index was 101. In 2003, the average jumped to 109. We added new markets every year, so it's not exactly a constant measure, but it went up to 112 in 2004, at the peak of the market, and held that average in 2005. This year, for 2006 sales, it dropped to 111."

When we ask Sonkin who J.D. Power's customer is, she names both consumers and builders. "We fund the studies ourselves and sell the information we gather to builders on a subscription basis. The full study allows builders to make decisions and prioritize their investments where they do the most good in satisfying buyers," she says. "But we publish our rankings for consumers to see on our Web site. Half the builders ranked will always be above the average for the market and half below. That helps shoppers make educated buying decisions."

J.D. Power maintains that its studies turn the voice of the customer into a shout that home shoppers want to hear. It's a unique perspective on the product, service and buying experience. "We're not Consumer Reports," says a J.D. Power insider from the auto industry, the source of its brand power. "We don't test products and report the results. That's a valuable service to consumers, but different from what we do.

Centex Homes is an example of a home builder that displays the J.D. Power and Associates logo on its Web site. It shows buyers - whoe most likely recognize it - the company is considered a customer satisfaction winner.

A different picture emerges when we ask builders and other industry insiders what they think of the way the firm conducts business — and the way builders use the survey.

Part of the contention lies with J.D. Power. Plenty of builders say J.D. Power uses high-pressure tactics to sell its subscriptions at prices many builders regard as exorbitant. What's unsettling is the worry in builders' voices when they talk about J.D. Power. Almost everyone we asked to discuss the company would only do it without being attributed.

Many builders distrust J.D. Power's survey methodology and believe the firm manipulates results to the benefit of large builders who pay the hefty fees the company charges for full access to its data and use of its name by award winners. We find no one with direct evidence of such shenanigans, but no one, including J.D. Power, should be surprised about the claims. It's a natural consequence of being a firm that promotes independent, unbiased surveys of buyers but derives its income from the companies selling the product.

We found no tangible evidence to support the suspicions of unethical behavior by J.D. Power; however criticism of its survey results is rampant. For example, builders must have 150 closings and 50 returned questionnaires to qualify for a J.D. Power survey. One could infer the awards are for production builders only. But nowhere on the Web site, in press releases or in the advertising of winners is the absence of small builders from the survey ever mentioned, even though such firms still build more houses than giants and make up the majority of builders.

"We never make their surveys," reports Atlanta builder Pam Sessions, president of former PB Builder of the Year Hedgewood Properties. "Sometimes we get enough closings, but then we have to get a really high response rate, and we never seem to make it. We get 50 responses on our own customer satisfaction surveys, but I guess they don't."

Sessions says J.D. Power is not a big thing with Atlanta buyers because so many builders are excluded. "This is a small builder-dominated market," she says. "We have a lot more builders that don't make the rankings than those that do. I'm sure consumers just look at it as a list that represents a small portion of the market. That could affect the perception of the validity of the results.

"The presence of J.D. Power in our market is still a positive. It forces all of us to hear that voice of the customer. It makes all of us want to survey our own buyers, even if they don't."

Response rate is another bone of contention with many builders. Those who have their own — or even other third-party — surveys regularly get response rates of 50 percent or higher. But our checks show that J.D. Power virtually never gets more than 25 percent and often much less. "We've had our clients show up in their rankings in the inverse order to the way they should be ranked," says one customer satisfaction measurement pro. "Who do you want to believe? Their response rate is 18 percent; ours is 80 percent. That's a variable that can happen, based on who they pick up in county records and how many of them mail back the questionnaire, which is not easy to fill out."

Other reasons for contention with the survey stem from builder abuse, specifically concern of widespread manipulation of survey results by some builders. We found many builders willing to cite examples of builders that have abused the survey system but only one builder willing to share what he knew and also be attributed for this article. "I have a niece who works for a prominent entry-level builder here in Houston," says Trendmaker Homes President Will Holder. "[She says] they were calling buyers, asking them how they planned to respond to the J.D. Power survey. If the buyers said they were not happy, they'd have the salespeople run out to the house and give the buyers a garage door opener — and try to talk them into changing their opinion of the builder."

An industry consultant says he knows of a case where a large public builder paid buyers to bring their questionnaires into the model home so salespeople could "help" them fill in the answers. "J.D. Power knows that kind of stuff is happening, but there's not much they can do about it. When the stakes get high, people will do this stuff, and a lot of the public builders are bonusing their people on J.D. Power scores. There are big bucks at stake."

J.D. Power's competitors in customer satisfaction measurement are critical of the way J.D. Power's methodology leads the firm to misinterpret data and give faulty advice to builders. Houses are not manufactured products, and J.D. Power's research methods — as well as the power of its brand — migrated from the auto industry. Customer satisfaction in home building is a complex puzzle. The process from first sales contact through closing and into warranty takes many months, often more than a year. What concerns buyers during sales or at closing can be long forgotten six months into the warranty period. Yet J.D. Power's average survey respondent fills out the questionnaire nine months after closing.

"They capture a large swath of data from people who have been in their home four to 18 months," says a competitor to J.D. Power. "The result is that J.D. Power believes warranty service is the No. 1 factor driving customer satisfaction. They lead builders to pump more money into warranty service than they should. Our data shows that quality at closing is more important. If you emphasize building the house right the first time, you maintain better satisfaction deep into warranty without spending nearly as much money on service."

Sonkin confirms J.D. Power's belief that warranty service is the top driver of customer satisfaction. "Our average survey is completed at about nine months after closing," she says, "which does create an emphasis on warranty service. That's the highest driver of customer satisfaction in our studies, followed this year by the role of the construction super. The most important aspect of warranty is timeliness of service, followed by cleanliness of the work."

The customer satisfaction professional mentioned earlier also disagrees with the emphasis on warranties. "We've done a study of our data base and we find that our best clients — the ones with the highest customer sat scores — spend less on warranty service than all our other clients. And they're spending less on warranty today than they were when their results were not as good."

One Midwest consultant's big beef against J.D. Power is that the firm allows public builders to advertise their J.D. Power awards in markets where no surveys are taking place. In other words, local private builders have no opportunity to compete for the advertised award, so it's unfair for J.D. Power to allow such advertising.

"It's not an award," responds Sonkin. "We have a certification program for builders to use in markets where we are not doing syndicated studies. In this program, we will survey a year's worth of the builder's home buyers. If the builder performs in the top 20th percentile of our national average for the syndicated studies, it can become a 'J.D. Power certified builder.' It's not a competitive ranking. And any builder can try for it, as long as it has 150 closings in a calendar year and we get a minimum of 50 surveys back."

But perhaps only a national builder anxious to play the J.D. Power card in other divisions in other markets would be interested.