Seeking the Best Coffee in a Popularity Contest

Carl Bialik, “The Numbers Guy” examines in the Wall Street Journal Blog the way numbers in survey’s are used, and abused.

He writes, when Zagat Survey announced earlier this month that Starbucks Corp. was the winner of its Fast Food Survey for best fast-food coffee, the Seattle coffee maker made the most of it, running an ad campaign, a press release and a blog post celebrating the designation.

The survey results caught the eye of Nate Silver, the baseball-numbers analyst turned political-numbers analyst whose site, fivethirtyeight.com, aggregated polls and other data to forecast President Obama’s electoral victory last November. Silver wasn’t sure how Zagat compiled votes in its online survey for best coffee, but figured that if the food-guide makers used their usual methodology, it might not mean that Starbucks is favored head to head over runner-up Dunkin’ Donuts, or third-place finisher Peet’s.

For Zagat’s city restaurant guides, diners submit their ratings to Zagat, which compiles the results. The more popular eateries get more votes. Since Starbucks and Dunkin’ Donuts are far more ubiquitous — about 7,000 U.S. locations each for Starbucks and Dunkin’ Donuts, compared to about 200 for Peet’s — far more votes were likely to have come from people who had tried the two big chains than their smaller rival. It could be that those who had tried all three preferred Peet’s, on balance, but their votes would be drowned out by the non-initiates to Peet’s, Silver noted.

I checked with Zagat, and it turns out the survey they used to rate fast-food restaurants was even tougher on the little guy than Silver imagined. “The designation of ‘Best Coffee’ was determined by how many people answered the question: ‘Which fast food chain has the best coffee?’ ” Nicholas Sampogna, a spokesman for Zagat, told me. Respondents were given a list of 30 chains that serve coffee, and the one with the most votes won. Since Peet’s has much lower name recognition — it has retail locations in only six states — it’s impressive that it won 9% of the vote, to 28% for Dunkin’ Donuts and 38% for Starbucks.

That the survey was more of a test of which chain has the most coffee-drinking consumers than which chain has the best coffee was reflected in the San Francisco results. There, where Starbucks’s edge in stores is narrower — roughly 66 to 23, according to the company Web sites — Starbucks beat Peet’s in the survey by a margin of just 50% to 30%.

Lara M. Wyss, a spokeswoman for Starbucks, told me it was “a bit of a false premise to assume that the number of our stores is the reason for the result,” noting that in other categories the winners didn’t always have the most stores. She added, “I’d like to think that the ranking reflects the quality of Starbucks coffee and the taste that millions of regular customers enjoy.”

Wall Street Journal Blogs

 Print This Post