Expert scores and red wine bias

red wine biasIs the Winestream media biased in favor of red wine? Our study seems to show just that

Do experts rate red wines more highly than white wines, regardless of price, vintage, and region? Does this mean there is a critical bias in favor of red wines?

That may well be the case. Data scientist, wine lover, PhD, and former college math professor Suneal Chaudhary did the numbers, analyzing more than 64,000 wine scores dating to the 1970s and taken from the major wine magazines. The results are something I’ve been trying to get a handle on for years, the idea that critics favor reds over whites. The details are after the jump:

The study found:

• More reds score higher than whites, while red wines are over-represented above 90 points and whites are over-represented below 90 points. In fact, reds are 1.2 times more likely to be rated higher than 90.

• As an expert score crossed 90 points, selling price and selling price variation increased quickly – in some cases leading to nonintuitive results, such as median reds costing more than more highly-rated whites.

• When two experts rate the same wine, about the only thing they agree on is if a wine is better or worse than 90 points. When the wines are scored higher than 90, the variation in the ratings increases considerably. In this, the wine experts’ rating may not be as accurate as those for other agricultural products, like potatoes.

We don’t pretend that these results are conclusive, given the variables involved. Red wines may be inherently better than white wines (though that seems difficult to believe). They certainly cost more to make, and that might affect the findings. The review process itself may have influenced the study. Not every critic publishes every wine he or she reviews, and those that were published may have been more favorable to reds than whites. And, third, the scoring process, flawed as it is, may have skewed the results regardless of what the critics did.

Still, given the size of the database, and size matters here, Suneal’s math shows something is going on. And that’s just not our conclusion. I asked three wine academics to review our work, and each agreed the numbers say that what is happening is more than a coincidence. That’s the point of the chart that illustrates this post – 90 percent of the 2010 red wines that we had scores for got 90 points or more. You can click on the chart to make it bigger.

In this, Suneal found what he calls the chicken-egg-chick dilemma, where critics rate red wine more highly because it’s more prestigious; where producers spend more money to make red wine because critics see it as more prestigious and consumers are willing to pay for that prestige; and where consumers are willing to pay a premium for red wine because producers and critics see it as more prestigious.

Finally, about the database: We obtained 14,885 white wine scores and 46,924 red wine scores dating from the 1970s that appeared in the major U.S. wine magazines. They were given to us on the condition of anonymity because the scores do not include every wine that the magazines reviewed, and the source didn’t want to claim that the data was complete or was originally collected with the goal of being a representative sample.

You can download a PDF of the report here.

17 thoughts on “Expert scores and red wine bias

  • By Steve Heimoff -

    This is a very interesting and potentially important study. I too have long wondered why red wines almost always score higher than whites, among my (former) fellow critics and myself. Even if the critic is tasting blind, he/she knows the wine’s color, and there may well be a bias towards red wines. Thanks for bringing this to our attention.

  • By Matt -

    Hmmm, seems this bias would be difficult to correct. As mentioned in your blog, don’t our brains just sort of “automatically” assume that crafting red wines are the height of the winemaking art? Or maybe it is the pinnacle, and we’ve got right? I don’t know, but I think it’s true that somewhere along the lines we all decided (producers included, though they’ll never admit it) that reds are always better, the truer expression of what winemaking is all about.

    • By Wine Curmudgeon -

      Thanks for this — it ties into some reporting I did with producers about the extra cost of making red wine. One of the reasons they told me that making red was more expensive than making white was the idea that red was more important, something I had never really considered.

  • By Trader Bill -

    I have been writing lately on wine ratings, including a post just a few minutes ago on my blog. This one compares the UCDavis 20-point systems with the various 100-point systems. There is only one subjective category in UCDavis system; In 100-point it also included potential for further evolution and improvement – aging. SInce few white wines have true aging potential, the scores are likely to be lower. Just my opinion, but it seems that you get a better evaluation on the UC system…would like to see that comparison.

    • By Wine Curmudgeon -

      I’ve judged on the Davis scale, and it does seem a little better than the 100 point scale. But I’m also wondering, after talking about the study with Suneal (who did the math), if grades wouldn’t be better — A, B, C, and so forth. Assuming, of course, we need scores at all. Which we don’t.

  • By Dwayne Morrison -

    Seems to go hand in hand with the wine snobs who say they only drink red wines, as if that makes them superior to the rest of us. I laugh to myself when I hear that and can’t imagine losing the pleasure of whites, and roses too for that matter.

    • By Wine Curmudgeon -

      I agree — one reason why I volunteer to judge roses at wine competitions is they’ll be tasted by someone who who is willing to give them a gold. It’s amazing how many wine judges don’t even consider them to be worth high medals just because they are rose.

      • By Susan Tipton -

        As the owner/winemaker of an all white wine winery, I found your article most interesting. When tasting whites in Chateauneuf du Pape with a French winemaker, I commented on the whites commanding a higher price than the reds. His response was, “as a winemaker you should know that whites are more difficult to make than the reds, thus the higher price.” This was exactly why I chose to focus on whites and rosés, because I found them a challenge.

        Also I must mention that making award winning whites can be a more expensive process than reds. Sure you have the cost of barrels with reds, but we have expensive stainless tanks that are chilled for months. We have a costly press that allows gentle whole cluster pressing. Our estate grapes are lovingly cared for and we keep crop yields low and hand pick at harvest.

        Thanks for the discussion!

  • By Chris -

    It is a common issue in the production of white wines. I once had a few glasses of wine with a reviewer that I had only met that day (not a close friend) who said that he did not believe Pinot Gris was a great grape and he would never score one above 90 points. I argued the point that the “best” Pinot Gris sould be a 100 point effort, but he disagreed.
    I make an aromatic white variety that often gets marginalized. I get many very glowing reviews, including comments on age-ability and expression of varietal characters, yet rarely get over 90 points. There have even been reviews that state it was their favorite version in the area, state, or even country, but 90 points.
    You learn to be thankful for any crumb thrown to your whites and people that feel scores matter seem to be aware of the bias. When a reviewer that is known for loving big reds likes your white enough to say kind words, the readers do listen sometimes. My wife, mom and some winemaker friends that I trust their palates like my wine, so I am happy.

  • By Daniel McKeown -

    interesting as time goes on in this biz, I find myself drawn more and more to interesting white wines and find most big red wines (those that score so high) to be sometimes exhausting to drink. To be fair, the weather up here in the PNW has turned much cooler and damp, and red does go better with the foods of the fall. I find myself looking for higher acid reds like Nebbiolo to cut through the fat of the richer foods, but still like a nice un-oaked white to start the meal, or at least while I’m cooking. Big reds get big scores for the same reason big steaks are expensive and seen as worthy of praise, it is what we expect. I would agree with the above that a great white wine is much harder to make (or to be more correct, to grow) because you can hide behind extraction, tannin, oak or the like. The fruit has to be immaculate to be amazing.
    Feel like a Soave tonight….

  • By Myron -

    As a winemaker for 40 years, making reds, whites & Roses, I have another take on the data. In my experience it is much, much harder to find an outstanding white than red. There are more mediocre & just acceptable whites than reds. Also fewer good & way fewer excellent whites. People seem to think it is easy to make whites & more difficult to make reds. I would respectfully disagree. We also need to consider market forces that push whites into easy selling non-distinguished cash flow wines. For example two of the previously best makers of Pinot blanc in Oregon have sweetened up their wines past the point of balance & joined the herds of sweet Pinot Gris makers. On the other side are those who think “Dry” means searing acid which is okay because it “will mellow with aging”. Also, as was mentioned, most “serious” wine drinkers have a bias against expensive whites.” Why pay that much when they didn’t have to age it in barrels & can release it in a year. The pressure for “reasonably” priced whites pressures growers to overcrop so they can make a profit. The number of balanced & concentrated whites is miniscule.

  • Pingback: Do red wines get higher scores than whites due to “bias”? | STEVE HEIMOFF| WINE BLOG

  • By jason Smith -

    What about Champagne?

    • By Wine Curmudgeon -

      We only analyzed red and white still wines, so someone else can try sparkling.

  • Pingback: Liquor Industry News/Links 10-29-16 | Franklin Liquors

  • Pingback: In Defense Of White Wine (Thoughts On Expert Scores And Red Wine Bias) | 1 Wine Dude

  • Pingback: Wine News: What I'm Reading the Week of 10/30/16 - Vinography: A Wine Blog

Comments are closed.