Understanding Whiskey
As a whiskey connoisseur with a strong preference for American whiskeys, I've long been curious about the conventional attitudes towards whiskey. Longer aged whiskey is prized above younger whiskies, but is it really better? Many of the oldest whiskies are off the Scotch variety, and Scotch seems to be preferred to other varieties; could it be that age doesn't matter as much as people think, or that it only matters for Scotch? And what is really the relationship between quality and price? Many people may mistake price for quality, so perhaps high prices are a self-fulfilling prophecy. Can the experts tell the difference, or are they just faking it?
Motivation
As a whiskey connoisseur with a strong preference for American whiskeys, I've long been curious about the conventional attitudes towards whiskey. Longer aged whiskey is prized above younger whiskies, but is it really better? Many of the finest bourbons I've tasted have been aged for under a decade, while more mature ones were merely average; could it be that maturation in American oak is largely complete after only a few years? Perhaps age isn't a good indication of quality altogether, and is merely a novelty fixation for the pretentious and undiscerning? In order to answer these questions, I went to the experts at Whisky Advocate magazine.
The Dataset
I used Selenium to download all regularly categorized whiskey reviews from whiskyadvocate.com. In total, there were 11 categories with 3774 whiskey reviews written by 13 expert reviewers.. The reviews were written between 2000 and 2018, and cover whiskeys aged up to 70 years, and with prices ranging from $8, all the way up to $157,000. It took approximately one hour for the script to cycle through all of the pages and download all of the reviews.
Processing the Data
In order to answer the above questions, I extracted the age of the whiskey from the titles of the whiskey reviews, which yielded 1,359 whiskey ages (slightly more than one third of the whiskeys). I also extracted and analysed the text from all of the reviews, and filtered out filler words and neutral/non-tasting related text. Next, I examined the relative distributions of reviewers, categories, review words, scores, prices, and age, and looked for important correlations.
What's the Deal with Whiskey?
A heatmap of cross-correlations between all of the variables
The above heatmap shows the correlations between all of the variables, including the seven basic variables of date, reviewer, category, age, price, score, and alcohol percentage, and nearly 200 of the highest frequency tasting words. The resulting map shows mostly single digit numbers (maroon color), with a white self-correlation line across the diagonal, and scattered sparks of red and orange (10%-70% correlations). The brightest of these are quasi-self-correlations: words which usually go together in the reviews, such as "chocolate" and "milk." These aren't much more informative than the self-correlation line. There were a few interesting correlations in the words however, and they speak to the differences between the expectations for whiskeys in each category.
Scotch Whiskey Aromas and Flavors
Positively correlated descriptive words for scotch prominently feature smooth and mild aromas, as well as references to the smokey and peaty aromas from the malting.
The Irish whiskey descriptions were similar, but notably lacked references to the smokey and peaty character of scotch, and instead emphasized sweetness and honey. The Canadian and American whiskey reviews showed altogether different characteristics though.
Irish Whiskey
Bourbon (American corn whiskey)
Bourbon reviews focused mostly on oakiness and vanilla flavor, both of which come from the bolder character of American oak, the wood used for the barrels in which it is aged. Darker and bolder flavors such as caramel, cinnamon, leather, and tobacco also show up.
Rye Whiskey
Rye is also very different, and differs significantly from bourbon too. In place of vanilla, cinnamon is king in the rye flavors. Oak, is still prominent, but sweetness makes a comeback too. We also start to see more of the sharp and spicy aromas, such as mint and pepper.
Canadian Whiskey
The Canadian whiskey is also its own category, but it seems to combine aspects of the European and American whiskeys. Sweetness and fruit show up as they do in Irish whiskey, but vanilla and caramel are also strong, like they are in the other American whiskeys. Canadian whiskey also shows some of the hallmarks of rye such as pepper and spiciness: not surprising, given the common usage of rye in Canadian whiskey making, but surprising in how much more prominent the word "pepper" is than in the rye whiskey of the United States.
Bad Words
The above whiskeys all have different flavors that they are most appreciated for, but they all fail in similar ways:
The word "young" is typically used when describing a flavor that is unrefined, ad could have used more time aging. "Dominating" is used when talking about a lack of balance, when one flavor is too strong, and detracts from the rest of the flavor profile. Words like "hot" and "bitter" aren't necessarily bad, but they're often used to talk about off-flavors, and the rubbing-alcohol taste of a poorly refined whiskey.
The Reviewers
The above results are interesting, but don't tell the whole story. The reviewers may all be experts, but their reviewing behavior varies quite a bit.
John Hansell has over 1100 whiskey reviews to his credit, and they appear to be spread fairly proportionately over the full range whiskey offerings. Adam Polonski, on the other hand, has a total of six reviews, and four of those are rye whiskeys. Meanwhile, Gavin Smith has 822 reviews, and appears to drink Scotch exclusively. The different experience levels and palates of the raters do introduce some questions about whether there is uniformity in the scoring and reviewing, but this analysis leaves that question for future exploration. For now, it's sufficient to point out that while the question does exist, the data shows that categories are at least each given a fair shake, with similar mean review scores, and descriptive reviews (Unspecified single malt and New Zealand whiskey are scotch categories with a total of 18 reviews between the two of them. At a mean score of 83, they are still reasonably within range of the typical 86-88 mean. They were eliminated from later analysis due to their small sample sizes.)
How does Aging Relate to Quality?
A significant correlation does exist between the review score, the price, and the age of a whiskey, but in a subdued manner, and with a few important caveats. The most significant factor affecting price seems to be the collectible status of a given bottle, rather than merit of quality. This conclusion is based on the very wide distribution, which is spread over three orders of magnitude, and is dominated by vintage Scotches at the upper end. Scotch has the most well established collector culture, and an inherent need for longer aging due to the usage of used barrels, and fine-grained French oak. 
The result is that the top half of the above logarithmic scale plot (everything in the $1,000 and up price range) is dominated by Scotch whiskeys. Many well-reviewed Scotches are priced in proportion with the other categories too, but only Scotch can achieve ultra-high prices.
In the ordinary price range, all categories show an increase of quality (based on the review score) with age, but still a significant amount of variance around that line. As expected, the slower-aging categories don't increase in quality as rapidly as the faster (American oak barreled) whiskeys. A similar phenomenon exists with the price vs age chart below. Overall, this analysis suggests an average of 86 points as the intercept, increasing with age at a rate of 3/4 of a point per year of aging, and an associated price of $85, increasing by $2 per year of aging.
Conclusions
The experts do appear to be rating the whiskeys meritocratically, and award similar average ratings in all categories, but there are many questions that still need answering regarding the existence of single standard. The reviewers clearly have categorical preferences, and there are reasons to believe that they may be modulating their standards by price (it's otherwise a bit mystifying that an $8 bottle of supermarket Scotch can fall along the same distribution as a $157,000 bottle of half-century vintage Scotch).
The desired flavors and aromas of whiskey do differ along categorical lines, but uniformly exclude certain hallmarks of poor refining and insufficient aging. Balance in the flavor profile is important across all categories.
In light of the above, a good direction for future exploration of this topic would be to analyse demographic consumer ratings from a site such as Distiller.com in order to put the expert reviews in perspective. It would also be interesting to consider Scotch and Irish Whiskeys which are barrelled in American oak in order to see how that affects the distribution.
The skills the authors demonstrated here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
About Author
