Decoy effects in the purchase of 3,649,027 bottles of wine

Cognitive bias and big data in a new paper from Devine et al 2025

A new paper uses supermarket purchase data to show how a classic cognitive bias manifests in the real world.

How decoy options ferment choice biases in real-world consumer decision-making (Devine et al, 2025) reports the trade-off consumers make between quality and price when choosing bottles of wine, and how that trade-off can be shifted around by the presence of items which aren’t chosen.

The scenario is a familiar one - choose a bottle of wine from the many options on the shelves. Some wines are more expensive (and better quality), other wines are cheaper (but worse quality). For the best options, these two attributes of quality and cheapness trade-off against each other - you can’t get more of one without sacrificing some of the other.

But not all choices are the best. There are some wines which are both more expensive and lower quality than others. In decision theory these are called “preference dominated” options - regardless of whether you prioritise quality or cheapness, you are going to prefer one of the other options.

This makes it sound like these options are irrelevant to the choice, and in theory they should be. Except that a curious phenomenon called the Decoy Effect has been reported, where the presence of a dominated option swings people’s preferences between two good (dominating) options.

Dan Ariely in his book Predictably Irrational gives a memorable example. Imagine you are 50/50 between two holiday choices: Paris and Rome. Rome has the Colosseum, Paris has the Louvre. You can’t decide. Part of the difficulty is comparing two items which differ - they are far apart in the space of features they possess and it isn’t clear how you should weigh the different attributes. How do you measure the Colosseum versus the Louvre?! Now imagine how the introduction of a simple, preference dominated, third option affects your decision. Say that the Paris and Rome holidays include a hotel, and both hotels include a free breakfast. The new, third, option is a holiday in Rome, with a hotel, but without free breakfast. This option is simply not as good as the original Rome offer (with free breakfast). On every attribute it is equal or worse, so there is no scenario where you would choose it. Ariely’s idea is that this third option (Rome-) makes the original option (Rome+) seem even better, better than Rome-, but also better than Paris, and so it swings your choice in that direction.

The phenomenon is robust when tested in controlled conditions - a third, preference dominated, option moves people’s choices towards the closest, dominating, option. However, like many cognitive biases, it is often demonstrated in artificial lab scenarios, asking people to choose based on descriptions, or between stimuli without much intrinsic value (such as clouds of dots which vary by size and number).

This is why the Devine et al paper is so exciting. They take real world decisions - real people in their real lives, spending their real money. Wine, they argue, is a smart choice for an object of analysis. Unlike many products, it comes in standard sizes. People often only buy one bottle at a time (suggesting that all the bottles on the shelf are, in some sense, in competition with each other), and there is the trade-off of quality and cheapness which allows us to identify different options for which we have equal preferences (and also to identify dominated options).

The data came from a supermarket loyalty card scheme, from hundreds of thousands of UK shoppers, across thousands of stores. After some sorting and filtering - excluding purchases of multiple bottles, or purchases from shops with too few or too many alternatives available for example, the researchers were left with over 3 million wine purchase decisions.

Taking the wines and matching them to their Vivino ratings (a crowdsourced wine rating app), they were able to show that the hypothesised quality-cheapness trade-off really did exist:

Figure 3 from Devine et al 2025 “Relationship between wines’ price and quality (star rating). The figure shows a strong negative relationship, suggesting that price and quality of wines are inversely related”

Then, looking at the wines available in individual stores they identified 20 pairs of wines which were equally likely to be chosen, with one of them being better (but more expensive) and the other being cheaper (but not as good).

They then looked at the other wines available in that store, in the quality-cheapness space around these pairs. In some stores, on some days, they found that the cheap wine dominated the competition (i.e. the distractor wines were similar in price, but actually more expensive and lower quality than the cheap wine). In other cases the high quality wine dominated the competition (there were comparable quality wines, but they were actually more expensive and lower quality than the good wine).

The decoy effect predicts that the similarity of the distractor wines should move customers preferences for the target wines, and in which direction (the option nearby in the space should get a boost), and this is exactly what they found. The left pair of bars in this figure show the effect:

Figure 5 Devine et al 2025

It’s a great result, using large and interesting data to demonstrate a lab phenomenon has real world consequences. The size of the result may not be big - on the order of a 1% change in preferences - but that amounts to hundreds of thousands of pounds of consumer spending across the whole data set (let alone all wine choices made). As well as the size, the fact that the analysis can detect the effect is notable - so many other things are happening when someone is going shopping - psychologically, economically, so many differences between individual shoppers - that even a phenomenon which might be completely reliable in the lab might not manifest in the world.

So we’re all irrational then?

No.

Taking the result at face value suggests our preferences can be pushed around by context. How could it be otherwise? Choosing wine, for most of us, is a great example of bounded rationality: we don’t have the time, or knowledge, to make the optimal choice, so we fall back on shortcuts - what looks nice, how rich we feel today, and so on. It’s not irrational to do this - the decoy effect helps us understand something of how decision making in the real world has to happen. And if we believe it, this result gives us a handle on how strong and how ubiquitous this effect is.

Tried to make a meme of today’s main takeaway. Does it work?

The analysis also shows something else which fits with what we know about cognitive biases. When situations are repeated, even our limited cognitive machinery can often deduce optimal strategies, wiping out one-time effects of cognitive bias. Frequent shoppers showed no decoy effect (the figure above, right side). Similarly, other analyses have suggested that the real-world effect of similar nudges is often zero.

What if I don’t want to believe it?

I’ve done analyses of this scale and complexity with messy real world data before (e.g chess tournament data). I know how hard it is to do, and I believe in the value of leveraging real-world, observational data sets, to test lab based phenomena.

Because I know how hard it is to do, I also know that the very complexity of the analysis opens up any conclusion to the criticism that it is contingent on the particular analysis choices made (e.g. see this critique of my chess analysis).

There are a few indicators of reliability that researchers can display, including for complex analyses. They can - as Devine et al did - argue that the analysis they have done is principled and follows standard and best practices for analysis.

They can share data and analysis code, a transparency measure which allows anyone to audit what they did and how they did it, increasing our confidence that there is nothing unusual in their choices. Devine et al use commercial data so decline to share it (although a purist would argue that in this kind of circumstance you still have other options - you can share processed or summary data, and/or share analysis code without the data. All this is more work though).

In the discipline of economics, with large observational data sets it is customary to show sensitivity analyses, showing that alternative analysis methods produce similar results. (Not shown here). Other comparable strategies are to use “held out” data (the analysis is finalised on 90% of the data and then tested on a part that is kept back; also not reported in this case).

Ultimately the best check is to use the same analysis on new data. The supermarket data would seem ideal for this, since comparable data must exist for time periods beyond the original data set.

It feels unfair to demand these things - an analysis has value without them, but until it does, I will regard the result as provisional. It shows a cool way cognitive bias could be demonstrated in real-world choices. But I’m wouldn’t advise a supermarket to change their marketing strategy just yet.

Cognitive biases are real. They show how people manage the uncertainty and limited cognitive resources to make decisions. This study does great work on translating a lab phenomenon to a real-world choice. In doing so it trades-off relevance against the certainty of the possible conclusions. A solid choice.

Citation

Devine, S., Goulding, J., Harvey, J., Skatova, A., & Otto, A. R. (2025). How decoy options ferment choice biases in real-world consumer decision-making. npj Science of Learning, 10(1), 60. https://www.nature.com/articles/s41539-025-00341-2

At the time of writing the captions for Figures 2 and 3 are wrong. Consult the preprint for the correct captions: https://osf.io/preprints/psyarxiv/7bjqs_v1

Other stuff..

Dan Davies: dangers of the explicit

Dan Davies ponders whether requiring people in large organisations to do all their thinking on the record (i.e. written down) is always a good thing:

Link: the magic button

Thinking Allowed: Wealth

Episode of the Radio 4 staple on extreme wealth: includes interview with Brooke Harrington whose work on offshore finance emphasises that this is not just a mechanism to avoid tax, but a way for the super-rich to buy privacy and exceptionalism from national laws - and so a clear and present danger to democracy and the social order (including the social order of capitalism).

Paper: Overconfidence Persists Despite Years of Accurate, Precise, Public, and Continuous Feedback: Two Studies of Tournament Chess Players

Abstract:

Overconfidence is thought to be a fundamental cognitive bias, but it is typically studied in environments where people lack accurate information about their abilities. We conducted a preregistered survey experiment and replication to learn whether overconfidence persists among tournament chess players who receive objective, precise, and public feedback about their skill. Our combined sample comprised 3,388 rated players aged 5 to 88 years from 22 countries with an average of 18.8 years of tournament experience. On average, participants asserted that their ability was 89 Elo rating points higher than their observed ratings indicated—expecting to outscore an equally rated opponent by nearly 2 to 1. One year later, only 11.3% of overconfident players achieved their asserted ability rating. Low-rated players overestimated their skill the most, and top-rated players were calibrated. Patterns consistent with overconfidence emerged in every sociodemographic subgroup we studied. We conclude that overconfidence persists in tournament chess, a real-world information environment that should be inhospitable to it.

Interesting because feedback usually erases biases (see the difference between frequent and infrequent shoppers above). If it persists - and Chess here seems like a strong test, allowing precise measurement of skill and of overconfidence in the estimation of it that suggests the bias is either due to something that cannot be corrected for OR it has some adaptive purpose (e.g. you need sustained optimism about your chances of winning to keep playing)

Heck, P. R., Benjamin, D. J., Simons, D. J., & Chabris, C. F. (2024). Overconfidence persists despite years of accurate, precise, public, and continuous feedback: Two studies of tournament chess players. Psychological Science, 09567976251360747. https://doi.org/10.1177/09567976251360747

…And finally

Charlie Parr sings his song “Cheap Wine”

END

Comments? Feedback? Recommendations for a crisp white wine? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online