The half life of trust

Or: You trusting something isn't enough for me to trust it

‘The Volvo Story’ has a special place in the history of research on judgement and decision making. It’s a thought experiment. Imagine, it asks:

Let us suppose that you wish to buy a new car and have decided that on grounds of economy and longevity you want to purchase one of those solid, stalwart, middle class Swedish cars―either a Volvo or a Saab. As a prudent and sensible buyer, you go to Consumer Reports, which informs you that the consensus of their experts is that the Volvo is mechanically superior, and the consensus of the readership is that the Volvo has the better repair record. Armed with this information, you decide to go and strike a bargain with the Volvo dealer before the week is out. In the interim, however, you go to a cocktail party where you announce this intention to an acquaintance. He reacts with disbelief and alarm: “A Volvo! You’ve got to be kidding. My brother-in-law had a Volvo. First, that fancy fuel injection computer thing went out. 250 bucks. Next he started having trouble with the rear end. Had to replace it. Then the transmission and the clutch. Finally sold it in three years for junk.”

How likely is it that you would proceed as planned and buy a Volvo?

Most people share the intuition that they would rethink the Volvo purchase.

This is despite, as the authors put it, “the logical status of this information is that the N of several hundred Volvo-owning Consumer Reports readers has been increased by one and the mean frequency of repair record shifted up by an iota on three or four dimensions.”

In other words, this guy at a party is just one guy. Why do we weight his report more than the hundreds compiled in the Consumer Reports consensus?

For years I swallowed the standard line from the Heuristics and Biases literature, that this story highlighted something flawed in our thinking.

It’s been called The Anecdotal Fallacy (which is really just providing a name, something which ‘dulls the ache of incomprehension’ but isn’t really an explanation). It has been called an example of the Availability Heuristic, where vivid information dominates our judgements of risk (in this case, the personal testimony of the man at the party is more vivid than the hundreds anonymously compiled in the report. Another frequently invoked example of the Availability Heuristic is that people fear flying, with its extremely vivid, but rare, accidents, more than they fear being driven to the airport, which is actually the higher risk activity).

The Volvo story has also been used to illustrate Base Rate neglect, which is an example of a failure of Bayesian rationality. The thought here is that the Consumer Report provides a prior expectation of faults among Volvos, which should be integrated with the new information (one additional Volvo with many faults). If you do this integration mathematically your estimate of the rate of faults should move from “very small” to “very small + a small fraction” (i.e. still very small).

One cause of the success of the heuristics and biases programme, for which Daniel Kahneman got the Nobel Prize, and which gave us “Nudge”, behavioural economics and the rest, was that Kahneman and Tversky worked very hard on refining their experiments, which often relied on exactly the kind of vignettes represented by the Volvo Story, so that they precisely captured some quirk of our intuitions.

Fans of the literature will recognise this approach in the (increasingly unfortunately named) Asian Disease Problem, The (increasingly anachronistic) Linda Problem and others.

These vignettes bring into view two ‘obvious’ answers which are in contradiction. With the Volvo story, obviously - ‘logically’! - we should ignore the party guy. Also obviously, we wouldn’t.

This is the basic contradiction which requires an explanation. The authors of the chapter which introduced the story were aware that recognising the contradiction was only half of the issue: if, ‘logically’, the man should ignore the party-goer, only someone ignorant of human nature would expect that he would. What, they ask, explains why some information (like the Consumer Report) is ignored, and some (like the party-goer’s anecdote) is prioritised?

A 1980s Volvo, so this is what we’re talking about here

Now, many years after first hearing this thought experiment, I look at it differently. Rather than seeing the intuitive reaction (to abandon Volvo-purchase plans) as resulting from one of the many cognitive biases I was taught about, I see it as showing us something important about the nature of trust.

Trust, fundamentally, isn’t transmissible in the same way as information. We believe the people we trust, but it isn’t reasonable to expect others to believe them in turn.

Here’s a way of thinking about it: trust fades quickly with transmission. The half-life is short, so that if A tells B, and B tells C, and C tells D, the credibility that D can reasonably assign to what C has told them (that B told after hearing it from A) is close to nothing.

This is why it is reasonable to discount the Consumer Reports conclusion. A bunch of people (Who? How selected?) told Consumer Reports which collated this information (How? Under what biases?) and then published it. The chain of transmission contains too many points where biases could creep in. Maybe Volvo pay bribes to Consumer Reports? Maybe Volvo owners are too proud to report faults? The point isn’t that any bias is plausible, only that there are just too many unknown unknowns, and with each point in the transmission chain these multiply, rather than add.

The party-goer’s anecdote is vivid, but it also appears immediately verifiable. The party-guy is standing right there, ready to defend or explain his story. The mechanisms of credibility are tangible, so trust is greater than for the distant and anonymous evidence of Consumer Reports¹.

It sounds obvious, but far too many important communications - from news, to health information about vaccines - ask people to take things on trust without seeming to recognise how fragile trust is, how poorly it travels across social distance.

One notable example: Consensus messaging has been promoted as the way to convince people about about climate change (e.g. “97% of scientists believe in human-caused climate change”). Science is a massive, anonymous, aggregator. A Consumer Reports for evidence on everything. Trusting science is reasonable if you have some understanding of how science works, but if you don’t then it is the equivalent to “97% of people different from me who do things in ways and for reasons I don’t understand believe something”. No wonder some of us prioritise vivid information from people we meet at parties instead.

When we recognise that my trust doesn’t convert 100% into your trust we can communicate more effectively - giving reasons to believe what we’ve come to believe, rather than hoping that the strength of our conviction will carry the day.

What of our hypothetical Volvo-purchaser? I’m arguing that it isn’t unreasonable to rethink his choice after the party, just as it would also be reasonable to press on and follow the Consumer Reports recommendation. Reasonableness in the real world often doesn’t have correct answers. Choices are usually matters of which risks you want to take, and what (or who) you put your trust in. We’ll understand choice better when we take trust seriously as a problem for each individual to solve for themselves, rather than something any of us can, or should, take as a given.

---

This newsletter is free for everyone to read and always will be. To support my writing you can upgrade to a paid subscription (more on why here)

Keep reading for the references and more on persuasion by LLM, design thinking for chatbots and a quote from Mary Wollstonecraft.

---

References

The Volvo story originally (as far as I can tell) appears in:

• Richard E. Nisbett, Eugene Borgida, Rick Crandall & Harvey Reed, “Popular Induction: Information is Not Necessarily Informative”, in Judgment Under Uncertainty: Heuristics and Biases, Daniel Kahneman, Paul Slovic & Amos Tversky, editors, (1985), pp. 112-113.

There’s a side-note here about using LLMs for research. I had misremembered the source of the quote, so was trying to track it down. Gemini persistently hallucinated that it was in Kahneman’s Thinking Fast and Slow. Despite having a hardcopy, this was hard to definitely disprove in the time I was willing to spend looking through the book. An internet search also turned up many people who attributed it to Kahneman or Tversky, although I couldn’t immediately find a source where they discuss it (or attribute it).

Eventually, I got to Gary N. Curtis’ Fallacy Files which provided the reference (which I verified by picking up my own copy of Judgment Under Uncertainty):

• The Anecdotal Fallacy

Talking LLM blues

I’ve always been interested in how psychological science is laundered to support grander narratives of human failing and irrationality. Normally this happens via popularisation in the mass media, but you can also catch it happening in the scholarly literature.

Here’s Rogier et al:

“LLM Systems appear capable at exploiting cognitive biases that make humans vulnerable to persuasion and manipulation”

Following the claim there are citations. This is from the abstract of the first one

“Arguments that included factual knowledge, markers of trust, expressions of support, and conveyed status were deemed most effective”

(Breum et al, 2024).

Now the Rogier et al claim isn’t flat wrong - the Breum study did show an effect of status, and relying on arguer status is a plausible cognitive bias - but it is definitely selective to read this study and see evidence of manipulative power of LLMs.

If arguments that include factual knowledge are more persuasive that isn’t exploiting a cognitive bias, at least in the way that most people would understand it.

Here’s another study of those cited by Rogier et al to support the claim: Griffin et al (2023). One thing they report, which really jumps the shark in my opinion, is that simulating participants with an LLM shows that LLMs can be persuasive (because they ‘persuade’ simulated participants). To be fair that isn’t all the paper reports, but this part at least seems very minimal evidence for the claim that LLMs are exploiting cognitive biases to manipulate people (and the wider implication that we should be worried about their persuasive powers).

People need support if “do your own research” is going to help rather than hurt

A story from early 2024:

Nieman Labs “Asking people to “do the research” on fake news stories makes them seem more believable, not less”

Original paper, Aslett et al. 2023 Online searches to evaluate misinformation can increase its perceived veracity

Part of this must be what I called (in 2021) The Epistemic IKEA effect - we put more faith in beliefs we’ve ‘assembled’ ourselves. The study here also highlights the role of skill - those who are most inept at using search are most likely to find and trust low quality sources and confirmation of false beliefs. Here’s their abstract:

Considerable scholarly attention has been paid to understanding belief in online misinformation with a particular focus on social networks. However, the dominant role of search engines in the information environment remains underexplored, even though the use of online search to evaluate the veracity of information is a central component of media literacy interventions. Although conventional wisdom suggests that searching online when evaluating misinformation would reduce belief in it, there is little empirical evidence to evaluate this claim. Here, across five experiments, we present consistent evidence that online search to evaluate the truthfulness of false news articles actually increases the probability of believing them. To shed light on this relationship, we combine survey data with digital trace data collected using a custom browser extension. We find that the search effect is concentrated among individuals for whom search engines return lower-quality information. Our results indicate that those who search online to evaluate misinformation risk falling into data voids, or informational spaces in which there is corroborating evidence from low-quality sources. We also find consistent evidence that searching online to evaluate news increases belief in true news from low-quality sources, but inconsistent evidence that it increases belief in true news from mainstream sources. Our findings highlight the need for media literacy programmes to ground their recommendations in empirically tested strategies and for search engines to invest in solutions to the challenges identified here.

Design thinking about chatGPT

Amelia Watternberger makes the case that “Chatbots Are Not the Future”. It’s a fantastically presented argument, making the case that text inputs have no affordances and the responses don’t aggregate, so chat is the wrong interface for many tasks. Ironically, the affordances of chat are the reason I think chatbots are *right* for some of the tasks I’m interested in: deliberation, articulating your own position, understanding the position of others, and so on.

Amelia Watternberger: https://wattenberger.com/thoughts/boo-chatbots

Quote: Mary Wollstonecraft

Truth must be common to all, or it will be inefficacious with respect to its influence on general practice

A Vindication of the Rights of Woman (1792)

… And finally

From poorlydrawnlines.com by Reza Farazmand

END

Comments? Feedback? Urban legends? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

1. One of the reason that urban myths always are attributed to “a friend of a friend” or “a friend’s cousin” or some such - it gives the appearance of verifiability in the way “just some guy doesn’t.