The marketplace of ideas
Reasonable People #11 A story about whether truth will out
There's a truth in art.
And there's a truth to whether you should drink bleach to cure an infection, or whether Brexit will let the UK spend £350 million a week extra on the NHS.
Ah, but they are not the same, you'll say
Maybe so, but there is a useful comparison.
Let's start our story in Detroit at the beginning of the 1970s. A folk musician called Sixto Rodriguez releases his first album. 27 years old, the sixth child of a working class Mexican family. Known on the local music scene. Hardworking. Undemonstrative. Definitely talented as a songwriter and performer.
Did Rodriguez hope for Dylan-esque fame? A release from the inner-city hardship of his early life? Like most young musicians, did he dream of the big time?
It wasn't to happen that way. There was a second album in 1971, and some limited success in the Australian charts, but then nothing. It looked like Rodriguez's musical career was over.
Talk to musicians and they'll speculate about what it takes to hit the big time. Talent and youth, sure, but there's lots of talented young musicians. You also need to be in the right place, and the right time, and put in the work.
And get lucky.
How much luck? Well opinions vary, but there's also at least one scientific study which offers us some insight. Matthew Salganik, Duncan Watts (of small worlds fame) and colleagues created 'an artificial cultural market' — a website where experiment participants' could download songs, but also see which songs other participants had downloaded and how they had rated them. They seeded the site with songs from 48 unknown bands — so unknown that I couldn't find most of the ones I tried on YouTube or Spotify, but here is one example, Lockdown by 52metro:
The research team were interested in how social influence affected these music markets. By running the experiment with different participants they could "re-run" the musical careers of their 48 bands in a toy world, see if the same tracks were always most popular. They could answer the eternal dilemma of how much Making It was due to luck, and how much due to the quality of the output.
The answer is that both factors matter:
"Success was also only partly determined by quality: The best songs rarely did poorly, and the worst rarely did well, but any other result was possible".
More surprising was the effect of increasing the power of social influence (they did this by making participants select which songs to listen to from a ranked table rather than an randomly ordered grid). The music market with the heavier social influence was less predictable in which songs would become hits. Which is to say, in these markets song quality counted for less than luck. Such markets were also far more unequal — a few tracks got the majority of the downloads.
This results make sense if you think of each track's quality being a "signal" which competes with the "noise" of everything else (the luck of which track each participant plays first, the particular mix of tastes found in each audience, etc). Social influence competes with each individual's own assessment of track quality, reducing the number of independent estimates of quality available in the whole network. That makes the market as a whole a less reliable judge of quality.
Music isn't the only place where we rely on other people's judgements. Social influence is everywhere, including in how we evaluate what is truth or false, fact or not.
When we see someone hit the big time as a musician it would be wrong to say there's no luck involved, but it would also be wrong to say that luck wasn't a factor. The Salganik study is an exercise in multicausal thinking.
You'd have thought that we'd be used to the idea that everything is influenced by multiple factors by now, but it seems its a lesson we have to keep learning.
Politicans lie and participants in psychology experiments believe things for irrelevant reasons. While some rush to declare this evidence of a post-truth era, just a modicum of multicausal thinking tells us that the existence of other factors of influence doesn't mean that the the truth is powerless. Other factors exist which affect how persuasive a fact is, beyond it’s status as true or not. But exactly how important is the truth of something to whether it is believed?
Fact checking can seem a futile exercise. The politician’s lie may heard half way around the world before the fact-checker has even pulled on their boots to start delivering the correction, but I was wondering if studies like Salganik's offered a different way of thinking about it.
There's a market for ideas, as well as a market for music. Just like the market for music, we could argue endlessly about what causes success in the market for ideas. Why not run an experiment like Salganik's and test some theories?
Create an artificial social network of commentators, who offer expertise of different levels of quality. Participants get to follow different experts, share insights they think are worthwhile, and - in turn - see who other participants follow and share.
What would your predictions be? Would the same experts become popular in each run of this artificial market of ideas? How important would luck be? And what factors would popular experts share? Clarity, humour, a talent for controversy?
Would the expert who became popular all share the quality of possessing true expertise?
Thinking about the long term and network effects of fact-checking gives us more reasons for optimism about the practice. Yes, individual lies sometimes often get vastly more attention than corrections, but take a network-level perspective: fact checking seeks to inject additional signal which helps, more often than not, truth to be sorted from untruth. In a network, not everybody needs to read a fact check for the true story to become widely accepted, and those who are unreliable to be discredited. Over time, quality as an information source can show through, despite the vagaries of other factors.
This manifesto — Fact checking doesn’t work (the way you think it does) — from Full Fact and partners, make it clear that they know that the audience for fact checks will always be niche, and that their aim is as much second-order effects of fact checks existings: making people afraid to issue false statements, being a source which supports credibility in the market of ideas.
Fact checkers aim to promote a world where truth is better recognised, where facts have heavy currency in the marketplace of ideas and unfacts less.
Back to Sixto Rodriguez. He got a second chance, long long after he'd given up. The story is told in the 2012 movie Searching For Sugar Man. Through the 80s and 90s his album became a slow-burn underground hit in South Africa, eventually prompting fans to go and look for him — and find him, still in detroit, not working in music, a family man without any idea he had a massive cult following on the other side of the world.
The market-perspective which tells us that the most popular music will be high quality in all possible worlds. This isn’t much comfort to the individual musician, who can also see that in every possible world there is high-quality music which, for random reasons, just doesn’t become popular.
The film created a new audience for Rodriguez's music, another possible world opened. In this run of the market he rose to near the top rather than sinking to the bottom.
He toured that first album again, nearly 30 years after releasing it.
Title: Cold Fact
Vox: What polarization data from 9 countries reveals about the US
Ezra Klein, 24th Jan 2020, on a new paper which shows that the US is an outlier in terms of increasing political polarisation (defined as how much party supporters hate each other). In some countries polarisation has actually decreased in recent years, and these are the ones which have had the most rapid uptake of internet usage (which seems to complicate the hypothesis that social media is driving polarisation). The bad news is that the increase in polarisation in the US is, at least in part, due to an exceptionally low level of polarisation mid-century, with the recent trend moving the degree to which people are prepared to hate each other based on their politics back towards a level which are more internationally typical.
Thanks to A. who sent me the link, following my mention of Kevin Dorst’s work in RP#8, and pointed out that the politics of the US shouldn’t be the only guide to our thinking on public reason.
Podcast: Philip E. Tetlock on Forecasting and Foraging as a Fox
This is episode #93 of Conversations With Tyler and has a lot on how to calibrate reasonable judgement.
Accuracy is only one of the things we want from forecasters, says Philip Tetlock, a professor at the University of Pennsylvania and co-author of Superforecasting: The Art and Science of Prediction. People also look to forecasters for ideological assurance, entertainment, and to minimize regret–such as that caused by not taking a global pandemic seriously enough. The best forecasters aren’t just intelligent, but fox-like integrative thinkers capable of navigating values that are conflicting or in tension.
The epistemic suspicion - moral suspicion axes
It’s good to be able to position yourself on the epistemic suspicion / moral suspicion axes.
…And finally
This is a hymn to the pleasures of deductive reasoning, even in a toy world. “It’s like the universe is singing to us”.
If you want to pause the video after he’s explained the rules (augmented from normal sudoku) you can try the puzzle here, but I found just listening to his think-aloud reasoning strangely compelling (top tip: you can watch at 2x speed for an extra level of frenetic excitement).