The Politicisation of Misinformation

How do we get out of here?

This from the US National Science Foundation: Updates on NSF Priorities (19 April), news that the Foundation is going to terminate ongoing grant funding for projects which don’t match the new priorities (which emanate from the new President) :

13. Are you still funding research on misinformation/disinformation?

Per the Presidential Action announced January 20, 2025, NSF will not prioritize research proposals that engage in or facilitate any conduct that would unconstitutionally abridge the free speech of any American citizen. NSF will not support research with the goal of combating "misinformation," "disinformation," and "malinformation" that could be used to infringe on the constitutionally protected speech rights of American citizens across the United States in a manner that advances a preferred narrative about significant matters of public debate.

Similar to the baseless accusation of political bias made by Meta when it terminated it’s 3rd party fact checking system in the US, this statement associates misinformation research with free speech infringement.

On the one hand, I can see that excess of control over information under the guise of “misinformation” could be free-speech infringing. Laws, such as that enacted in Greece against fake news, risk abuse by governments when they are combined with the power to arbitrarily claim what is true. (In related news, the Russian news agency TASS has just launched a fact-checking unit, which promises to be far from independent.)

However, If the assumed context for the NSF statement is speed on social media, this seems to confused freedom to speak with freedom to be heard, or to avoid contradiction.

By their works ye shall know them. There’s a team crowdsourcing a list of terminated NSF grants (Bsky thread by Noam Ross on this). A search for “misinformation” shows up seven grants so far terminated. Here are their titles:

• Using Data Science to Quantify the Impact of Misinformation, Mistrust, and Other Key Psychosocial Factors on Vaccine Hesitancy Among Vulnerable People Experiencing Psychopathology

• Investigating and identifying the heterogeneity in COVID-19 misinformation exposure on social media among Black and Rural communities to inform precision public health messaging

• LatiNET, a Multilevel Social Network Model to Examine and Address SARS-CoV-2 Misinformation in Low-Income Latinx Communities.

• Influence of Social Media, Social Networks, and Misinformation on Vaccine Acceptance Among Black and Latinx Individuals

• Cancer misinformation on social media and its correction

• Cancer Misinformation and Use of Complementary and Alternative Therapy (CAM) among Patients with Cancer.

• An inoculation theory-based messaging intervention addressing misinformation about HPV vaccine on social media: The Inoculate for HPV Vaccine randomized controlled trial

This is millions of dollars of research, already underway, which will now be stopped midway. Unsurprisingly, as a researcher I view this as both wasteful and an overreach by executive power. Governments can set priorities for grant funding, but if they have set up a systematic process for evaluating what research to fund researchers they shouldn’t intervene mid-grant except in the most exceptional circumstances.

Further up on the Updates on NSF Priorities page is this:

10. Is there a list of words I should be avoiding?

There is not a list of words. Investigators should focus on Intellectual Merit and Broader Impacts for proposals in a way that is consistent with NSF priorities.

You can make up your own mind about how these projects were most likely selected for termination.

Also, this

Now, since yesterday 22 April, the whitehouse.gov pages on covid highlight the lab leak theory of covid’s origins:

The pages make the case that Anthony Fauci (chief medical advisor to the president, 2021-22) suppressed “the truth” about covid’s origins, pushing social media companies such a Meta to suppress dissent on the grounds of public health.

Going back to check, fact checkers themselves seemed to have called this correctly all along. Here’s FactCheck.org, as US based fact checker, with Fact Checks from early 2021:

• May 2021: The Wuhan Lab and the Gain-of-Function Disagreement

• June 2021: The Facts – and Gaps – on the Origin of the Coronavirus

They are clear and consistent: this is an area where we cannot be certain, and the lab leak hypothesis is possible but unlikely.

Here’s a fact check with a stronger claim, but it is directed at a debunked variant of the lab leak hypothesis (that the coronovirus contained genetics showing deliberate insertions from HIV) , not lab leak in general:

• February 2020: Baseless Conspiracy Theories Claim New Coronavirus Was Bioengineered

My own view is informed by this (very lengthy) report of a (very thorough) debate : Practically-A-Book Review: Rootclaim $100,000 Lab Leak Debate (subtitle: “I watched 15 hours of COVID origins arguments so you don't have to - but you should!”). ACX is an independently minded, science educated, wicked smart, commentator and if he swings against the lab leak hypothesis (as did the prediction market following this debate), that’s good enough for me.

What I’m not clear on, is why the issue is such high stakes in the US. Here’s Andrew Sullivan, a right wing and and decisively independent, as well as anti-Trump, commentator: Why Did This Man Mislead Us? (“The disturbing tale of how the scientific establishment took aim at the lab leak theory”). Like the Whitehouse, and in contrast to the fact-checkers, Sullivan portrays the lab leak theory as far more certain than it is (not just more likely, but decisively established).

I don’t really understand where this is coming from. The lab leak may be true. It may legitimise action against China (which many in the US want to take anyway) or allow the current administration to victimise people from the previous administration, but I don’t really get why the strength of feeling, such that it seems to require confidently asserting which is at best uncertain.

I find it easy to believe that, during a time of unprecedented crisis, social media companies such as Meta bowed to government pressure, and caught some reasonable dissent in the net. In itself this hardly warrants the reaction against fact-checking, against misinformation research, or public health.

I guess I could better read the exact contours of this by becoming a more preoccupied observer of US politics, but the state of US politics doesn’t exactly inspire me there. At some point the signalling value of these positions overtook the facts of their actual reasonableness as the most important thing. In other words, they became politicised.

The trajectory of how it happened may be unimportant. Now lab leak has been politicised the Whitehouse has to confidently assert what it can’t know is true, and now misinformation research is tainted as being part of a censorship-industrial complex (which doesn’t exist), NSF has to terminate grants which use this language.

The historical details of how this happened may be besides the point, but we urgently need to understand how to depoliticise politicised issues. It may be hard to have a conversation about healthy epistemic communities with those who think conversations about healthy epistemic communities are suspect, but it certainly less doomed to failure than having the conversation without them.

Updates on Community Notes

Coming to Meta in 2026 Says their Chief Global Affairs Officer

Waitlist is open (via platformer)

To sign-up, you need to be based in the US, be over the age of 18, have a Meta account that's more than six months old and in good standing and have either a verified phone number or two-factor authentication. Meta says contributors will start to be taken off the waitlist once the Community Notes beta launches. The company hasn't given a hard date for the launch, but does mention that Community Notes will be phased in "over the next few months."

The Case for Facts

"This rigorous fidelity to facts — to gathering evidence before reaching conclusions — is the true meaning of objectivity in journalism. It's not about being neutral or passive, but about being relentlessly committed to uncovering what's true and accurate. When democracy is under threat, this disciplined approach to truth-seeking becomes more crucial than ever. It's what distinguishes journalism from content production or social media commentary. "

Angie Drobnic Holan, 10 Feb 2025 : The Case for Facts : Verifiable evidence remains journalism’s foundation, despite mounting attacks on the press

A nice quote

“argumentative discussions can also be harnessed for a kind of excoriating mutual improvement.”

The New Libertarian Elitists : What’s behind the dangerous new notion that democracy should be left to the well-educated. Henry Farrell Hugo Mercier Melissa Schwartzberg (2023). https://democracyjournal.org/

PODCAST: Hasok Chang on Epistemic Iteration (Dec 2023)

"We get this instinct that true science must start from a firm foundation. Time and again, that's what I see NOT happening in the practice of science. We start from where we stand. The foundation is never indubitable, the foundation is provisional."

Foundationalism is one of those things which was so obvious it took the naming of it for me to realise.

Link: The HPS Podcast Hasok Chang on Epistemic Iteration (Transcript)

PAPER Debating with More Persuasive LLMs Leads to More Truthful Answers

From the abstract…

…two LLM experts each argue for a different answer, and a non-expert [either a non-expert LLM or a human] selects the answer. We find that debate consistently helps both non-expert models and humans answer questions, achieving 76% and 88% accuracy respectively (naive baselines obtain 48% and 60%). Furthermore, optimising expert debaters for persuasiveness in an unsupervised manner improves non-expert ability to identify the truth in debates. Our results provide encouraging empirical evidence for the viability of aligning models with debate in the absence of ground truth.

Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakrishnan, A., ... & Perez, E. (2024). Debating with more persuasive llms leads to more truthful answers. arXiv preprint arXiv:2402.06782.

JOB! 30 month Postdoctoral Researcher on digital democracy.

… in Birkbeck, University of London - School of Psychological Sciences, working with Ulrike Hahn on the grant PERYCLES: Participatory dEmocRacY that sCaLES

Are you passionate about digital democracy? Birkbeck, University of London, is looking for a highly motivated Postdoctoral Researcher to join a dynamic, research project on a full-time, fixed-term contract this Spring.

Joining the School of Psychological Sciences, you will be working on a Horizon Europe project on digital democracy that focuses on threats to inclusion and safe participation in deliberation, including phenomena such as self-censoring, epistemic silencing, misinformation, and flooding the zone.

As Postdoctoral Researcher you will be involved in conducting both empirical research (survey and behavioural experiments) and simulations with agent-based models.

Deadline: 1st of May 2025.

LINK

And finally…

Two newsletters in two days! Catch up on yesterday’s : The End of History and the Last Man. Today’s post felt urgent, with some of the events of the last 30 hours. Yesterday’s had a different kind of urgency, with the events of the 30 years since the book publication.

In general, I try never to be relevant to contemporary events, so please excuse today’s post as an accident. My normal writing schedule will now resume, which is to say:

END

Comments? Feedback? Handy hints for depoliticising topics? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

The End of History and the Last Man

Francis Fukuyama’s book is over 30 years old, but the argument he makes has never been more relevant.

Let me embarrass myself, until this month I’d never read this book, and got my opinion of it second hand. Wasn’t Fukuyama a neoliberal shill, high on the triumphalism that followed the collapse of the USSR? Didn’t he say that history had stopped happening? in 1992?! Ridiculous!

It was me who was ridiculous. Like so many others, I couldn’t even base my misinterpretation of the book on the whole title, fixating merely on “the End of History” part.

I finished actually reading the book last night and I have to tell you about it¹. The Future may depend on it.

* *

First, The End of History.

Fukuyama’s book might have been less misunderstood if he had called it “The Ends of History” or “The Direction of History”. The “End” in the title is meant is the sense of destination, purpose or goal - telos.

To ask this question you have to define history in some sense more than just a record of events. Events will always keep happening, but Fukuyama is interested in the history of how societies are organised. The book is an enquiry into whether society is evolving, into whether this evolution has a direction, or if it is just randomly changing, as prone to revert as advance, or perhaps locked into endless cycles. As such, it is a broad sweep intellectual and sociological account of civilisations, broadly construed, which should interest anyone who enjoys Big History of the style popularised by Jared Diamond (who I’ve read) or Yuval Harari (who I haven’t).

The weakest version of Fukuyama’s claim is that, yes, history has a direction. We make innovations that we are not going to un-invent - everything from fire to computers. These innovations create possibilities, lock in certain ways of organising society, and there is no going back.

The strong version of Fukuyama’s claim is that both capitalism and democracy are stable organisational forms which naturally emerge when the contradictions of other organisational forms, such as monarchy, become too much. The contradictions of a society are not mere irrationalities or problems, but tensions which will eventually force change on the wider level. Fukuyama notes that critics strawman his thesis by listing problems with capitalist democracies (drugs, crime, etc), but none of these problems make the system untenable. They don’t qualify as the kind of contradictions he is talking about.

Fukuyama’s argues that organisational logic requires societies to organise around individual economic freedom and markets. The efficiency gains are too great for non-market forms to out-compete and replace market orientated societies.

States copy each other, accelerating the rationalisation of economic life. Labour organisation (specialisation, bureaucracy) and technological innovation are both ratchets - once they have been adopted it is hard to go back. This is not just because of the wealth they generate, but also because they, in their turn, create additional needs for specialist knowledge, for freedom for individual innovators, for market complexity. This complexity defeats central planning, at least in the realm of the market.

This is the case for Capitalism made by Fukuyama, but he is clear that these logics don’t require Democracy for their success. States which are market orientated modernisers but also authoritarian do better economically than Liberal Democracies. “If anything, democratic politics is a drag on economic efficiency”, he says (p205). So if Democracy cannot be explained solely in economic terms, something else is needed. For this Fukuyama develops an argument based on the philosophy of Hegel.

Woah, Hegel

Economic man is defined by reason and desires, says Fukuyama, and uses one to satisfy the other. This account is good enough for the theories of economics, but desperately impoverished when it comes to an account of what makes man human, it isn’t enough - says Fukuyama - to understand History.

The missing part is thymos, a key to Hegel’s philosophy. In Plato, thymos is usually translated as ‘spiritedness’ or ‘passion’ (and written ‘thumos’), but Fukuyama reads it as ‘pride’, ‘prestige’ or ‘self worth’². It is “that part of man which feels the need to place value on things - himself in the first instance” (p163). Desire is the drive to want things, thymos is the drive to have yourself, and your wants, respected by others. Thymos is fundamentally social, it is the desire to be recognised equally, or above, others. It is also fundamentally moral, since it evaluates the self and others.

Democracy, in Fukuyama’s view, is compelling because it more successfully solves the “struggle for recognition” which is part of what it means to be human, for all humans. Non-democracies do not just deny power to the majority, they deny recognition and so self-worth, creating an irresolvable tension which must eventually create revolution.

Democracy solves the problem of the universal desire for recognition. It allows all to be respected, via the mechanism of state power which recognises the rights of each individual, as an individual. This satisfies what Fukuyama calls isothymia - the drive to be recognised as an equal. For megalothymia - the drive to be recognsied as better than others - liberal democracy provides business, politics and arts as channels which provide temporary satisfaction.

“Desire for recognition, then, is the missing link between liberal economics and liberal politics” (p206). Fukuyama contrasts his account with the traditional liberal philosophy of government, via Hobbes and Locke, in which rational and selfish actors agree to a social contract (including submission to government) to avoid the war of all against all (and the nasty and brutish solitary life this entails). The omission of the fundamental desire for recognition makes the traditional account a negative one, not just an incomplete and unsatisfying one. It offers a diminished account of human possibility in which are common life is defined by the wish to avoid violence, rather than anything more.

Instead, says Fukuyama, we must recognise the influence on history of peoples’ desire for recognition and self-respect, from 1789 through to the foiling of the Russian coup in 1991.

This, then, is the argument for liberal democracy. First, it is a claim about twin dynamics - about the organisation of economies, on the one hand, that favours capitalism, and about the soul of man, on the other, that favours democracy. Second, it is an account of where we are in history. The claim is that these dynamics create a direction in history - that they explain why world society has developed as it has, and provide some understanding of how it may develop in the future. The claim is not that history is over - this is explicitly rejected in the last paragraph of the book, as well as being implicit in the discussion of the remaining unresolved contradiction of liberal democracy. Contradictions which may yet, says Fukuyama, generate their own revolutions against liberal democracy.

The Last Man

Now, the neglected second part of the title … “And the Last Man”. The Last Man is the man created by the conditions of liberal democracy³. His material needs are satisfied by the efficient economies of capitalism; his need to be recognised as an equal is satisfied by his democratic rights.

The Last Man is the objective of Fukuyama’s account. If, he says, we believe that the struggle for recognition has driven the major historical phenomena of the last centuries - religion, nationalism, democracy - and liberal democracy solves the struggle for recognition, then we are at the end of history. This is the key question, and it is the same question as this: is the Last Man fully satisfied by the kind of recognition provided by liberal democracy? Or may there yet be contradictions which will keep the gears of history turning?

There are two poles of criticism of liberal democracy, says Fukuyama.

At one pole, Marx made the case that legal equality is insufficient when there is wealth inequality. What use the recognition of my rights when I must sleep in the streets beneath the lit windows of the rich? Yet forcing greater equality will necessarily trade-off against individual rights. Fukuyama doesn’t dismiss this tension, just claims that economic leftism has failed to the extent that it has been radical (i.e. Communism) and is no alternative to liberal democracy to the extent that it is not radical.

At the other pole is Nietzsche, and the case that the Last Man - although rich in materials terms and rich in individual rights - may still not be fully satisfied. Fukuyama is clear that it is this he regards as the more profound challenge:

A society that grants such recognition may be the starting point for the satisfaction of thymos, and is clearly better than one that denies everyone’s humanity. But does the granting of liberal rights by itself constitute the fulfillment of that great desire that led the aristocratic master to risk death? And even if many people were satisfied by this humble sort of recognition, would it be satisfying for the few who had infinitely more ambitious natures? If everyone was fully content merely by virtue of having rights in a democratic society, with no further aspirations beyond citizenship, would we not in fact find them worthy of contempt? And on the other hand, if thymos remained essentially unfulfilled by universal and reciprocal recognition, would not democratic societies then have exposed a critical weakness?

(p302)

To use his earlier terms, the challenge is whether isothymia is enough, or whether to be truly stable society must deal with megalothymia, the desire of some or all to be recognised as greater than others.

Fukuyama also uses the notion of the The Last Man to talk about how the individualism and relativism of liberal democracies, while core to their definition, are also corrosive to the senses of commitment and community which have to come from outside liberal democracy. Historically, liberal democracies are built on top of cultural values that have their origins outside of liberalism. It’s an open question, he says, if liberal democracy can provide the nourishment required to sustain people’s participation in the habits of democracy. So, to the degree it entails a collapse of absolute values, The End of History is a threat to progress, not a culmination.

By the end of “The End of History and the Last Man” you are not left with the impression you have been listening to a victory march for democracy and capitalism. Rather, Fukuyama presents an enquiry which is by equal measures satisfying and unresolved. It is satisfying in that it paints a big picture account which combines political philosophy and the historical evolution of societies. It is unresolved in that it leaves you with the clear feeling that the stability of liberal democracy is very much not guaranteed, and the twin challenges of wealth inequality and unsatisfied megalothymia may yet provide destabilising forces.

“to the extent that liberal democracy is successful at purging megalothymia from life and substituting for it rational consumption, we will become last men. But human beings will rebel at this thought. That is, they will rebel at the idea of being undifferentiated members of a universal and homogeneous state, each the same as the other no matter where on the globe one goes. They will want to be citizens rather than bourgeois, finding the life of masterless slavery—the life of rational consumption—in the end, boring. They will want to have ideals by which to live and die, even if the largest ideals have been substantively realized here on earth, and they will want to risk their lives even if the international state system has succeeded in abolishing the possibility of war. This is the “contradiction” that liberal democracy has not yet solved.” (p314).

* *

In 2025

So, that, then is my brief and partial summary of the book. I hope I’ve made it clear why the standard criticism of it - criticism which I assumed, in an uncommitted kind of way, were correct until I read the book - is wrong. You can agree or disagree with his diagnosis that the serious ideological competitors to liberal democracy have failed. You can agree or disagree that the need for recognition is a fundamental driver of history. But don’t say that Fukuyama claimed history was over, or that liberal democracy couldn’t be bettered or existed without contradictions.

And please, please, don’t say that his book published in 1992 has been superseded, disproved or otherwise rendered irrelevant by the trends of recent decades. When I tally up Fukuyama’s pronouncements they look more accurate than not: the ongoing relevance of the nation state, the conflicts subliminated into cultural and economic differences; the prioritisation of identity. These all seem as strong, more strong even in 2025. The Fukuyama of 1992 talks directly to the desperate urgency of sentiment which has fuelled recent clashes over identity politics (whether on progressive or popularist) in the 2020s.

Even his unsettling description of the popular clamour for war in 1914, fed - in Fukuyama’s telling - by a desire for a unifying challenge which would shake off the stultifying comforts of peace doesn’t now seem so implausible (despite us all knowing how that story ended).

His description of the attraction of megalothymia seems to fit the worldview of the contemporary manosphere, as well as the behaviour of celebrity billionaires (liberal democracy must be friendly to business activities, says Fukuyama, precisely to channel the megalothymia of ambitious individuals and distract them from achieving political power).

The End of History and the Last Man is mostly a book of ideas, so few individuals are mentioned who aren’t philosophers. A few US presidents, some European monarchs. So it jumps out when one 1990s US celebrity is mentioned by name, Donald Trump. Fukuyama invokes him twice in his discussion of how business may only partially satisfy the drive to be esteemed above others, megalothymia. And from 2025 Fukuyama’s warning that the most ambitious individuals may move from business to politics and so destablise democracy looks as relevant as ever:

“ Thus, despite the apparent absence of systematic alternatives to liberal democracy at present, some new authoritarian alternatives, perhaps never before seen in history may assert themselves in the future. These alternatives, if they come about, will be created by two distinct groups of people: those who for cultural reasons experience persistent economic failure, despite an effort to make economic liberalism work, and those who are inordinately successful at the capitalist game.” (p235)

Coda

Thanks to H. who put this book in my hand (back in October).

Full text via archive.org: The End of History and the Last Man

Catch up

Normal service - news and views about the psychology of persuasion, argument and reason - will resume next time. In case you missed it, recent newsletters include:

This trilogy about Community Notes

The algorithmic heart of Community Notes: RP #58 What we know about the Fact Checking system Meta is adopting from Musk's X

Unhinged, unfortunate, unproven, inadequate, misdirected. RP#59 The bigger picture around Meta's announcement on fact-checking

The Making of Community Notes. RP#60 A third (and final?) comment on the Community Notes system recently picked up by Meta, following a great article about the original team which designed it.

This piece on a new report from me

Belief in the reasonableness of others: Reasonable People #57: Developing a measure of our faith in reason, new work from me

And this

Legibility and Legitimacy: Reasonable People #56: On recognising the limits of reasoned decision making.

And finally

Wacky 90s Fads

(Fukuyama doesn’t mention the internet, for what it is worth)

Credit: SMBC https://www.smbc-comics.com/comic/fads

END

Comments? Feedback? Alternatives to liberal democracy? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

1. This was true when I wrote the first half of this post, but I seem to have let 3 months go by before picking this up again and finishing writing up my thoughts.

2. I have also just learnt that Thumos is “is the name of an American progressive doom/post-metal band whose music is based on the works of Plato”. Yet another occasion when honest research on Wikipedia has led me astray.

3. Fukuyama, throughout his discussion of human nature, not just when referring to the Last Man, does refer to Man / He / His. Since my purpose it to address the common misreading of his explicit claim about The End of History, not to compensate or do apologetics for Fukuyama’s lacuna around gender. I have opted to preserve this convention here.

The Making of Community Notes

RP#60 A third (and final?) comment on the Community Notes system recently picked up by Meta, following a great article about the original team which designed it.

The Making of Community Notes

From November 2024 in Asterisk Magazine:

Jay Baxter Keith Coleman Lucas Neumann Emily Thai

The team that built X’s Community Notes talks about their design process and the philosophy behind their approach to combatting false information on the platform.

This interview with key members of the team which developed Community Notes shows them to be thoughtful and seriously engaged with trust and quality issues. Essentially this is a pre-rebuttal to many recent criticisms of Community Notes (highlighted in #RP59).

As you might expect, the designers are able to express algorithm function excellently (and far better than I did in #RP58), and in one line : “What the algorithm does is find notes where people who've disagreed on their ratings in the past actually agree that a particular note is helpful.”

Trust design

They highlight a few other features of Community Notes, features which are deliberate design choices during development. These include that the algorithm works only on public data (so not on something like views, which used to be internal to twitter only, not public). The code is open (and now incorporates modifications submitted from outside X/twitter).

They also describe the choices around user anonymity. This shifted the focus so that notes need to stand alone as claimed, rather than be linked to a user’s prior reputation for having notes with many upvotes. Choice of language was important:

I don't think you will ever hear any of us — anybody who worked on this project — ever say the word “fact check.” There's a care to avoid using that phrasing in any of the things we say about the product, any of the language about it, anything on the product surface, because it's entirely about providing context and information and then letting you make your own decision about how to trust it. That's what leads to that higher trust. But, as Lucas said, we're working through a lot of people's priors on what the box on a tweet means. And everybody else still calls it a “fact check.”

They iterated many versions of the Note label, ending with the current "Readers added context they thought people might want to know".

Early user-testing showed that people liked the specificity of Notes (that they comment on a particular tweet, correcting a specific claim, rather than being a generic “fact-check” on a topic which is applied to all tweets on that topic). This seems to me to be something that Community Notes delivers which is under valued by criticism of the system.

There is also an admirable “separation of powers” in Community Notes, as implemented on X.

"There's the fact that the data is open source, that the code is open source, and that we still don't have any buttons on the X side to promote or demote any note. We've never changed the status of an individual note. We either take the entire system down, or it's running. Those three non-negotiables created a lot of work for us"

Speed

There are also a couple of things I hadn’t appreciated, which moderate criticism of the slow speed of Community Notes. First, that Community Notes are flagged to users after they’ve shared or liked a misleading tweet

“ And then, even if you did see a post before it got noted, if you engaged with the post, we'll send you a notification afterwards with the note, once the note's been rated helpful.”

And that the system has a feature for identifying media (such as video) that gets recycles across the system in many individual posts.

"[If] it's a media note. It's instantly matching on every other post that uses that media. I would guess that prior to the existence of Community Notes, there would have been a lot of copies of that image being shared around. "

Finally, any slowness in Community Notes must a) be judged against the speed of other fact checking system (such as professional fact-checkers) and, b) is not inherent in Community Notes. With a larger, engaged, user base, it could be faster.

Success?

Another criticism of Community Notes is that most proposed notes don’t get shown. It seems from the interview that this is a deliberate choice, to focus on the quality of notes, meaning on only a minority get shown, so as not to jeapodise the reputation of the system.

Overall, I recommend the interview as a tonic for some of the negative takes on Community Notes. It may be part of an unfortunately political moment, and not a complete solution to the problems of our information ecosystem, but it has considerable virtues, and some of the current flaws (limited notes, limited speed) are not inherent - with a larger user base they could be address. The message for Meta must be: this will only come close to working if you invest resources in supporting the community around it, and make design choices with the same thoughtfulness as the original team at Twitter.

Final word to the team:

I remember very early after launch there was a note on a White House tweet, and they retracted the tweet and updated the statement. What an incredible power to have put into the people's hands — that regular people on the internet can call something out, and it can change the way an important topic is discussed. It was pretty remarkable.

Current events

The newly inaugurated President of the US signed many executive orders on Day One, including one which enshrined the claim that efforts around misinformation had evolved into censorship:

Washington Post: Trump signs order to end ‘government censorship’ of social media (£):

“Under the guise of combating ‘misinformation,’ ‘disinformation,’ and ‘malinformation,’ the Federal Government infringed on the constitutionally protected speech rights of American citizens across the United States in a manner that advanced the Government’s preferred narrative about significant matters of public debate,” the order said.

and

[Trump] promised to “break up the entire toxic censorship industry,” saying that the federal government should stop funding nonprofits and universities that engage in flagging social media posts for removal.

More in this story:

Rawstory.com: 'Direct assault on reality': Watchdog group slams Trump's 'censorship' executive order

The order, billed as a directive against "censorship," effectively ends any efforts by federal agencies to coordinate with social media companies and other large internet platforms on fact-checking misinformation about elections, medicine, and other topics of public interest — a long-running goal of right-wing activists who claim that such fact-checks are attempts to stifle conservative expression online.

Including some early reaction:

“Let’s be clear: responding to disinformation is not ‘censorship’ — it’s a vital safeguard against those who lie to erode our democracy and undermine trust in our institutions," said Jankowicz [the president of the watchdog group American Sunlight Project]. “As the Supreme Court made clear in its dismissal of Murthy v. Missouri, those who falsely alleged that the government, researchers, and tech companies were engaged in a censorship conspiracy 'fail[ed]... to link their past social-media restrictions and the defendants’ communications with the platforms.'

There’s a lot going on here. For an independent take I recommend Dan William’s There is no "censorship industrial complex"

Other stuff

• BJKS Podcast: Robert Wilson: 10 simple rules for computational modelling discussion covered many topics in research, but was loosely structured around this 2019 paper: Ten simple rules for the computational modeling of behavioral data

• Lately Podcast: The end of the fixed price which is a glimpse into another way The Algorithms are changing things.

• Andy Masley: Using ChatGPT is not bad for the environment, or, at least, it is nothing compared to choosing to eat a hamburger

• Nat Buckley: A cool 90’s chart type you might not have seen before nice Data Viz note

And finally…

Statue of the king from the Lewis chessmen, photographer unknown (via)

Bonus: read about it on Scots Wikipedia

END

Comments? Feedback? Design recommendations for this newsletter? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

Unhinged, unfortunate, unproven, inadequate, misdirected

RP#59 The bigger picture around Meta's announcement on fact-checking

Last week I reviewed Community Notes, the system Meta announced it would adopt for fact-checking from this point on. While I find positives in the Community Notes system, it is clear that alone it is an inadequate response to the problem of misinformation on social media platforms.

‘Unhinged’

Here’s Alexios Mantzarlis in his essential newsletter for anyone interested in fact-checking and misinformation, Faked Up:

Zuck chucks fact-checkers to cosplay as Elon Musk

…in which he describes Zuckerberg’s justification for dropping the industry leading Meta fact-check program as ‘unhinged’. More:

Zuckerberg justifying the termination of this program as a defense of free speech is particularly galling given that fact-checking labels did not lead to the underlying posts getting removed. Meta (rightly) took the approach of fact-checking as a contextual intervention which reduced a post’s reach but didn’t prevent users from continuing to access it.

‘Unfortunate’

And here’s Angie Drobnic Holan, Director of the International Fact-Checking Network (partnership with whom Meta just dropped):

Here's my statement as International Fact-Checking Network Director on Meta ending its US factchecking program: "This decision will hurt social media users who are looking for accurate, reliable information to make decisions about their everyday lives and interactions with friends and family. Fact-checking journalism has never censored or removed posts; it’s added information and context to controversial claims, and it’s debunked hoax content and conspiracy theories. The fact-checkers used by Meta follow a Code of Principles requiring nonpartisanship and transparency. It’s unfortunate that this decision comes in the wake of extreme political pressure from a new administration and its supporters. Factcheckers have not been biased in their work -- that attack line comes from those who feel they should be able to exaggerate and lie without rebuttal or contradiction."

Source: https://www.linkedin.com/in/angieholan/

Unproven?

The promise of the Community Note algorithm is that it can spin fact-check gold from highly partisan straw. The idea is appealing, but it surely must have boundary conditions. The information environment has bad actors - who can brigade together in crowdsourcing systems, manipulating things such as which sources are counted as trustworthy, and so by distorting inputs bias the outputs. Separately, there is also an open question (as far as I am aware) of how successfully the algorithm actually works in practice. Especially, since it wouldn’t be surprising if the crowd providing fact-checks on partisan information are themselves highly partisan (given the opt in is probably from some of the most-engaged, and so most polarised, users). Empirical work suggests this is the case :

PAPER: Birds of a feather don’t fact-check each other: Partisanship and the evaluation of news in Twitter’s Birdwatch crowdsourced fact-checking program

(excerpt from the abstract)

we find that contextual features – in particular, the partisanship of the users – are far more predictive of judgments than the content of the tweets and evaluations themselves. Specifically, users are more likely to write negative evaluations of tweets from counter-partisans; and are more likely to rate evaluations from counter-partisans as unhelpful. Our findings provide clear evidence that Birdwatch users preferentially challenge content from those with whom they disagree politically. While not necessarily indicating that Birdwatch is ineffective for identifying misleading content, these results demonstrate the important role that partisanship can play in content evaluation

Reference: Allen, J., Martel, C., & Rand, D. G. (2022, April). Birds of a feather don’t fact-check each other: Partisanship and the evaluation of news in Twitter’s Birdwatch crowdsourced fact-checking program. In Proceedings of the 2022 CHI conference on human factors in computing systems (pp. 1-19).

(via Faked Up)

Inadequate

Here’s Alexios again with his take on crowdsourcing fact-checks. He clarifies that he is pro- crowd-sourcing, but skeptical of relying on crowd-sourcing alone for all your fact-checking needs.

no project outside Wikipedia has succeeded at delivering a broadly legitimate fact-generating crowdsourced institution at the scale necessary for the modern information ecosystem.

In the (paywalled) post Alexios steps back and reviews the conditions required to make a crowdsourcing system work. Recommended. The conclusion (I strongly endorse) is that crowdsourcing can be a useful component of information moderation system, but it should not be the only component:

It’s too bad that instead of building from the lessons of complementary instruments to bring context to disputed facts online, Mark Zuckerberg chose to hyperbolize the benefits of one and the criticisms of the other.

A sentiment echoed in this open letter to Zuckerberg from ‘the world’s fact checkers’:

there is no reason Community Notes couldn’t co-exist with the third-party fact-checking program; they are not mutually exclusive.

Misdirected

Here Meta just flipped the switch that prevents misinformation from spreading in the United States, Casey Newton in his Platformer newsletter reports on the other changes Meta has made along with ditching professional fact-checking:

Last week, Meta announced a series of changes to its content moderation policies and enforcement strategies designed to curry favor with the incoming Trump administration. The company ended its fact-checking program in the United States, stopped scanning new posts for most policy violations, and created carve-outs in its community standards to allow dehumanizing speech about transgender people and immigrants. The company also killed its diversity, equity and inclusion program.

Behind the scenes, the company was also quietly dismantling a system to prevent the spread of misinformation. When the company announced on Jan. 7 that it would end its fact-checking partnerships, the company also instructed teams responsible for ranking content in the company’s apps to stop penalizing misinformation, according to sources and an internal document obtained by Platformer.

The story here is that Meta has developed successful classifiers (machine learning algorithms) which can identify hoaxes and is simply turning them off (so far only in the US). Instead it will rely on an untested and, as yet undeveloped, alternative system.

A final reminder that the Community Notes aspect is only a small part of this announcement. It is hard to see how this will do anything other than further the decline of Facebook, at least, as a useful platform, especially with respect to news information (there’ll still be a incentive to use it to contact friends, family or community groups for all those of the wrong age to be invested in other platforms).

In other news

De-bunking Pre-bunking

Love this line from Mike Caulfield

Not sure if I’ve said here how much I really hate the term “pre-bunking”, the adoption of which was largely a way for psychologists to engage misinformation with educational interventions without having to read educational theory.

Check out the whole post for a quick review of the information you need to get ahead of LA fire misinformation.

Other minds

This podcast Peter Godfrey-Smith on Sentience and Octopus Minds (on Sean Carroll’s Mindscape) brilliantly showcases the value philosophy can bring to biology, and the value research in evolution and animal behaviour can bring to philosophy of mind. Mind expanding.

Inspiration via antagonism

Ted Gioia: You Don't Need a Mentor—Find a Nemesis Instead. Thought-provoking essay. Would anyone like to fund my start-up idea : a dating app, but for a nemesis. We match you with someone you can hate, driving you to new heights of inspiration in your chosen field of endeavour?

Boundary conditions

Wrote this about proactively managing your workload. The thought is that lots of people will tell you how to manage your TODO list, but what you really need is advice on how to stop things getting on to your TODO list in the first place. Mostly aimed at academics (and one academic in particular, me).

And finally…

Source: redpenblackpen

END

Comments? Feedback? Suggested content warnings for this newsletter? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

The algorithmic heart of Community Notes

RP #58 What we know about the Fact Checking system Meta is adopting from Musk's X

Meta announced yesterday that they would move away from using professional fact-checkers on Facebook and Instagram (they also announced a set of changes to their moderation rules, allowing some content which would previously have been banned).

The move was widely greeted with concern.

• The Guardian: Meta is ushering in a ‘world without facts’, says Nobel peace prize winner.

• BBC: Huge problems with axing fact-checkers, Meta oversight board says

• Canadian CTV News: 'It’s just going to be a nightmare': Experts react to Meta’s decision to end fact-checking.

Among the concern was also derision:

• Algorithm Watch: Zuckerberg Makes Meta Worse to Please Trump

• And lots commentary and cartoons which were a lot ruder than this:

  

But what do we know about Community Notes, the fact-checking system Meta will adopt from X? And what do we know about how the algorithm at the heart of the Community Notes system actually works? That’s where we’re goingtoday - Let’s check the facts on this fact-checking set up!

Community Notes: the basics

Community Notes is a crowd-sourcing system. Users opt-in to submit notes on content which is posted on the platform. Other opt-in users vote on submitted notes, and when a threshold is reached that note appears to everyone on the platform, appended to the original content.

Here’s what it looks like¹:

Community Notes, originally called Birdwatch, was launched in 2021 and rolled out platform wide in 2023 (Musk acquired Twitter in 2022). Wikipedia reports that Yoel Roth, Twitter the head of Trust and Safety until November 2022, stated the Community Notes system was originally intended to complement, not replace, professional moderation.

Community Notes: what we’ve learnt

The context for the deployment of Community Notes looms large - Musk’s takeover, his cost cutting, scaling back of Trust and Safety staff and infrastructure, bullishness over free speech and over AI, as well as the general anti-establishment milieu of US tech culture. The skeptical readings of Community Notes are legion: that is a cost effective smokescreen for neglecting responsibility for combating misinformation, and is hand-in-glove with providing a friendly environment for various right-wing talking points.

Regardless, and separate from whether it is a good solution for the problems with the platform, Community Notes is interesting in its own right. Crowd-sourcing has had some demonstrable successes, such as Wikipedia and prediction markets, and Community Notes uses an interesting new algorithm for determining which fact-checks to promote, one which promises to overcome political polarisation. Further, on X, the Community Notes algorithm, and the Notes themselves and their votes, are all open for inspection.

Research on Community Notes has shown a few things:

Community Note fact checks can be high quality. This study of covid related Notes in 2022-23 found 97% accuracy (according to medical professionals)

Community Notes, like other fact-checks, work. This study found that notes on inaccurate tweets reduce retweets by half, and increase the probability that the original author deletes the tweet by 80%. This mirrors research by Twitter’s own team.

Community Notes are too slow. Most sharing of misinformation is in the immediate hours after it is posted. Community Notes take time to be written and voted on, by which time - arguably - lots of the damage has been done. This report on misinformation about the Israel-Hamas conflict showed that relevant notes typically took more than seven hours to show up, with some taking as long as 70 hours.

Most notes aren’t seen, most misinformation doesn’t receive notes. Professional fact-checkers have been cool about the practical import of Community Notes, as summed up in this Poynter headline Why Community Notes mostly fails to combat misinformation. An analysis of the 2024 US Election by Alexios Mantzarlis and Alex Mahadevan concluded:

At the broadest level, it’s hard to call the program a success.

Only 29% of fact-checkable tweets in our sample carry a note rated helpful. And of the helpful notes, only 67% are assigned to a fact-checkable tweet. This is not the kind of precision and recall figures that typically get a product shipped at a big tech platform.

…

Zooming out, the picture doesn’t look any brighter. In the three days leading up to the election, fewer than 6% of the roughly 15,000 notes reached helpful status.

Crowd sourcing risks crowd-biases. This study found that more sources cited in Community Notes were left-learning, while this study analyses the risks that groups of bad faith actors can skew the algorithm by targeting which sources are counted as credible.

Speak, algorithm,

The current operation of Community Notes as a fact-checking system is one thing, but at the heart is the bridging algorithm which is designed to select quality notes, despite polarisation among the crowd of users providing and voting on notes. Algorithmic approaches to harnessing collective intelligence in an era of increasing tribalism deserve our attention. How does it work?

Jonathan Warden has written about the algorithm's implementation of the algorithm , as has Vitalik Buterin. If you want technical discussions I recommend them, and what I say is derived from reading them and is a significant simplification (errors are all mine, of course).

At heart, the bridging algorithm works by using Users’ voting behaviour across all Notes to weigh the votes on a specific Note. A typically crowd-sourcing algorithm will judge quality based on averaging or summing votes (i.e. the Note with the most upvotes would be judged best). The bridging algorithm doesn’t do this. Rather it tries to model why notes receive votes, using the population of all users and all their votes to identify clusters. If, for example, there are a bunch of conservative users who tend to upvote fact-checks on progressives, and a bunch of progressive users who tend to upvote fact-checks on conservatives, the algorithm will identify these clusters of votes and be able to predict that notes from certain users, on certain topics, are likely to win votes simply because of partisan leaning, not because of the factfulness of the content. So, under the conditions where there is polarisation, the algorithm can discount predictable votes - e.g. your left-wing fact check on some right-wing misinformation that is upvoted by other left-wing users.

This is more nuanced that you might imagine. It is NOT the case that the bridging algorithm boosts Notes which receive support from across the political spectrum. Rather - and I owe this point to Jonathan Warden - the algorithm boosts Notes which receive support despite, or disregarding, the political leaning of voting users. To drop in some technical language for moment, the algorithm attempts to discover latent factors in voting behaviour and then discount them. A further nuance, suggested by Warden, is that if the primary polarisation in the community of users is along some other dimension than political leaning (e.g between naive users and experts) the algorithm might discover this dimension instead and discount it (i.e. actively work to disregard expertise in votes!).

All the Wisdom of the World

Despite limitations, in speed and reach, and vulnerabilities to biases or abuse, Community Notes is a fundamentally legitimate approach to moderation. The community of users provide and vote on Notes. Making it clear why and how fact-checks appear engenders trust in platforms which are too often opaque or perceived as biased. The problems of polarisation, speech moderation and misinformation are bigger than any crowd-sourced approach alone can handle, but that’s not a flaw of Community Notes as a system in itself.

Large platforms may currently be using Community Notes as a smokescreen for neglect of their responsibilities to our information environment, but that shouldn’t stop us recognising the value and potential for future development here. Collective intelligence is enhanced when the wisdom of the crowd is structured and channeled. Making transparent that algorithms do this, and building in a commentary layer on top of the raw feed of social media posts, are both hugely positive steps.

More

Community Notes

• code: https://github.com/twitter/communitynotes

• data: https://communitynotes.x.com/guide/en/under-the-hood/download-data

Faked up

If you’re interested in this topic, you must read Faked Up, the newsletter from Fact-Check and Misinformation beat doyen Alexios Mantzarlis.

Research on Community Notes

Accuracy :

• Allen, M. R., Desai, N., Namazi, A., Leas, E., Dredze, M., Smith, D. M., & Ayers, J. W. (2024). Characteristics of X (Formerly Twitter) Community Notes Addressing COVID-19 Vaccine Misinformation. JAMA. 2024;331(19):1670–1672. doi:10.1001/jama.2024.4800

Biases:

• Saeed, M., Traub, N., Nicolas, M., Demartini, G., & Papotti, P. (2022, October). Crowdsourced fact-checking at Twitter: How does the crowd compare with experts?. In Proceedings of the 31st ACM international conference on information & knowledge management (pp. 1736-1746). https://doi.org/10.1145/3511808.3557279

• Kangur, U., Chakraborty, R., & Sharma, R. (2024). Who Checks the Checkers? Exploring Source Credibility in Twitter's Community Notes. arXiv preprint arXiv:2406.12444.

Effectiveness:

• Renault, T., Amariles, D. R., & Troussel, A. (2024). Collaboratively adding context to social media posts reduces the sharing of false news. arXiv preprint arXiv:2404.02803.

• Wojcik, S., Hilgard, S., Judd, N., Mocanu, D., Ragain, S., Hunzaker, M. B., ... & Baxter, J. (2022). Birdwatch: Crowd wisdom and bridging algorithms can inform understanding and reduce the spread of misinformation. arXiv preprint arXiv:2210.15723.

Speed:

• Bloomberg.com How Musk’s X Is Failing To Stem the Surge of Misinformation About Israel and Gaza 2023-11-21

• Faked Up: X's Community Notes on Election Day

Algorithm

Jonathan Warden's posts about this

• https://jonathanwarden.com/understanding-community-notes

• https://jonathanwarden.com/multidimensional-community-notes/

And code for his implementation of the algorithm

• https://github.com/social-protocols/bridge-based-ranking

Notes from Vitalik Buterin

• https://vitalik.eth.limo/general/2023/08/16/communitynotes.html

Catch up

Belief in the reasonableness of others: Reasonable People #57: Developing a measure of our faith in reason, new work from me

Legibility and Legitimacy : Reasonable People #56: On recognising the limits of reasoned decision making.

The truth about digital propaganda: Reasonable People #55: Our piece in New Scientist bring evidence to worries about online manipulation

And finally…

Dragon on a Rock with Waters Beneath. Japan, late 19th-early 20th century.

Source. MFA Boston via @Sardonicus

END

Comments? Feedback? Algorithmic re-weighting which bridges across the divides within the soul? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online

1. This is the first Community Note I could find, after searching for Trump, Biden and then Wildfires, and scrolling “Top”. It was much easier to find posts with “Grok” notes - AI generated context. But that’s another story.