The Making of Community Notes

RP#60 A third (and final?) comment on the Community Notes system recently picked up by Meta, following a great article about the original team which designed it.

The Making of Community Notes

From November 2024 in Asterisk Magazine:

Jay Baxter Keith Coleman Lucas Neumann Emily Thai

The team that built X’s Community Notes talks about their design process and the philosophy behind their approach to combatting false information on the platform.

This interview with key members of the team which developed Community Notes shows them to be thoughtful and seriously engaged with trust and quality issues. Essentially this is a pre-rebuttal to many recent criticisms of Community Notes (highlighted in #RP59).

As you might expect, the designers are able to express algorithm function excellently (and far better than I did in #RP58), and in one line : “What the algorithm does is find notes where people who've disagreed on their ratings in the past actually agree that a particular note is helpful.”

Trust design

They highlight a few other features of Community Notes, features which are deliberate design choices during development. These include that the algorithm works only on public data (so not on something like views, which used to be internal to twitter only, not public). The code is open (and now incorporates modifications submitted from outside X/twitter).

They also describe the choices around user anonymity. This shifted the focus so that notes need to stand alone as claimed, rather than be linked to a user’s prior reputation for having notes with many upvotes. Choice of language was important:

I don't think you will ever hear any of us — anybody who worked on this project — ever say the word “fact check.” There's a care to avoid using that phrasing in any of the things we say about the product, any of the language about it, anything on the product surface, because it's entirely about providing context and information and then letting you make your own decision about how to trust it. That's what leads to that higher trust. But, as Lucas said, we're working through a lot of people's priors on what the box on a tweet means. And everybody else still calls it a “fact check.”

They iterated many versions of the Note label, ending with the current "Readers added context they thought people might want to know".

Early user-testing showed that people liked the specificity of Notes (that they comment on a particular tweet, correcting a specific claim, rather than being a generic “fact-check” on a topic which is applied to all tweets on that topic). This seems to me to be something that Community Notes delivers which is under valued by criticism of the system.

There is also an admirable “separation of powers” in Community Notes, as implemented on X.

"There's the fact that the data is open source, that the code is open source, and that we still don't have any buttons on the X side to promote or demote any note. We've never changed the status of an individual note. We either take the entire system down, or it's running. Those three non-negotiables created a lot of work for us"

Speed

There are also a couple of things I hadn’t appreciated, which moderate criticism of the slow speed of Community Notes. First, that Community Notes are flagged to users after they’ve shared or liked a misleading tweet

“ And then, even if you did see a post before it got noted, if you engaged with the post, we'll send you a notification afterwards with the note, once the note's been rated helpful.”

And that the system has a feature for identifying media (such as video) that gets recycles across the system in many individual posts.

"[If] it's a media note. It's instantly matching on every other post that uses that media. I would guess that prior to the existence of Community Notes, there would have been a lot of copies of that image being shared around. "

Finally, any slowness in Community Notes must a) be judged against the speed of other fact checking system (such as professional fact-checkers) and, b) is not inherent in Community Notes. With a larger, engaged, user base, it could be faster.

Success?

Another criticism of Community Notes is that most proposed notes don’t get shown. It seems from the interview that this is a deliberate choice, to focus on the quality of notes, meaning on only a minority get shown, so as not to jeapodise the reputation of the system.

Overall, I recommend the interview as a tonic for some of the negative takes on Community Notes. It may be part of an unfortunately political moment, and not a complete solution to the problems of our information ecosystem, but it has considerable virtues, and some of the current flaws (limited notes, limited speed) are not inherent - with a larger user base they could be address. The message for Meta must be: this will only come close to working if you invest resources in supporting the community around it, and make design choices with the same thoughtfulness as the original team at Twitter.

Final word to the team:

I remember very early after launch there was a note on a White House tweet, and they retracted the tweet and updated the statement. What an incredible power to have put into the people's hands — that regular people on the internet can call something out, and it can change the way an important topic is discussed. It was pretty remarkable.

Current events

The newly inaugurated President of the US signed many executive orders on Day One, including one which enshrined the claim that efforts around misinformation had evolved into censorship:

Washington Post: Trump signs order to end ‘government censorship’ of social media (£):

“Under the guise of combating ‘misinformation,’ ‘disinformation,’ and ‘malinformation,’ the Federal Government infringed on the constitutionally protected speech rights of American citizens across the United States in a manner that advanced the Government’s preferred narrative about significant matters of public debate,” the order said.

and

[Trump] promised to “break up the entire toxic censorship industry,” saying that the federal government should stop funding nonprofits and universities that engage in flagging social media posts for removal.

More in this story:

Rawstory.com: 'Direct assault on reality': Watchdog group slams Trump's 'censorship' executive order

The order, billed as a directive against "censorship," effectively ends any efforts by federal agencies to coordinate with social media companies and other large internet platforms on fact-checking misinformation about elections, medicine, and other topics of public interest — a long-running goal of right-wing activists who claim that such fact-checks are attempts to stifle conservative expression online.

Including some early reaction:

“Let’s be clear: responding to disinformation is not ‘censorship’ — it’s a vital safeguard against those who lie to erode our democracy and undermine trust in our institutions," said Jankowicz [the president of the watchdog group American Sunlight Project]. “As the Supreme Court made clear in its dismissal of Murthy v. Missouri, those who falsely alleged that the government, researchers, and tech companies were engaged in a censorship conspiracy 'fail[ed]... to link their past social-media restrictions and the defendants’ communications with the platforms.'

There’s a lot going on here. For an independent take I recommend Dan William’s There is no "censorship industrial complex"

Other stuff

• BJKS Podcast: Robert Wilson: 10 simple rules for computational modelling discussion covered many topics in research, but was loosely structured around this 2019 paper: Ten simple rules for the computational modeling of behavioral data

• Lately Podcast: The end of the fixed price which is a glimpse into another way The Algorithms are changing things.

• Andy Masley: Using ChatGPT is not bad for the environment, or, at least, it is nothing compared to choosing to eat a hamburger

• Nat Buckley: A cool 90’s chart type you might not have seen before nice Data Viz note

And finally…

Statue of the king from the Lewis chessmen, photographer unknown (via)

Bonus: read about it on Scots Wikipedia

END

Comments? Feedback? Design recommendations for this newsletter? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online