Innovation collaboration
Innovation collaboration
Reasonable People #40 some recent metascience results on how to support innovative research
An Escape Room is an activity where your team are locked in a room and have to solve a system of puzzles to escape, typically within a one hour time limit. There are puzzles of different types - some require fine motor manipulation, some insight, some logical deduction. Some puzzles require two or more of you to coordinate or combine information across different aspects of the room. One might wonder exactly what kind of team is best at escape rooms - presumably it is some combination of individuals with complementary skills. You might also wonder exactly what kind of team is worse at escape rooms, and one day I had the opportunity to ask a man who ran his own escape room company, designing the rooms and shepherding groups through them.
“Is there any type of group”, I asked, “which you can predict are going to get locked in?”
“That”, he said, “is easy: families with dads who think they know best.”
Now I was interested.
And not just because of potential personal relevance (after all, in my family, I am a dad who actually knows best, not just thinks he knows best). I was interested because it seemed like the most salient factor for team failure was not something about the membership of the teams, but something about how they coordinate.
* * *
Research is a team game, increasingly so. It’s a series of interlocking puzzles, resources - and time - are limited - and if you don’t solve the puzzles then you get locked in a rut, or out of successful journal publication. Here are two recent pieces of evidence on the kinds of teams which enhance research innovation.
Item: Flat teams drive scientific innovation
Xu, We & Evans (2022) looked at the contributor statements from 89,575 scientific papers. These are footnotes to papers which indicate which authors took which roles in the research: securing funding, conceptualisation, testing, analysis, drafting the report, editing, etc. Using these statements Xu and colleagues said authors could be divided into leader and support roles. They then extrapolated their classification to a further 16,397,750 papers, calculating the ratio of leaders to total team size (“L ratio”).
Their findings:
relative to flat, egalitarian teams, tall, hierarchical teams produce less novelty and more often develop existing ideas, increase productivity for those on top and decrease it for those beneath, and increase short-term citations but decrease long-term influence. These effects hold within person—the same person on the same-sized team produces science much more likely to disruptively innovate if they work on a flat, high-L-ratio team
Here is one of their results, which looks at the L-ratio percentile (so more egalitarian research teams on the right) and plots the likelihood of the published papers combining novel topics (calculated using word co-occurrence frequencies).
And, further, they show that not only do these more egalitarian teams produce papers with more novel combinations of topics, but papers produced by them are more highly cited in the longer term (after 20 years), but less highly cited in the shorter term (within 10 years).
Item: Academic Freedom and Innovation: A Research Note
Academic freedom is “the right to choose one’s own problem for investigation, to conduct research free from any outside control, and to teach one’s subject in the light of one’s own opinions” (Polanyi, 1947). Audretsch and colleagues (2023) identify three main contemporary threats to academic freedom (my labels):
social: ideological interests groups attacking professors for their opinions or research;
political: direct political intervention in control over Universities (e.g. appointment of deans or departmental chairs);
governance: for-profit opportunities erode the collegiate model of Universities in favour of a managerial governance structure. As they say “This may induce academics to conform to institutional priorities and to eschew research themes that may be disliked by powerful donors and constituents, thus constraining research exploration.”
Although academic freedom is often held as important for innovation, Audretsch and colleagues claim the relation to innovation has not been directly tested before. To do this they combined data from the World Bank, PATSTAT (“the most comprehensive database on worldwide patent activity maintained by the European Patent Office”) and the 2022 release of the Academic Freedom Dataset from the V-Dem Institute of the University of Gothenburg. This gave them a dataset covering 157 countries, from 1900 - 2015 with 62.8M patent applications and 36.8M patent citations.
Here’s the headline graph
:
They followed this with control analyses, for example looking only within liberal democracies to control for type of government, and also looking across time (i.e. looking at the lag between prior academic freedom and current innovation) to control for the direction of causation. They conclude:
improving academic freedom by one standard deviation increases patent applications and forward citations by 41% and 29%, respectively. The results hold in a representative sample of 157 countries over the 1900-2015 period. This research note is also an alarming plea to policymakers: Global academic freedom has declined over the past decade for the first time in the last century. Our estimates suggest that the decline of academic freedom has resulted in a global loss quantifiable with at least 4.0% fewer patents filed and 5.9% fewer patent citations.
* * *
Here’s Zoltan Dienes (2023) with an attack on the way Universities are managed (at least in the UK): increasingly centralised, with coherence across departments monitored by a ever growing management class:
How is the rise of managerialism experienced by university staff? ... ‘The sense that conditions for the pursuit of high quality academic work have worsened and are continuing to worsen is widespread, even in institutions that are most obviously successful. '
Narratives of decline have a strong hold over the human psyche, and probably academics 30 years ago would have said the same thing. Academics today and 30 years ago, could both be wrong in their perception, or they could both be right. The analysis of academic freedom by Audretsch and colleagues certainly suggests this perception may be accurate. Dienes argues that we need to experiment with democratic reforms in the University to protect research integrity. If democratic reforms support academic freedom and encourage more egalitarian research teams, these two recent findings suggest such democratic reforms might also have the side effect of allowing more novel and impactful research as well.
References
Audretsch, D., Fisch, C., Franzoni, C., Momtaz, P. P., & Vismara, S. (2023). Academic Freedom and Innovation: A Research Note. arXiv preprint arXiv:2303.06097.
Dienes, Z. (2023). The credibility crisis and democratic governance: How to reform university governance to be compatible with the nature of science. Royal Society Open Science, 10(1), 220808.
Polanyi, M. (1947). The foundations of academic freedom. The Lancet, 249(6453), 583-586. (Paywalled, available for a mere $39.95 for 48 hours online access)
Xu, F., Wu, L., & Evans, J. (2022). Flat teams drive scientific innovation. Proceedings of the National Academy of Sciences, 119(23), e2200927119.
A footnote on the state of academic publishing
The implications for innovation are left as an exercise for the reader
Event: Future of Peer Review
Speaking of the conditions of academic idea production, join us for this free online workshop, jointly hosted by UKRN and RoRI, May 18 @ 1:00 pm - 3:00 pm
Peer review remains the primary means by which we ensure the quality of research grants and outputs. However, no system is perfect. Peer review could be improved, and there are different schools of thought on how this could be achieved. This event – hosted jointly by UKRN and the Research on Research Institute – will include a series of talks that outline approaches that, in part, reflect these different schools of thought. A panel discussion will enable participants and speakers to explore the commonalities, differences and implications of these approaches
Link: https://www.ukrn.org/event/future-of-peer-review-may2023/
Event: Straight to the Facts: Can we save public debate? A dialogue with Will Moy
Join us (in person, in Sheffield) to hear from fact-check superhero Will Moy, and my brilliant colleague Kate Dommett, at 5pm, Saturday 15th of April.
In the wake of a spate of high profile fake news stories and disinformation, calls for fact checking have reached new heights, but can fact checking really save public debate? Is it able to meet public demands? Or is political speech so polarized it is no longer possible to establish commonly held facts? At this event, Will Moy, Chief Executive of Full Fact shares his insight from over a decade working at the forefront of the fact checking debate. Responding to new findings from a team of University of Sheffield academics, this event asks whether public debate is condemned to be inherently unhealthy, or whether fact checking can restore democratic debate.
I, for one, salute our new generative AI overlords
Last newsletter I wrote about text generation models. Here, something different - voice synthesis and matching video lip syncing. As one of the comments says, a good example of how if you spend too much effort on something stupid it can become something amazing.
And finally…
Another thought about the uses of large language models.
END
Comments? Feedback? Offers of shelter in a bunker so we can weather the coming AI rebellion together? I am tom@idiolect.org.uk and on Mastodon at @tomstafford@mastodon.online
Note that these graphs show variation across time and across countries - the previous graphs in this post showed variation across different papers/research teams