Making research funding a lottery could help tackle ‘status bias’

The writer is a science commentator

Star names wield an outsize influence over research, as well as in sport and entertainment. A recent analysis revealed that a research paper written jointly by a Nobel laureate and a novice was rejected by 65 per cent of reviewers when only the novice’s name was made visible as the corresponding author — but by just 23 per cent if the laureate’s name was used instead.

One might argue that “status bias”, also called the Matthew effect, makes for a reasonable short-cut in decision-making, given that prizes are one benchmark of quality. But findings like these chime with persistent concerns that established names and institutions are unfairly crowding out newer research talent when it comes to publishing papers and winning grants.

Now, two UK funding agencies, the British Academy and the Natural Environment Research Council, will try to counter that bias by awarding some of their research grants by lottery. Given that about £15bn of public money is spent annually on research and development in the UK, the move should prompt other funding agencies to rethink how they can best spread their bets.

Most agencies usually use peer review to pick winners and losers: a selection panel in a particular research field appraises the applicants (their peers, who can also sometimes be rivals). Panels can, however, be as prone to confirmation bias, motivated reasoning and tribalism as other groups. Nesta, which describes itself as the UK’s innovation agency for social good, says peer review can be “biased against radical new thinking, geography, ideology and gender”.

It certainly tends to reward seniority over youth, orthodoxy over originality, and incrementalism over conceptual leaps. That is why the British Academy, which champions research in humanities and social sciences, is piloting a “partial randomisation” approach for its Small Research Grants programme, which receives twice the number of applications it can fund. Selection will become a two-stage process. First, proposals must meet a minimum-quality threshold, decided by peer review. All those making the grade will enter a lottery, with grants of up to £10,000 allocated using a random number generator.

“We modelled what would have happened with our small grants if we had allocated them via partial randomisation over the past three years,” says Hetan Shah, the academy’s chief executive, “[and] they would have been more geographically and institutionally dispersed, outside of the ‘golden triangle’ of Oxford, Cambridge and London.” That modelling partly clinched the academy’s decision to pilot a lottery. Shah additionally hopes the strategy, to take effect next year, will lighten panel workloads and nurture ideas that challenge current thinking.

Both the British Academy and NERC have drawn on work by the Research on Research Institute at Sheffield university, an organisation dedicated to improving the research landscape. Professor James Wilsdon, its director, says that randomised allocation can particularly help with “grey zone” proposals: those not stellar enough to earn unanimous approval but not shoddy enough to be junked outright.

In that situation, yes-or-no calls can end up being made on very fine margins, with the difference sometimes boiling down to biases. “Panel members might say, ‘Oh, she’s had grant money before so she must be good’, or, ‘He’s at Oxford so he must be brilliant’,” Wilsdon says, revealing it can take just one sceptic to scupper a proposal’s chances.

Random allocation, he maintains, can be fairer — and easier on those who miss out. Unlucky applicants can be reassured that they were just that: unlucky, not undeserving. The potential downsides are that abandoning merit-based rankings could undermine the credibility of the research enterprise and foster public mistrust. Shah said the scheme would be evaluated over the coming years and that, so far, the reaction had been surprisingly positive.

Lotteries have been adopted by a handful of progressive grant-giving bodies, including the Volkswagen Foundation, the Swiss National Science Foundation and the Health Research Council of New Zealand. It is encouraging to see British organisations testing the waters. Brexit has all but cut off the UK from the EU’s R&D largesse. This makes it essential to find ways of stimulating more innovation from the country’s shrinking financial pot.