Study Finds No Evidence of Bias in First Stage of National Institutes of Health Grant Review Process

A new study from researchers at the University of Wisconsin-Madison has found that bias towards women and Black scientists is unlikely in the initial phase of the process the National Institutes of Health uses to review applications for grants, even though reviewers at this stage are aware of the each applicant’s identity.

For the study, researchers selected 48 actual grant proposals sent to NIH and stripped them of any identifying information. Each study was reproduced four times with new fictitious names and information implying the applying scientist was a White man, White woman, Black man, or Black woman. Next, they recruited more than 400 scientists with grant application experience to serve as reviewers for the experiment. Each reviewer received three of the experiment’s grant applications, two of which where seemingly written by White men, and one with names reworked to appear as authored by a Black woman, Black man, or White woman.

The reviewers read the applications and returned detailed critiques as they would in an actual NIH review and gave the application scores in various areas. There was no consequential difference in the scores or the reviewers’ use of descriptive language that can be consequential for how grants are perceived in their reports.

“The social science literature tells us that when things are ambiguous, men tend to get a bit of a bonus while women and Black people are kind of downgraded,” says co-author Patricia Devine. “We didn’t see that in the reviews supplied in this experimental study.”

The researchers believe the lack of bias at this step in the review process could be because this first assessment is an in-depth affair, with long, written justifications for judgement that don’t lend themselves well to the usual trappings of bias. In later steps in the application process, which can involve shallower analysis, sometimes brief debate and less individual accountability for reviewers, it’s more likely bias could creep in.

“But we don’t want to be understood to suggest that we don’t think there is any bias in the process,” says Devine. “If there is bias, I don’t know yet where it is and how it manifests.”

The full study, “Little Race of Gender Bias in an Experiment of Initial Review of NIH R01 Grant Proposals,” was published in the journal Nature Human Behavior. It may be accessed here.