OPA Publications

OPA regularly publishes advances in methodology and metascience research in internationally recognized peer-reviewed journals. Access all OPA publications below.

The effect of mentee and mentor gender on scientific productivity of applicants for NIH training fellowships (February 3, 2021)

The OPA team has posted a research paper examining the role of mentoring, and in particular mentor gender, on the productivity of scientists early in their careers. Such efforts have been limited in the past due to difficulties in unambiguously linking mentees to their mentors, and measuring the research productivity resulting from those relationships. OPA uses its novel author disambiguation method to investigate the role of self-identified gender in mentorship of trainees applying for NIH fellowships, applying a multi-dimensional framework to assess research productivity. The analysis finds that, after normalizing for the funding level of mentors, the productivity of female and male mentees is indistinguishable, and is independent of the mentor gender, other than in measures of clinical impact, where women mentored by women outperform other mentee-mentor dyads. Read the article here.

It has also been posted to bioRxivdisplay-none.

NIH Funding for Surgeon-Scientists in the US: What Is the Current Status? (February 12, 2021)

Together with our collaborators in the Surgical Oncology Program at NCI, OPA published an analysis of NIH funding for surgeon scientists. The analysis revealed that there has been an increase in NIH-funded surgeon scientists between 2010 and 2020 and that these investigators have diversified their interests over time to include more clinical research in addition to basic science. 

Read the research articledisplay-none and an invited commentarydisplay-none contextualizing the results published in February 2021 in the Journal of the American College of Surgeons.

The NIH Open Citation Collection: A public access, broad coverage resource (October 10, 2019)

Bibliometric analyses focused on determining the impact of a portfolio of grants or publications rely on accurate, reliable citation data. Historically, citation data have remained locked behind restrictive licensing agreements, hampering the ability of researchers to identify reference linkages between scientific articles. To address this barrier, the NIH Open Citation Collection (NIH-OCC) was created using unrestricted data sources and full-text articles that have been made freely available on the internet. This dataset underlies the updated version of iCite and has been made publicly available in the Open Cites module. These data can be used to perform reproducible, trustworthy bibliometric analyses.

Methodology used to create the NIH-OCC was published in PLOS Biology. Full article access is available heredisplay-none.

Link to iCite 2.0

Predicting translational progress in biomedical research (October 10, 2019)

Fundamental research can take decades to translate into clinical outcomes. To reduce this time interval and speed up the discovery of human therapeutics, a machine learning model was created by OPA researchers to accurately predict whether a scientific publication will have an impact on clinical research. These studies, recently published in PLOS Biology, demonstrate that with as little as two years of post-publication citation data, OPA scientists were able to accurately predict whether an article will eventually be cited by a clinical article (clinical trial or guideline). This article-level metric, the Approximate Potential for Translation (APT), can be used by decision-makers hoping to identify research with a high likelihood to contribute to clinical outcomes and is available to the public in the Translation module of the OPA-created iCite 2.0 tool.

Full article access is available heredisplay-none.

Link to iCite 2.0

Topic choice contributes to the lower rate of NIH awards to African-American/Black scientists (October 9, 2019)

In a study published in the journal Science Advances, researchers with the NIH Office of Portfolio Analysis (OPA) found that African-American/black (AA/B) scientists that apply for NIH R01 funding are more likely to study topics with lower funding rates. Topic choice alone accounted for 20% of the observed funding gap after controlling for multiple variables. Community and population-level research tend to receive more AA/B applicants compared with fundamental/mechanistic topics, the latter tend to have higher award rates. These findings can be used to assist with the direction of future funding priorities within the NIH.

Full article access is available heredisplay-none.

Article-level assessment of influence and translation in biomedical research (October 13, 2017)

The OPA Director discusses next generation portfolio analysis, and using RCR as one component of a diverse, multifaceted assessment.

Full article access is available heredisplay-none.

Additional support for RCR: a validated article-level measure of scientific influence (October 2, 2017)

Response to critique of the Relative Citation Ratio (RCR), objecting to the construction of both the numerator and denominator of the metric. While we strongly agree that any measure used to assess the productivity of research programs should be thoughtfully designed and carefully validated, we believe that the specific concerns outlined in the correspondence are unfounded.

Full article access is available heredisplay-none.

NIH scientists develop new metric to measure influence of scientific research (September 6, 2016)

Relative Citation Ratio (RCR) press release.

Link to press release

Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level (September 6, 2016)

We describe here an improved method to quantify the influence of a research article by making novel use of its co-citation network to field-normalize the number of citations it has received.

Full article access is available heredisplay-none.

This page last reviewed on