Repeat: A framework to assess empirical reproducibility in biomedical research

Leslie D. McIntosh, Anthony Juehne, Cynthia Vitale, Xiaoyan Liu, Rosalia Alcoser, J. Christian Lukas, Bradley Evanoff

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Background: The reproducibility of research is essential to rigorous science, yet significant concerns of the reliability and verifiability of biomedical research have been recently highlighted. Ongoing efforts across several domains of science and policy are working to clarify the fundamental characteristics of reproducibility and to enhance the transparency and accessibility of research. Methods: The aim of the proceeding work is to develop an assessment tool operationalizing key concepts of research transparency in the biomedical domain, specifically for secondary biomedical data research using electronic health record data. The tool (RepeAT) was developed through a multi-phase process that involved coding and extracting recommendations and practices for improving reproducibility from publications and reports across the biomedical and statistical sciences, field testing the instrument, and refining variables. Results: RepeAT includes 119 unique variables grouped into five categories (research design and aim, database and data collection methods, data mining and data cleaning, data analysis, data sharing and documentation). Preliminary results in manually processing 40 scientific manuscripts indicate components of the proposed framework with strong inter-rater reliability, as well as directions for further research and refinement of RepeAT. Conclusions: The use of RepeAT may allow the biomedical community to have a better understanding of the current practices of research transparency and accessibility among principal investigators. Common adoption of RepeAT may improve reporting of research practices and the availability of research outputs. Additionally, use of RepeAT will facilitate comparisons of research transparency and accessibility across domains and institutions.

Original languageEnglish (US)
Article number143
JournalBMC Medical Research Methodology
Volume17
Issue number1
DOIs
StatePublished - Sep 18 2017

Fingerprint

Biomedical Research
Research
Data Mining
Manuscripts
Information Dissemination
Electronic Health Records
Documentation
Publications
Research Design
Research Personnel
Databases

All Science Journal Classification (ASJC) codes

  • Epidemiology
  • Health Informatics

Cite this

McIntosh, Leslie D. ; Juehne, Anthony ; Vitale, Cynthia ; Liu, Xiaoyan ; Alcoser, Rosalia ; Lukas, J. Christian ; Evanoff, Bradley. / Repeat : A framework to assess empirical reproducibility in biomedical research. In: BMC Medical Research Methodology. 2017 ; Vol. 17, No. 1.
@article{9bf274b752034d52a7c367092994f39a,
title = "Repeat: A framework to assess empirical reproducibility in biomedical research",
abstract = "Background: The reproducibility of research is essential to rigorous science, yet significant concerns of the reliability and verifiability of biomedical research have been recently highlighted. Ongoing efforts across several domains of science and policy are working to clarify the fundamental characteristics of reproducibility and to enhance the transparency and accessibility of research. Methods: The aim of the proceeding work is to develop an assessment tool operationalizing key concepts of research transparency in the biomedical domain, specifically for secondary biomedical data research using electronic health record data. The tool (RepeAT) was developed through a multi-phase process that involved coding and extracting recommendations and practices for improving reproducibility from publications and reports across the biomedical and statistical sciences, field testing the instrument, and refining variables. Results: RepeAT includes 119 unique variables grouped into five categories (research design and aim, database and data collection methods, data mining and data cleaning, data analysis, data sharing and documentation). Preliminary results in manually processing 40 scientific manuscripts indicate components of the proposed framework with strong inter-rater reliability, as well as directions for further research and refinement of RepeAT. Conclusions: The use of RepeAT may allow the biomedical community to have a better understanding of the current practices of research transparency and accessibility among principal investigators. Common adoption of RepeAT may improve reporting of research practices and the availability of research outputs. Additionally, use of RepeAT will facilitate comparisons of research transparency and accessibility across domains and institutions.",
author = "McIntosh, {Leslie D.} and Anthony Juehne and Cynthia Vitale and Xiaoyan Liu and Rosalia Alcoser and Lukas, {J. Christian} and Bradley Evanoff",
year = "2017",
month = "9",
day = "18",
doi = "10.1186/s12874-017-0377-6",
language = "English (US)",
volume = "17",
journal = "BMC Medical Research Methodology",
issn = "1471-2288",
publisher = "BioMed Central",
number = "1",

}

Repeat : A framework to assess empirical reproducibility in biomedical research. / McIntosh, Leslie D.; Juehne, Anthony; Vitale, Cynthia; Liu, Xiaoyan; Alcoser, Rosalia; Lukas, J. Christian; Evanoff, Bradley.

In: BMC Medical Research Methodology, Vol. 17, No. 1, 143, 18.09.2017.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Repeat

T2 - A framework to assess empirical reproducibility in biomedical research

AU - McIntosh, Leslie D.

AU - Juehne, Anthony

AU - Vitale, Cynthia

AU - Liu, Xiaoyan

AU - Alcoser, Rosalia

AU - Lukas, J. Christian

AU - Evanoff, Bradley

PY - 2017/9/18

Y1 - 2017/9/18

N2 - Background: The reproducibility of research is essential to rigorous science, yet significant concerns of the reliability and verifiability of biomedical research have been recently highlighted. Ongoing efforts across several domains of science and policy are working to clarify the fundamental characteristics of reproducibility and to enhance the transparency and accessibility of research. Methods: The aim of the proceeding work is to develop an assessment tool operationalizing key concepts of research transparency in the biomedical domain, specifically for secondary biomedical data research using electronic health record data. The tool (RepeAT) was developed through a multi-phase process that involved coding and extracting recommendations and practices for improving reproducibility from publications and reports across the biomedical and statistical sciences, field testing the instrument, and refining variables. Results: RepeAT includes 119 unique variables grouped into five categories (research design and aim, database and data collection methods, data mining and data cleaning, data analysis, data sharing and documentation). Preliminary results in manually processing 40 scientific manuscripts indicate components of the proposed framework with strong inter-rater reliability, as well as directions for further research and refinement of RepeAT. Conclusions: The use of RepeAT may allow the biomedical community to have a better understanding of the current practices of research transparency and accessibility among principal investigators. Common adoption of RepeAT may improve reporting of research practices and the availability of research outputs. Additionally, use of RepeAT will facilitate comparisons of research transparency and accessibility across domains and institutions.

AB - Background: The reproducibility of research is essential to rigorous science, yet significant concerns of the reliability and verifiability of biomedical research have been recently highlighted. Ongoing efforts across several domains of science and policy are working to clarify the fundamental characteristics of reproducibility and to enhance the transparency and accessibility of research. Methods: The aim of the proceeding work is to develop an assessment tool operationalizing key concepts of research transparency in the biomedical domain, specifically for secondary biomedical data research using electronic health record data. The tool (RepeAT) was developed through a multi-phase process that involved coding and extracting recommendations and practices for improving reproducibility from publications and reports across the biomedical and statistical sciences, field testing the instrument, and refining variables. Results: RepeAT includes 119 unique variables grouped into five categories (research design and aim, database and data collection methods, data mining and data cleaning, data analysis, data sharing and documentation). Preliminary results in manually processing 40 scientific manuscripts indicate components of the proposed framework with strong inter-rater reliability, as well as directions for further research and refinement of RepeAT. Conclusions: The use of RepeAT may allow the biomedical community to have a better understanding of the current practices of research transparency and accessibility among principal investigators. Common adoption of RepeAT may improve reporting of research practices and the availability of research outputs. Additionally, use of RepeAT will facilitate comparisons of research transparency and accessibility across domains and institutions.

UR - http://www.scopus.com/inward/record.url?scp=85029593391&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029593391&partnerID=8YFLogxK

U2 - 10.1186/s12874-017-0377-6

DO - 10.1186/s12874-017-0377-6

M3 - Article

C2 - 28923006

AN - SCOPUS:85029593391

VL - 17

JO - BMC Medical Research Methodology

JF - BMC Medical Research Methodology

SN - 1471-2288

IS - 1

M1 - 143

ER -