You are here

Sci-Score, a tool suite to support Rigor and Transparency NIH and Journal Guidelines

Award Information
Agency: Department of Health and Human Services
Branch: National Institutes of Health
Contract: 9R44MH119094-02
Agency Tracking Number: R44MH119094
Amount: $1,496,339.00
Phase: Phase II
Program: SBIR
Solicitation Topic Code: 101
Solicitation Number: PA17-302
Solicitation Year: 2017
Award Year: 2018
Award Start Date (Proposal Award Date): 2018-09-17
Award End Date (Contract End Date): 2020-07-31
Small Business Information
San Diego, CA 92122-1810
United States
DUNS: 080111340
HUBZone Owned: No
Woman Owned: Yes
Socially and Economically Disadvantaged: No
Principal Investigator
 (650) 483-0697
Business Contact
Phone: (650) 483-0697
Research Institution

ProjectSummary While standards in reporting of scientific methods are absolutely critical to producing reproducible sciencemeeting such standards is tedious and difficultChecklists and instructions are tough to follow often resulting in low and inconsistent complianceScientific journals and societies as well as the National Institutes of Health are now actively proposing general guidelines to address reproducibility issuesparticularly in the reporting of methodsbutthetrickierpartistotrainthebiomedicalcommunitytousethesestandardstoeffectivelyTo support new standards in methods reportingespecially the RRID standard for Rigor and Transparency of Key Biological Resourceswe propose to build Sci Score a text mining based tool suite to help authors meet the standardSci Score will provide an automated check on compliance with the RRID standard already implemented by overjournals including CellJournal of Neuroscienceand eLife and other Rigor and transparency standards put forward by the NIHThe innovation behind Sci score is the provision of a scorewhich can be obtained by individual investigators or journalsThis score reflects an aspect of quality of methods reportingWe posit that the score will serve as a tool that investigators can use to compete with themselvesandeachotherthewaytheycurrentlycompeteonmetricsofpopularityi etheH indexIn Phase I of this project and beforeour group has successfully developed a text mining algorithm that can detect antibodiescell linesorganisms and digital resourcesallRRID typesand has created a preliminary scoreWe propose to extend this approach to all research inputslike chemicals and plasmids that are requested as part of Cell pressSTAR Methodshttpwww cell com star methodsWe also propose to build a set of algorithms to detect whether authors discuss the major sources of irreproducibility outlined by NIHincluding investigator blindingproper randomization and sufficient reporting of sex and other biological variablesResource identification along with other quality metrics will be used to score the quality of scientific methods section textIf successfulthe tool could be used by editorsreviewersand investigators to improve thequalityofthescientificpaperOur Phase II specific aims includeenhancing and hardening the core natural language processing pipelines to recognize a broader range of sentences in near real timebuilding a set of modular tools that will be provided for different groups of users to take advantage of the text mining capability we develop in aimAt the end of Phase IIwe should have a commercially viable product that will be able to be licensed to serve the needsofthepublishersandthebroaderresearchcommunity Standardsforscientificmethodsreportingareabsolutelycriticaltoproducingreproduciblesciencebutmeeting suchstandardsisdifficultChecklistsandinstructionsaretoughtofollowoftenresultinginlowandinconsistent complianceTosupportnewstandardsinmethodsreportingespeciallytheRRIDstandardforRigorand TransparencyweproposetobuildSci ScoreatextminingbasedtoolsuitetohelpauthorsmeetthestandardSci ScorewillprovideanautomatedcheckoncompliancewiththeRRIDstandardimplementedbyoverjournalsincludingCellJournalofNeuroscienceandeLifeSci scoreprovidesanautomatedratingthequality ofmethodsreportinginsubmittedarticleswhichprovidesfeedbacktoauthorsreviewersandeditorsonhow toimprovecompliancewithRRIDsandotherstandards

* Information listed above is at the time of submission. *

US Flag An Official Website of the United States Government