Curate Science

Accelerating the growth of cumulative knowledge and innovation by facilitating and incentivizing the independent verification of published scientific findings.

Learn More

Independent verification is the cornerstone of science, however, in practice, the independent verification of published findings is difficult and not rewarded. Hence, gauging the reliability and validity of published scientific findings is typically very difficult (if not impossible). Curate Science is a web application that aims to help in these regards by facilitating and incentivizing the independent verification of published scientific findings in several respects.

Independent verification is facilitated by enabling researchers to (1) link independent replications to original studies (and eventually linking conceptual replication studies meta-analytically), (2) curate available data/syntax to verify reproducibility of results within the browser, and (3) critically evaluate the interpretation/validity of findings via in-line comments (i.e., post-publication peer review on stereoids given evaluation done in the context of available data and replicability information).

Curate Science will incentivize independent verification by (1) prominently displaying recent user contributions on the homepage to reward researchers and (2) offering unique high-utility features that motivate users to regularly use and engage with site content (e.g., notifications of new replications, new activity on articles a user has contributed to, and discovery of new related articles). Furthermore, knowing that other researchers can benefit from one's independent verification activities acts as an incentive to actually engage in those activities. For example, a researcher is more likely to verify the reproducibility of a study's results if they know other researchers' may benefit from those activities (e.g., researchers wanting to re-analyze a study's data from a different theoretical perspective can greatly benefit from a researcher who curated a study's data files to verify reproducibility). See Features, for a flowchart of our core features.

Lightning-fast Search with Auto-complete

Our homepage will feature a lightning-fast search with auto-complete so that you can quickly find what you're looking for. To browse, you can select from the Most Curated or Recently Updated articles lists.

1 search page

Innovative Search Results Page

Easily find relevant articles via icons that indicate availability of data/syntax, materials, replication studies, reproducibility info, and pre-registration info. Looking for articles that have specific components available? Use custom filters to only display those articles (e.g., only display articles with available data/syntax)!

2 search results

Article Page: Putting it all Together

Our flagship feature is the consolidation and curation of key information about published articles, which all come together on the article page. The page will feature automatically updating effect size graphs, in-browser R analyses to verify to reproducibility of results, editable fields to add, modify, or update study information, and element-specific in-line commenting.

3 article page 03

User Profile Dashboard Page

The user dashboard will display a user's recent contributions, a list of their own articles, reading and analyses history, recent activities by other users, and notifications customization.

4 profile dashboard
Main Team
Lebel o Kyllo o Hasselman o
Etienne P. LeBel
Founder & Lead
Alex Kyllo
Technical Advisor
Fred Hasselman
Lead Statistician


Advisory Board
Borsboom o Pashler o Simons o Holcombe o Wagenmakers o
Denny Borsboom
University of Amsterdam
Hal Pashler
University of California - San Diego
Daniel Simons
University of Illinois
Alex Holcombe
University of Sydney
E-J Wagenmakers
University of Amsterdam

Roberts o

Eich o

Kievit o

John o

Earp o
Brent Roberts
University of Illinois - Urbana-Champaign
Eric Eich
University of British Columbia
Rogier Kievit
University of Cambridge
Leslie John
Harvard University
Brian Earp
Oxford University

Schimmack o

Vazire o

Cleeremans o

Donnellan o

Lucas o
Uli Schimmack
University of Toronto
Simine Vazire
Washington University in St. Louis
Axel Cleeremans
Universite Libre de Bruxelles
Brent Donnellan
Michigan State University
Richard Lucas
Michigan State University

Perugini o

Brandt o

Grossmann o

Finkel o

Cesario o
Marco Perugini
University of Milan-Bicocca
Mark Brandt
Tilburg University
Igor Grossmann
University of Waterloo
Eli Finkel
Northwestern University
Joe Cesario
Michigan State University

Dijksterhuis o

Simpson o

Dehouwer o

Campbell o
Ap Dijksterhuis
Radboud University Nijmegen
Jeffry Simpson
University of Minnesota
Jan De Houwer
Ghent University
Lorne Campbell
Western University


Foundational Members
Battista o Coe o Demjanenko o
Christian Battista
Technical Advisor
Ben Coe
Technical Advisor
Stephen Demjanenko
Technical Advisor
Who is Curate Science's intended audience?
Our primary audience is the community of academic, government, and industry researchers. However, we are designing the website so that the organized information is also useful to students, educators, journalists, and public policy makers (e.g., a journalist could look up an article to see whether limitations/flaws have been identified enabling them to write a more balanced news article). Our initial focus involves published articles in the life and social sciences (starting with psychology/neuroscience), however we may eventually expand to other areas.
What is curation?
Digital curation is the process of selecting, filtering, and extracting information as to increase its quality. Curate Science organizes, selects, filters, and extracts information from a diverse set of sources with the goal of increasing the quality of fundamental information about scientific articles.
Who can access and consume information about articles on Curate Science?
Anyone, including non-registered users, can lookup information on Curate Science.
Who can add, modify, and update article information on Curate Science?
Only registered users will be able to add, modify, and update information about articles. That being said, anyone that is affiliated with a research organization can become a registered user, as long as they provide their real names, email address, affiliation, and title (e.g., post-doc, graduate student, undergraduate student, research assistant).
How will you ensure quality control of the information posted about articles on Curate Science?
We will employ a two-stage verification process for some of the information whereby information initially posted will be labeled as "unverified" until a second user confirms it, at which time it will appear as verified. This will be the case for key statistics, independent replication information, and publication bias indices. Like Wikipedia, we will also have a revision history for each editable/updatable field showing which user changed what information on what date (and any notes regarding the edit left by the user).
Is it really feasible to organize and curate information for scientific articles? In other words, why would researchers be willing to spend their precious time curating information on Curate Science?
Our view is that researchers should be highly motivated to add and update information regarding published articles in their own area of research because there is an intrinsic interest to update the scientific record to more accurately reflect the totality of the evidence. We also expect -- just like what happened with Wikipedia -- that more influential and/or controversial articles will be curated first given that a large number of researchers are interested in these articles. Other articles will likely be curated commensurate to the level of interest commanded by the market, though of course article authors are free to curate their own articles as much as they want (and for good reasons, e.g. available data citation advantage, see Piwowar & Vision, 2013)
What is Curate Science's business model?
We will soon be incorporating as a non-profit organization. Curate Science organizes and curates fundamental information about unpublished and published scientific articles and we see this information as a public good that belongs to the community of scientists. Consequently, information organized on Curate Science is and will forever remain a free public good accessible to anyone with an internet connection.
Who is funding Curate Science?
We have received a $10,000 USD seed grant from the Center for Open Science to help with initial development. Wellspring Advisors LCC have confirmed that they have allocated funds to support Curate Science as part of a renewal grant to be given to the Center for Open Science. Templeton Foundation has accepted our initial grant proposal and we will soon be submitting a full proposal to them. We are also currently in discussions with Sloan Foundation.
Is Curate Science associated with the Open Science Framework hosted by the Center for Open Science?
Though Curate Science has formed an informal partnership with the Center for Open Science (with respect to funding, see above), our web application is completely independent from the Open Science Framework (though we will of course be linking to available data, materials, and pre-registration information from the OSF).
How is Curate Science different from Figshare.com, DataDryad.org, and Harvard's Dataverse?
Figshare.com, DataDryad.org, and Harvard's Dataverse (and many other similar websites) are data repositories where researchers can make their data, syntax, and materials publicly available and get credit for doing so. Curate Science organizes, consolidates, and curates all of this publicly available information from as many different sources as possible at the study-level for all published articles with a DOI. Curate Science also provides a platform for the crowd to verify analyses and post comments regarding specific issues such as reproducibility of analyses, problems with posted materials/stimuli, etc.
How is Curate Science different from the Open Science Framework?
The Open Science Framework is a place for researchers to archive, collaborate on, and pre-register their research projects to facilitate researchers' workflow to help increase the alignment between scientific values and scientific practices (see more details on OSF's about page). In contrast, Curate Science, as its name implies, is focused primarily on the curation of scientific information tied to published articles by providing a platform for users to add, modify, update, and comment on published article's replication and reproducibility information (among other things, see features).
How is Curate Science different from the PsychFileDrawer.org?
PsychFileDrawer.org is a highly useful website that was designed to overcome the pernicious file drawer problem in psychology where researchers can manually upload serious replication attempts whether they succeeded or failed. Curate Science aims to significantly build upon PsychFileDrawer's venerable efforts by automatically identifying as many extant independent replication results as possible (via text mining) and also will provide a simple interface for the crowd to add any missing replication results. Curate Science will also feature an innovative article page that visually depicts the complex inter-relationships between original and replication studies (to facilitate the difficult task of interpreting replication results) and which will also allow the crowd to curate key information about the original and replication studies (in addition to several other features).
Will I be able to post data/materials directly on an article page on Curate Science?
Yes, eventually. Our current focus is to organize and curate the available data/materials that are already hosted by the many public repositories that already exist. However, in our quest to radically simplify the fundamental scientific practice of sharing data/materials, we are working on forging a partnership with a major data repository website so that users can easily drag-and-drop data for their articles via our interface using our partner's infrastructure (indeed, we're currently in discussions with Figshare.com and the OSF in this regard).
Will I be able to post unpublished manuscripts on Curate Science?
Yes, this is possible because many independent replications are not published. That being said, our initial focus is on organizing and curating information for published articles, however, we will eventually allow researchers to post working papers with corresponding data/syntax and materials files as a way to significantly facilitate the pre-publication peer review approach that has been so successful in physics and mathematics (i.e., arxiv.org).
When will I be able to add, modify, and update information on Curate Science?
Registered users will be able to add/modify information once we launch our Beta (a "Beta" is a stage of software development where a product has implemented all of its core features and is ready for testing by a small number of users). Our projected Beta launch is June 2015. If you're interested in being one of the first Science Curators, you can sign up below to be automatically notified when our Beta website version is ready!
We're working hard on a Beta-release of Curate Science, which should be ready in a few months.
Sign-up below to be notifed when the Beta is ready!