Science Cleaned Up.

Scientists only trust an empirical finding when (at minimum) it can be consistently replicated by independent researchers in independent samples. Curate Science is a crowdsourced platform to track, organize, and interpret replications of published findings in the social sciences.

UPDATE (December 3, 2017): New searchable table of N=1,008 curated replications (see announcement for details).

Searchable table of N=1,008 replications of 163 effects from the cognitive and social psychology published literature.

Example searches: "RPP" for Reproducibility Project: Psychology replications; "ML1" or "ML3" for Many Labs 1 or 3 replications; "RRR" for Registered Replication Reports; "SP: Spec" for Social Psychology's Special Issue replications; for topical searches, try "priming", "love", "moral" (for replications on morality), "anchoring", or "power posing."

Icon legend: = data; = experimental materials; = pre-registered protocol; = link to evidence collection a replication is part of. Replications can be sorted (in ascending/descending order) by clicking on column headers. Overflow text can be revealed by hovering over cell. For details about replication outcome values hover over cell and see about section. For additional replication study characteristics (& to see hidden imprecise ML1/ML3 replications [N=216], see our public gSheet (see also our GitHub repo & our curated list of large-scale replication efforts).

Evidence collections (or "replication collections") group together replications of specific effects/hypotheses, organizing them by different ways of operationalizing or generalizing an effect. This structure allows enhanced visualization of replication effect sizes via familiar forest plots and depicts meta-analytic effect size estimates of replications for each operationalization of an effect (assuming >1 replication is available). Evidence collections also allow us to curate replications of sets of effects as predicted by a broader theory (e.g., see ego depletion theory evidence collection).

Below are example evidence collections on the following effects/phenomena: cleanliness priming, money priming, Macbeth effect, warmth embodiment, ego depletion, mood boosts helping, verbal overshadowing.

Cleanliness priming -- Replications (7)  

Original Studies & Replications N Effect size (d) [95% CI]
Operationalization #1: Cleanliness priming (scrambled task) reduces severity of moral judgments
Schnall et al. (2008a) Study 1 40
Arbesfeld et al. (2014) 60
Besman et al. (2013) 60
Huang (2014) Study 1 189
Lee et al. (2013) 90
Johnson et al. (2014a) Study 1 208
Johnson et al. (2014b) 736
Current meta-analytic estimate of operationalization #1 replications:
Generalization #1: Cleanliness priming (hand washing) reduces severity of moral judgments
Schnall et al. (2008a) Study 2 43
Johnson et al. (2014a) Study 2 126
[Underlying data (CSV)] [R-code]

Summary : The main finding that cleanliness priming reduces the severity of moral judgments currently exhibits replication difficulties (overall meta-analytic effect: r = -.08 [+/-.13]). In a follow-up commentary, Schnall argued that a ceiling effect in Johnson et al.'s (2014a) studies render their results uninterpretable and hence their replication results should be dismissed. However, independent re-analyses by Simonsohn, Yarkoni, Schönbrodt, Inbar, Fraley, and Simkovic appear to rule out such ceiling effect explanation, hence, Johnson et al.'s (2014a) results should be retained in gauging the replicability of the original cleanliness priming effect. Of course, it's possible "cleanliness priming" may be replicable under different operationalizations, conditions, and/or experimental designs (e.g., within-subjects).

Money priming -- Replications (42)  

Original Studies & Replications N Effect size (d) [95% CI]
Operationalization #1: Exposure to money (scrambled sentence) reduces helping (coding sheets for RA)
Vohs et al. (2006) Study 3 39
Grenier et al. (2012) 40
Generalization #1: Exposure to money (scrambled sentence) strengthen social inequality beliefs (just world beliefs)
Caruso et al. (2013) Study 2 168
Schuler & Wänke (in press) Study 2 115
Rohrer et al. (2015) Study 2 420
Current meta-analytic estimate of generalization #1 replications:
Generalization #2: Exposure to money (scrambled sentence) strengthen social inequality beliefs (social dominance orientation)
Caruso et al. (2013) Study 3 80
Rohrer et al. (2015) Study 3 156
Generalization #3: Exposure to money (instruction background) strengthen social inequality beliefs (fair market ideology)
Caruso et al. (2013) Study 4 48
Rohrer et al. (2015) Study 4 116
Generalization #4: Exposure to money (instruction background) strengthen social inequality beliefs (system justification scale)
Caruso et al. (2013) Study 1 30
Hunt & Krueger (2014) 87
Cheong (2014) 102
Devos (2014) 162
Swol (2014) 96
John & Skorinko (2014) 87
Davis & Hicks (2014) Study 1 187
Kappes (2014) 277
Klein et al. (2014) 127
Packard (2014) 112
Vranka (2014) 84
Cemalcilar (2014) 113
Bocian & Frankowska (2014) Study 2 169
Huntsinger & Mallett (2014) 146
Rohrer et al. (2015) Study 1 136
Schmidt & Nosek (2014, MTURK) 1000
Hovermale & Joy-Gaba (2014) 108
Vianello & Galliani (2014) 144
Schmidt & Nosek (2014, PI) 1329
Bernstein (2014) 84
Adams & Nelson (2014) 95
Rutchick (2014) 96
Vaughn (2014) 90
Levitan (2014) 123
Brumbaugh & Storbeck (2014) Study 1 103
Smith (2014) 107
Kurtz (2014) 174
Brumbaugh & Storbeck (2014) Study 2 86
Wichman (2014) 103
Pilati (2014) 120
Davis & Hicks (2014) Study 2 225
Furrow & Thompson (2014) 85
Bocian & Frankowska (2014) Study 1 79
Brandt et al. (2014) 80
Nier (2014) 95
Woodzicka (2014) 90
Schmidt & Nosek (2014) 81
Morris (2014) 98
Current meta-analytic estimate of generalization #4 replications:
[Underlying data (CSV)] [R-code]

Summary: The claim that incidental exposure to money influences social behavior/beliefs currently exhibits replication difficulties (overall meta-analytic effect: d = -.01 [+/-.05]). This appears to be the case whether money exposure is manipulated via instruction background images (Caruso et al., 2013, Study 1 & 4) or descrambling sentence task (Vohs et al., 2006, Study 3) and whether outcome variable is helping others (Vohs et al., 2006, Study 3), system justification beliefs (Caruso et al., 2013, Study 1), just world beliefs (Caruso et al., 2013, Study 2), social dominance beliefs (Caruso et al., 2013, Study 3), or fair market beliefs (Caruso et al., 2013, Study 4). Of course, it's possible money exposure reliably influences behavior under other (currently unknown) conditions, via other operationalizations, and/or using other experimental designs (e.g., within-subjects).

Macbeth effect -- Replications (11)  

Original Studies & Replications N Effect size (r) [95% CI]
Operationalization #1: Moral purity threat (transcribe text) boosts need to cleanse oneself (cleaning products desirability)
Zhong & Liljenquist (2006) Study 2 27
Earp et al. (2014) Study 3 286
Siev (2012) Study 2 148
Earp et al. (2014) Study 2 156
Siev (2012) Study 1 335
Earp et al. (2014) Study 1 153
Gamez et al. (2011) Study 2 36
Current meta-analytic estimate of operationalization #1 replications:
Generalization #1: Moral purity threat (recall [un]ethical act) boosts need to cleanse oneself (product choice)
Zhong & Liljenquist (2006) Study 3 32
Fayard et al. (2009) Study 1 210
Gamez et al. (2011) Study 3 45
Current meta-analytic estimate of generalization #1 replications: