More Complex Designs: Interaction Effect

Playboy Effect: Kenrick, Gutierres, & Goldberg (1989) -- Replications (3)  
Influence of Popular Erotica on Judgments of Strangers and Mates 

Original Studies & Replications N Effect size (Δd) [95% CI]
Kenrick et al. (1989) Study 2 65
Balzarini et al. (2015) Study 1 223
Balzarini et al. (2015) Study 2 263
Balzarini et al. (2015) Study 3 225
Random-effects meta-analytic estimate of all replications:
[Underlying data (CSV)] [R-code]
Original Studies & Replications Independent Variables Dependent Variables Design Differences Active Sample Evidence
Kenrick et al. (1989) Study 2 Playboy centerfolds vs. control
Participant sex
Love for partner (Rubin Love-scale) -
Balzarini et al. (2015) Study 1 Playboy centerfolds vs. control
Participant sex
Love for partner (Rubin Love-scale) Updated pictures of abstract art & male/female nudes (as suggested by Kenrick)
Two attention check questions (online sample)
Nudes rated as more pleasant than abstract art
Balzarini et al. (2015) Study 2 Playboy centerfolds vs. control
Participant sex
Love for partner (Rubin Love-scale) Updated pictures of abstract art & male/female nudes (as suggested by Kenrick)
Two attention check questions (online sample)
Nudes rated as more pleasant than abstract art
Balzarini et al. (2015) Study 3 Playboy centerfolds vs. control
Participant sex
Love for partner (Rubin Love-scale) Updated pictures of abstract art & male/female nudes (as suggested by Kenrick)
Two attention check questions (online sample)
Nudes rated as more pleasant than abstract art
[Underlying data (CSV)] [R-code]

Summary: There appears to be replication difficulties...

Original authors' and replicators' comments: TBA

Related Commentary

More Complex Designs: Multiple Outcomes

Crowd-Within Effect: Vul & Pashler (2008) -- Replications (3)  
Measuring the Crowd Within: Probabilistic Representations Within Individuals 

Summary: The crowd-within effect appears to be replicable...

Original authors' and replicators' comments: TBA

Related Commentary

Pre-viewing Open Data

RRR1 & RRR2: Alogna et al., ..., Zwaan (2014)  
Schooler & Engstler-Schooler (1990) -- Replications (23)   
Verbal overshadowing of visual memories: Some things are better left unsaid
DOI:10.1016/0010-0285(90)90003-M  

Original Studies & Replications Data/Syntax Materials/Pre-reg N Effect size [95% CI]
Schooler & Smith (1990) Study 1 approved_protocol.pdf* 88
Michael et al. (2014) Study2Protocol.xlsx approved_protocol.pdf* 615
Alogna et al. (2014) lab_data_2.xlsx approved_protocol.pdf* 137
Random-effect meta-analytic estimate of all lab replications:
[Underlying data (CSV) & R-code]

Summary: The verbal overshadowing effect appears to be replicable; verbally describing a robber after a 20-minute delay decreased correct identification rate in a lineup by 16% (from 54% [control] to 38% [verbal]; meta-analytic estimate = -16% [+/-.04], equivalent to r = .17). Still in question, however, is the validity and generalizability of the effect, hence it's still premature for public policy to be informed by verbal overshadowing evidence. Validity-wise, it's unclear whether verbal overshadowing is driven by a more conservative judgmental response bias process or driven by a reduced memory discriminability process because no "suspect-absent" lineups were used. This is important to clarify because it directly influences how eye-witness testimony should be treated (e.g., if verbal overshadowing is primarily driven by a more conservative response bias process, identifications made after a verbal descriptions should actually be given *more* [rather than less] weight, see Mickes & Wixted, 2015). Generalizability-wise, in a slight variant of RRR2 (i.e., RRR1), a much smaller overall verbal deficit of -4% [+/-.03] emerged, when the lineup identification occured 20 minutes after verbal description (which occurred immediately after seeing robbery). Future research needs to determine the size of verbal overshadowing when there's a delay between crime and verbal description and before lineup identification, which better reflect real-world conditions.

STATUS: It works, but it'd be better to use modal windows, which is a bit more complicated to implement.

Gauging Analytic Reproducibility of Evidence

RRR1 & RRR2: Alogna et al., ..., Zwaan (2014)  
Schooler & Engstler-Schooler (1990) -- Replications (23)   
Verbal overshadowing of visual memories: Some things are better left unsaid
DOI:10.1016/0010-0285(90)90003-M  

Original Studies & Replications Data/Syntax Materials/Pre-reg N Effect size [95% CI]
Schooler & Smith (1990) Study 1 approved_protocol.pdf* 88
Michael et al. (2014) Study2Protocol.xlsx approved_protocol.pdf* 615
Alogna et al. (2014) lab_data_2.xlsx approved_protocol.pdf* 137
Random-effect meta-analytic estimate of all lab replications:
[Underlying data (CSV) & R-code]

Summary: The verbal overshadowing effect appears to be replicable; verbally describing a robber after a 20-minute delay decreased correct identification rate in a lineup by 16% (from 54% [control] to 38% [verbal]; meta-analytic estimate = -16% [+/-.04], equivalent to r = .17). Still in question, however, is the validity and generalizability of the effect, hence it's still premature for public policy to be informed by verbal overshadowing evidence. Validity-wise, it's unclear whether verbal overshadowing is driven by a more conservative judgmental response bias process or driven by a reduced memory discriminability process because no "suspect-absent" lineups were used. This is important to clarify because it directly influences how eye-witness testimony should be treated (e.g., if verbal overshadowing is primarily driven by a more conservative response bias process, identifications made after a verbal descriptions should actually be given *more* [rather than less] weight, see Mickes & Wixted, 2015). Generalizability-wise, in a slight variant of RRR2 (i.e., RRR1), a much smaller overall verbal deficit of -4% [+/-.03] emerged, when the lineup identification occured 20 minutes after verbal description (which occurred immediately after seeing robbery). Future research needs to determine the size of verbal overshadowing when there's a delay between crime and verbal description and before lineup identification, which better reflect real-world conditions.

STATUS: It works, but it'd be better to use modal windows, which is a bit more complicated to implement.

Allowing Crowd to Add Replications

Unicorn Priming -- Replications (2)
Schooler & Engstler-Schooler (1990)
Verbal overshadowing of visual memories: Some things are better left unsaid

Original Studies & Replications Data/Syntax Materials/Pre-reg N Effect size [95% CI]
Schooler & Smith (1990) Study 1 approved_protocol.pdf* 88
Michael et al. (2014) Study2Protocol.xlsx approved_protocol.pdf* 615
Alogna et al. (2014) lab_data_2.xlsx approved_protocol.pdf* 137
Random-effect meta-analytic estimate of all lab replications:

Social Science Dashboard