The Living Thing / Notebooks :

Publication bias

Replication crises, P-values, bitching about journals etc

The noise mining process

We’re out here everyday, doing the dirty work finding noise and then polishing it into the hypotheses everyone loves. It’s not easy. —- John Schmidt, The noise miners

Multiple testing across a whole scientific field, with a side helping of uneven data release.

This is a complicated issue which I won’t actually engage with right now. Perhaps you’d prefer to see an actual train-wreck in progress in social psychology.

Keywords: “file-drawer process” and the “publication sieve”, which are the large-scale models of how this works in a scientific community and “researcher degrees of freedom” which is the model for how this works at the individual scale.

This is particularly pertinent in social psychology, where it turns out the there is too much bullshit with \(P\leq 0.05\).