1. Here, the inverse probability fallacy occurs when someone mistakes the probability of proposition p given proposition q for the probability of proposition q given proposition p. Consider the difference between the probability that someone is a human given that they are a female human (a probability of 100%) versus the probability that someone is a female human given that they are a human (a probability of around 50%). In this sense, someone would commit the fallacy if they believed that the p value specifies that probability that the null hypothesis is true given the results instead of it specifying the probability of obtaining the results if the null hypothesis were true. Consequently, if someone believes that the p value specifies the probability of the null hypothesis given the evidence, then why would they worry about the statistical power—the a priori probability of correctly rejecting the null hypothesis and detecting an effect when it is really present?

2. Like Feest (2016), other philosophers of science may recognize a resemblance between the experimenter’s regress and the Duhem-Quine problem: both problems concern the limitations of evidence insofar as it putatively underdetermines which propositions to accept.

3. Technically, the term \(p(T \mid F_N)\) should instead be \(p(T \mid F_1 \amp F_2 \amp \ldots \amp F_N)\) since \(p(T \mid F_N)\) is the probability of T conditional on only the last replication failure \(F_N\) and not the other failures (unless, of course, by \(p(T \mid F_N)\), they really mean some other probability function \(p\prime\) which has conditionalized on the other failures and would thus be a different probability function than the one used for \( p(T))\). The other replications failures would ideally feature in the antecedent conditions of particular other conditional probabilities too. Regardless, we have reproduced Earp and Trafimow’s presentation of the Bayesian framework here despite these technical details.

4. Technically, this is only given an additional principle not discussed explicitly by Earp and Trafimow. This is the principle of conditionalisation whereby one’s so-called posterior degree of belief in a proposition P after the receipt of some evidence E should be equal to their prior degree of belief in P given E before the receipt of evidence E. Earp and Trafimow only explicitly discuss the relations between degrees of belief at a given time (synchronic relations), rather than the relations between degrees of belief at two different points in time (diachronic relations).

5. For example, critics have referred to replication and open science advocates as “replication police” and “shameless little bullies” (Gilbert 2014 online post) and accused “angry mobs” of conducting an “inquisition” (Bohannon 2014 online post). High profile replication studies, such as that of Amy Cuddie’s power poses, have been used in attempts to establish bullying charges (Dominus 2017). In a publicly released draft of a guest column for the Association of Psychological Science (APS) Observer, past APS president Susan Fiske referred to those involved in replication and meta-science work as “methodological terrorists” (Fiske 2016 draft). She was especially scathing of “public shaming and blaming” by “online vigilantes” and “destructo-critics”. In the final published version of this column (Fiske 2016), she retracted some of these phrases, but the draft had already sparked many social media responses from open science advocates who largely view the tone debate as a distraction from their key issues (e.g., Chambers 2017 online post, and Sutton 2018).