The paper's rule sets the rival aside, at least provisionally, when the uniform-weights Bayes factor is 20 or more and the worst-case Bayes factor is above 1. A reader who doubts the result may ask whether the search for evidence favored the working theory. Write \(\omega\) for the bias factor: a search with bias \(\omega\) is \(\omega\) times as likely to find a pro-working-theory observation as a pro-rival one. Granting more bias lowers both Bayes factors, because a search that finds pro-working-theory observations more easily would produce mostly pro-working-theory counts even from evidence that mostly favored the rival.
Arguments
- y_W
Non-negative whole number. Count, or summed weight, of observations supporting the working theory.
- y_R
Non-negative whole number. Count, or summed weight, of observations supporting the rival.
- threshold_uniform
Positive number. The uniform-weights Bayes factor must be at least this large for the rule to set the rival aside. Default
20.- threshold_worst
Positive number. The worst-case Bayes factor must be above this value for the rule to set the rival aside. Default
1.
Value
A list with elements
bf_uniform,bf_worstThe two Bayes factors at an unbiased search.
omega_uniformThe bias at which the uniform-weights Bayes factor falls to
threshold_uniform.omega_worstThe bias at which the worst-case Bayes factor falls to
threshold_worst.omega_starThe smaller of the two.
Details
This function returns the bias at which each condition of the rule fails and the smaller of the two, which the paper writes \(\omega^\star\): the smallest bias at which a researcher following the rule would no longer set the rival aside. At nine observations for the working theory and three for the rival, the uniform-weights value falls to 20 at a bias of 1.0098 and the worst-case value falls to 1 at 1.559, so \(\omega^\star = 1.01\): a search about one percent more likely to find pro-working-theory observations would change the decision. With the smoking-gun memo counted as ten observations, the counts are 18 and 3 and \(\omega^\star = 2.33\).
Whether a search could have been that lopsided is a question about how the evidence was collected, which readers who know the case can answer and the function cannot.
Special values
A tipping point is 0 when its condition already fails at an unbiased
search, because then there is no decision for bias to change. It is
Inf when no bias makes the condition fail: with no observations for
the rival, both Bayes factors fall toward 1 as the bias grows and never
reach it, so a threshold of 1 or less is never crossed. It is NA when
the search, which tries biases up to about \(10^{12}\), finds no
crossing; omega_star then reports the other tipping point.
See also
bf_uniform_weights() and bf_worst_case(), whose omega
argument this function varies; sens_coding() for sensitivity to
coding.
Examples
# The paper's running example.
sens_bias(9, 3)
#> $bf_uniform
#> [1] 20.67196
#>
#> $bf_worst
#> [1] 2.732168
#>
#> $omega_uniform
#> [1] 1.009794
#>
#> $omega_worst
#> [1] 1.559262
#>
#> $omega_star
#> [1] 1.009794
#>
# The memo counted as ten observations.
sens_bias(18, 3)$omega_star
#> [1] 2.334075