The problem
A researcher studying democratic breakdown has read the sources and coded each observation as supporting a working theory or a single rival. Nine observations support the working theory and three support the rival. The researcher wants to say how much more probable those twelve observations are if the working theory holds than if the rival does, in a way a critic cannot call arithmetic chosen to suit the conclusion.
A Bayes factor is that ratio of probabilities. Getting it takes a probability for the counts under each theory, and a theory stated in words does not supply one. Lopez, Bowers, and Gajardo Cooper (2026) derive both probabilities from one model, which this package implements. This vignette works their running example: nine observations for the working theory, three for the rival.
One model
Suppose each observation the researcher might have made supports the working theory with probability . Read as the share of the available evidence that supports the working theory. The working theory claims that share is above one half. The rival claims it is at or below one half.
At a given share, the probability of supporting observations out of is the binomial probability, so every number below is one a reader can recompute.
Why one model gives two Bayes factors
Each theory names a range of shares, and a range is not a probability for the counts. The researcher turns the working theory’s range into a probability by averaging the binomial probability over the shares above one half, giving each share the same weight. That average is the numerator of both Bayes factors.
For the rival there are two answers. The first holds the rival to every share in its range equally and averages the same way. The second gives the rival the share in its range that makes the counts most probable, which is one half when the counts favor the working theory. Both denominators, and the numerator they share, are numbers anyone can compute:
n <- 12
k <- 9
# Averaging over a range half as wide as the interval from 0 to 1 means
# dividing the integral by one half, which is the factor of 2 below.
numerator <- 2 * integrate(function(t) dbinom(k, n, t), 0.5, 1)$value
den_avg <- 2 * integrate(function(t) dbinom(k, n, t), 0, 0.5)$value
den_half <- dbinom(k, n, 0.5)
round(c(numerator = numerator, den_avg = den_avg, den_half = den_half), 5)
#> numerator den_avg den_half
#> 0.14675 0.00710 0.05371Dividing the shared numerator by each denominator gives the two Bayes factors, and the package computes each directly:
c(by_hand = numerator / den_avg, from_package = bf_uniform_weights(9, 3))
#> by_hand from_package
#> 20.67196 20.67196
c(by_hand = numerator / den_half, from_package = bf_worst_case(9, 3))
#> by_hand from_package
#> 2.732168 2.732168The twelve observations give 20.67 under uniform weights and 2.73 in the worst case. The numerator cancels from the ratio of the two Bayes factors, so that ratio is the ratio of the denominators:
round(c(ratio_of_bayes_factors = bf_uniform_weights(9, 3) / bf_worst_case(9, 3),
ratio_of_denominators = den_half / den_avg), 2)
#> ratio_of_bayes_factors ratio_of_denominators
#> 7.57 7.57Nine of twelve is 7.57 times as probable at a share of exactly one half as it is on average across the shares from zero to one half. The average includes shares near zero, under which nine of twelve is very improbable, and those shares are what lift 2.73 to 20.67.
The worst-case value is the smallest Bayes factor any weights on the rival’s shares can give, because no share in the rival’s range gives nine of twelve a higher probability than one half does. A critic who says the researcher chose weights for the rival that flattered the working theory can be answered with that number.
The rule for setting the rival aside
The paper proposes a rule that uses both numbers. The researcher sets the rival aside, at least provisionally, when the uniform-weights Bayes factor is 20 or more and the worst-case Bayes factor is above 1.
The first threshold follows Kass and Raftery (1995), who call a Bayes factor of 20 or more strong evidence. With even prior odds a Bayes factor of 20 puts the posterior probability of the rival at no more than . The second condition says that no weights on the rival’s shares make the counts more probable under the rival than under the working theory. Nine of twelve meets both:
c(uniform_weights = bf_uniform_weights(9, 3), worst_case = bf_worst_case(9, 3))
#> uniform_weights worst_case
#> 20.671958 2.732168With fewer than 101 observations, a uniform-weights value of 20 or more always comes with a worst-case value above 1, so the second condition changes the decision only in larger bodies of evidence. With 59 observations for the working theory and 42 for the rival it does:
c(uniform_weights = bf_uniform_weights(59, 42), worst_case = bf_worst_case(59, 42))
#> uniform_weights worst_case
#> 20.7775540 0.9804411Observations that carry more weight than others
A researcher who judges one observation, say a secret memo, worth ten routine ones enters it as ten observations by summing whole-number weights:
# Eight routine supporting observations plus the memo at weight ten.
w_W <- c(10, rep(1, 8))
w_R <- rep(1, 3)
c(uniform_weights = bf_uniform_weights(sum(w_W), sum(w_R)),
worst_case = bf_worst_case(sum(w_W), sum(w_R)))
#> uniform_weights worst_case
#> 2336.9621 143.2847What a reader would have to grant
A critic can press on the coding and on the search. For each, the package reports how far the critic’s concern would have to go before a researcher following the rule would no longer set the rival aside.
Coding
A peer cannot re-read every document, but can ask how many of the nine supporting observations would have to be re-coded as supporting the rival:
sens_coding(9, 3)
#> $bf_uniform
#> [1] 20.67196
#>
#> $bf_worst
#> [1] 2.732168
#>
#> $x_uniform
#> [1] 1
#>
#> $x_worst
#> [1] 2
#>
#> $x_star
#> [1] 1One re-coding takes the uniform-weights value to 6.49, below 20. The
worst-case value stays above 1 after one re-coding and falls below it
after two. So x_star, the number at which the decision
changes, is one. The conclusion tolerates so little re-coding because
20.67 starts so close to 20.
The search
Write
for the bias factor: a search with bias
is
times as likely to find a pro-working-theory observation as a pro-rival
one. Such a search turns a share
of the evidence into a share
among the observations found, and both Bayes factors take an
omega argument that applies that change.
sens_bias() finds the bias at which each condition of the
rule fails:
sens_bias(9, 3)
#> $bf_uniform
#> [1] 20.67196
#>
#> $bf_worst
#> [1] 2.732168
#>
#> $omega_uniform
#> [1] 1.009794
#>
#> $omega_worst
#> [1] 1.559262
#>
#> $omega_star
#> [1] 1.009794A search about one percent more likely to find pro-working-theory observations would change the decision. With the memo counted as ten observations, the search would have to be more than twice as likely:
sens_bias(18, 3)$omega_star
#> [1] 2.334075Under strong bias the rival’s most favorable share is no longer one
half. The biased search carries the rival’s shares onto shares found of
at most
,
while nine of twelve is most probable at a share found of
.
Once
passes
,
which happens once
passes
,
a share below one half fits the counts better, and
bf_worst_case() uses it:
tilt <- function(t, om) om * t / (om * t + 1 - t)
c(prob_at_half = dbinom(9, 12, tilt(0.5, 5)),
prob_at_0.375 = dbinom(9, 12, tilt(0.375, 5)),
worst_case = bf_worst_case(9, 3, omega = 5))
#> prob_at_half prob_at_0.375 worst_case
#> 0.1973957 0.2581036 0.2146125The tipping points do not depend on this. At the bias where equals , the denominator is the largest probability nine of twelve can have at any share, so the worst-case value is already below 1 there. It therefore falls to 1 at a smaller bias, where one half is still the rival’s most favorable share.
Weights on the shares other than the uniform
Giving every share in a range the same weight is a choice, and a
reader may want a different one. bf_rescaled() takes a Beta
distribution rescaled onto each theory’s half, one pair of parameters
per side; the uniform default is the pair
on both sides. Below one the weight collects at both ends of each half,
including next to one half where the two theories claim nearly the same
share. Above one it collects in the middle of each half, so the two
theories claim shares far apart:
c(uniform = bf_rescaled(9, 3),
next_to_half = bf_rescaled(9, 3, 0.5, 0.5, 0.5, 0.5),
far_apart = bf_rescaled(9, 3, 10, 10, 10, 10))
#> uniform next_to_half far_apart
#> 20.671958 9.526137 252.917211The worst-case Bayes factor takes the same pair for the working theory’s shares only, because the rival stays at its most favorable share whatever weights its proponent might state:
c(uniform = bf_worst_case(9, 3),
next_to_half = bf_worst_case(9, 3, a1 = 0.5, b1 = 0.5),
far_apart = bf_worst_case(9, 3, a1 = 10, b1 = 10))
#> uniform next_to_half far_apart
#> 2.732168 2.070490 4.384581The weights on shares are a different thing from the weights on observations above: those act as extra observations, these say what each theory predicts.
An archive of known size
The binomial model treats the observations as independent draws. A
researcher who reads documents from an archive of
documents does not read any document twice.
bf_uniform_weights_finite() and
bf_worst_case_finite() compute both Bayes factors for that
case:
sapply(c(20, 100, 1000), function(M)
c(M = M,
uniform_weights = bf_uniform_weights_finite(9, 3, M),
worst_case = bf_worst_case_finite(9, 3, M)))
#> [,1] [,2] [,3]
#> M 20.00000 100.000000 1000.000000
#> uniform_weights 162.88462 25.222782 21.048391
#> worst_case 16.84375 3.194593 2.770414Both values fall toward the binomial ones, 20.67 and 2.73, as the archive grows. In every case the paper’s supplement computed, counts whose binomial Bayes factors meet the rule also meet it from an archive of any size, so the researcher need not state .