The problem
A researcher classifying within-case evidence for a working theory () against a single rival () wants to summarize the weight of evidence with a Bayes factor. The challenge is that the per-observation probabilities a Bayes factor requires are hard to motivate without overcommitting. DrWrinch supplies two fully specified generative models that produce those probabilities, both tilted against by construction, so the reported Bayes factor is a lower bound on the evidence in favor of the working theory.
This vignette works the running example from Lopez, Bowers, and Gajardo Cooper (2026): seven pieces of evidence favoring and three favoring .
Two models
Binomial model. Each observation is an independent Bernoulli draw from an infinite evidence universe with success probability . corresponds to . Under a uniform prior on , the Bayes factor is the ratio of posterior masses on the two sides of . This model fits research designs where the evidence universe is open-ended: ongoing interviews, an expanding archive.
bf_binomial(y_W = 7, y_R = 3)
#> [1] 7.827586Hypergeometric urn model (Formulation C). Evidence is drawn without replacement from a finite urn. The Working Theory Favorable (WTF) sub-model has urn composition ; the Rival Theory Favorable (RTF) sub-model has urn composition . The construction adds one unobserved pro- item to the working theory’s urn and tilts the rival’s urn toward by exactly one item. This model fits research designs where the evidence base is bounded: a closed historical archive, a fixed roster of documents.
bf_urn(y_W = 7, y_R = 3)
#> [1] 39The hypergeometric BF of 39 has a closed form: .
When does each model apply?
| Design | Model |
|---|---|
| Ongoing interviews, growing archive, open-ended fieldwork | bf_binomial() |
| Closed historical archive, fixed roster of documents | bf_urn() |
When in doubt, compute both. They report different but compatible quantities: the binomial conditions on the evidence universe being infinite; the urn conditions on it being bounded.
A property worth knowing
Under symmetric evidence (), both Bayes factors are exactly – no evidence in either direction.
bf_binomial(5, 5)
#> [1] 1
bf_urn(5, 5)
#> [1] 1For bf_urn() this is the central property distinguishing
Formulation C from naive constructions: the urn model returns 1 rather
than
or
at the equipoise point.
Sensitivity to observation bias
A reviewer might ask: how robust is the conclusion to the possibility
that
pro-
items were more likely to be observed than
pro-
items? sens_urn() reports the smallest observation-bias
factor
at which the Bayes factor first drops below a decision threshold (the
default threshold is
,
the conventional cutoff for “strong” evidence).
s_urn <- sens_urn(7, 3, threshold = 20)
s_urn$bf
#> [1] 39
s_urn$omega_star
#> [1] 1.304023The reading: pro- items would have had to be roughly 1.3 times more likely to be noticed than pro- items before the conclusion reverses.
Sensitivity to the prior on
For the binomial model, sens_binomial() additionally
reports the smallest rival-tilted prior at which the conclusion
reverses. The prior
posits
pseudo-observations all favoring the rival; M_star is the
smallest such
that drives the Bayes factor below threshold.
s_binom <- sens_binomial(17, 3, threshold = 20)
s_binom$bf
#> [1] 1341.607
s_binom$omega_star
#> [1] 2.185972
s_binom$M_star
#> [1] 6In this stronger example (17 pro- items, 3 pro- items), neither the observation bias nor a moderate number of rival-favoring pseudo-observations is enough to overturn the conclusion.
Weighted evidence
A researcher who believes one observation carries more probative weight than the others can sum integer weights and pass the totals:
# Six pro-H1 items at weight 1 and one "smoking gun" at weight 10
w_W <- c(10, rep(1, 6))
w_R <- rep(1, 3)
bf_binomial(sum(w_W), sum(w_R))
#> [1] 775.148
bf_urn(sum(w_W), sum(w_R))
#> [1] 1023512The “+1” rival-favoring construction goes through unchanged. See the paper’s discussion of probative weight for the rationale.