Find a p-value given a certain number of observations in favor of the working hypothesis among a total number of observations when there are two types of observations

When an observation can either support a working theory or a rival theory, this function returns the p-value summarizing the evidence against the rival theory provided by the working theory.

Usage

find_p_two_types(
  obs_support,
  total_obs,
  odds = 1,
  weights = NULL,
  interpretation = FALSE
)

Arguments

obs_support: An integer representing the number of observations in favor of the working hypothesis. Must be less than or equal to the total.
total_obs: An integer representing the total number of observations made
odds: The odds of observing a rival versus working-theory observation. This can be interpreted as "bias" in observation. Or "relative ease" of observation.
weights: Double. Default is equal weight for each observation when weights=NULL is rep(1,obs_support). To indicate that one observation should have twice the weight of any other one might use rep(c(2,1),c(1,obs_support-1))
interpretation: Logical. TRUE if the function returns text helping to interpret the result, FALSE (default option) to returns only the p-value

Value

Either a p-value (numeric, scalar) or a list containing the p-value and text containing an interpretation

Details

This function accomodates urns where working theory supporting observations are systematically easier or more difficulty to observe that rival supporting observations (odds). And it currently also supports designs where one observation out of the total supporting the working theory is particularly compelling (a "smoking gun"). For example, if observation 1 is worth 2 other observations, the urn can reflect this by increasing the number of observations supporting the rival theory in the urn, accounting for the weight of that smoking gun observation.

Examples

# Equal probability, 2 kinds of evidence
find_p_two_types(obs_support = 7, total_obs = 10)
#> [1] 0.01864802
# Equal probability, 2 kinds of evidence with interpretation printed
find_p_two_types(obs_support = 7, total_obs = 10, interpretation = TRUE)
#> The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=1 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.0186.
#> $thep
#> [1] 0.01864802
#> 
#> $interp
#> [1] "The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=1 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.0186."
#> 
# Unequal probability, 2 kinds of evidence with interpretation printed
find_p_two_types(obs_support = 7, total_obs = 10, interpretation = TRUE, odds = .5)
#> The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=0.5 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.003.
#> $thep
#> [1] 0.002989537
#> 
#> $interp
#> [1] "The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=0.5 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.003."
#> 
find_p_two_types(obs_support = 7, total_obs = 10, interpretation = TRUE, odds = 2)
#> The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=2 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.0761.
#> $thep
#> [1] 0.07612251
#> 
#> $interp
#> [1] "The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=2 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.0761."
#> 
# Equal probability, Unequal evidentiary weight, 2 kinds of evidence
find_p_two_types(
  obs_support = 7, total_obs = 10, weights = rep(1, 7),
  interpretation = TRUE, odds = 1
)
#> The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=1 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.0186.
#> $thep
#> [1] 0.01864802
#> 
#> $interp
#> [1] "The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=1 and evidentiary weights are (1,1,1,1,1,1,1), is p <=0.0186."
#> 
find_p_two_types(
  obs_support = 7, total_obs = 10,
  weights = rep(c(2, 1), c(1, 7 - 1)), interpretation = TRUE, odds = 1
)
#> The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=1 and evidentiary weights are (2,1,1,1,1,1,1), is p <=0.0105.
#> $thep
#> [1] 0.01048951
#> 
#> $interp
#> [1] "The maximum probability of drawing 7 observations which support the working theory from an urn model supporting a rival theory, where the odds of observing working theory information is odds=1 and evidentiary weights are (2,1,1,1,1,1,1), is p <=0.0105."
#>