Package 'phack'

Title: Detecting p-Hacking using Elliot et al. (2022)
Description: Implements the tests from Elliot et al. (2022) for detecting p-Hacking. The package is essentially a simple wrapper to the code provided in the code and data supplement of the article, with some cosmetical changes. The original code can be found in the code and data supplement of the article. s References Elliott, G., Kudrin, N., & Wüthrich, K. (2022). Detecting p‐Hacking. Econometrica, 90(2), 887-906.
Authors: Sebastian Kranz based on code by Graham Elliott, Nikolay Kudrin and Kaspar Wuethrich
Maintainer: Sebastian Kranz <[email protected]>
License: GPL >= 2.0
Version: 0.1.0
Built: 2024-08-21 03:01:12 UTC
Source: https://github.com/skranz/phack

Help Index

Binomial test for p-hacking

Description

Elliot et al. (2022) show that absent rounding errors, p-hacking and publication bias, the density of p-values across many different tests should be decreasing in the p-value.

Usage

phack_test_binomial(
  p,
  p_min = 0.04,
  p_max = 0.05,
  open_interval = FALSE,
  min_bunch = 3
)

Arguments

p

vector of p-values (should be derounded if there are rounding errors)
p_min

lower bound of the interval used for the test
p_max

upper bound of the interval used for the test
open_interval

if TRUE take p-values from open interval (p_min, p_max).
min_bunch

minimal number of elements of p that have exactly the same p-value in order to show a warning that there seems to be a rounding problem.

Details

This means if we split any interval [p_min, p_max] in the center, a significantly higher proportion in the right half than the left half suggests a violation of the no p-hacking and no publication bias assumption (if there are no rounding errors or p-values are approbriately de-rounded).

This function tests this via a Binomial test.

Cox-Shi histogram test and more general test for K-monotonicity and bounds on [p_min, p_max] interval

Description

For the defaults K=1 and use_bounds=FALSE we have a basic histogram test.

Usage

phack_test_cox_shi(
  p,
  article = NA,
  p_min = 0,
  p_max = 0.15,
  J = 30,
  K = 1,
  use_bounds = FALSE,
  min_bunch = 3
)

Arguments

p

vector of p-values (make sure rounding problems are dealt with)
article

vector of unique article ids for approbriate clustuering
J

number of subintervals
K

degree of K-monotonicity (see Section 4.3 in Elliot et al. 2022)
use_bounds

use bounds or test without bounds (see Appendix A in Elliot et al. 2022)
min_bunch

minimal number of elements of p that have exactly the same p-value in order to show a warning that there seems to be a rounding problem.

Discontinuity test

Description

Discontinuity test

Usage

phack_test_discontinuity(p, c, min_bunch = 3)

Arguments

p

vector of p-values
c

potential discontinuity point
min_bunch

minimal number of elements of p that have exactly the same p-value in order to show a warning that there seems to be a rounding problem.

Fisher's test

Description

Similar to phack_binomial_test but using Fisher's test instead of the binomial test.

Usage

phack_test_fisher(p, p_min, p_max, min_bunch = 3)

Arguments

p

vector of p-values (should be derounded if there are rounding errors)
p_min

lower bound of the interval used for the test
p_max

upper bound of the interval used for the test
min_bunch

minimal number of elements of p that have exactly the same p-value in order to show a warning that there seems to be a rounding problem.

Details

For this test the half-open interval [p_min, p_max) is used.

LCM test on [p_min, p_max]

Description

LCM test on [p_min, p_max]

Usage

phack_test_lcm(p, p_min, p_max, F_LCMsup = get.phack.F_LCMsup(), min_bunch = 3)

Arguments

p

– vector of p-values
p_min

lower bound of the interval used for the test
p_max

upper bound of the interval used for the test
F_LCMsup

cdf for LCM test
min_bunch

minimal number of elements of p that have exactly the same p-value in order to show a warning that there seems to be a rounding problem.

Simulate Brownian Bridge (BB) and ||LCM(BB)-BB||

Description

Simulate Brownian Bridge (BB) and ||LCM(BB)-BB||

Usage

SimBB(M)

Arguments

M

– number of repetitions