Forking paths problem
fallacy in statistical hypothesis testing
The garden of forking paths is a problem in frequentist hypothesis testing through which researchers can unintentionally produce false positives for a tested hypothesis through leaving themselves too many degrees of freedom. In contrast to fishing expeditions such as data dredging where only expected or apparently-significant results are published, this allows for a similar effect even when only one experiment is run, through a series of choices about how to implement methods and analyses, which are themselves informed by the data as it is observed and processed.
History
Exploring a forking decision-tree while analyzing data was at one point grouped with the multiple comparisons problem as an example of poor statistical method. However Gelman and Loken demonstrated that this can happen implicitly by researchers aware of best practices who only make a single comparison and only evaluate their data once.
The fallacy is believing an analysis to be free of multiple comparisons despite having had enough degrees of freedom in choosing the method, after seeing some or all of the data, to produce similarly-grounded false positives. Degrees of freedom can include choosing among main effects or interactions, methods for data exclusion, whether to combine different studies, and method of data analysis.
Multiverse analysis
A multiverse analysis is an approach that acknowledges the multitude of analytical paths available when analyzing data. The concept is inspired by the metaphorical "garden of forking paths," which represents the multitude of potential analyses that could be conducted on a single dataset. In a multiverse analysis, researchers systematically vary their analytical choices to explore a range of possible outcomes from the same raw data. This involves altering variables such as data inclusion/exclusion criteria, variable transformations, outlier handling, statistical models, and hypothesis tests to generate a spectrum of results that could have been obtained given different analytic decisions.
This brief starts where responsible research should: with the source description of “Forking paths problem” as fallacy in statistical hypothesis testing. Everything that follows is an evidence route, not borrowed authority.
Why this record matters
The subject matters to the general reference register because the source frames it as fallacy in statistical hypothesis testing. Its deeper value depends on whether names, dates, institutions and citations support that framing.
The citation trail is more important than the brevity of the summary: it shows where individual claims can be examined in context. The source revision retrieved here is dated Jun 23, 2026. The linked authority identifier is Q121365276.
A concise general-reference account can conceal disagreements about scope, terminology or the weight assigned to individual sources. The source lead contains qualifying language; that uncertainty should survive quotation, summary and reuse. Authority statements aid reconciliation but still require their own references, qualifiers and ranks to be checked.
How to read it
Use the entry as an orientation point, then follow its citations and revision history. Names, dates and institutional relationships should be checked against the original record.
- Subject orientation
- Search vocabulary
- Locating named sources
The closest primary source, responsible institution and strongest cited specialist reference.
Three-step research path
- Establish the record: confirm the title “Forking paths problem”, its source revision and the description used here.
- Expand the search: follow Forking paths problem primary sources, Forking paths problem archive and Forking research across catalogues and specialist indexes.
- Test the account: compare the strongest cited source with the responsible institution’s current record and note any disagreement.
Questions for further research
- Which source most directly establishes the central claim about “Forking paths problem”?
- Which cited source is closest to the event, object or claim?
- What terminology or title could unlock a more precise catalogue search?
Search terms from this dossier
This entry incorporates text from “Forking paths problem” on English Wikipedia. Contributors are listed in the page history. Text is available under the Creative Commons Attribution-ShareAlike 4.0 License. Selected authority identifiers and statements are retrieved from Wikidata under CC0; their references and qualifiers remain part of the verification path.