METHODOLOGY — Guiding Premises for Abductive Process Tracing
Metodologia — Premissas que Guiam o Process Tracing Abdutivo
This is a living document, updated incrementally as the literature is processed and the database schema evolves. It is the canonical reference for the principles guiding database design. For the history of design decisions, see
NEWS.md.
1. What This Document Is
This document synthesizes the epistemological and methodological commitments guiding the design of the process tracing evidence database. It is a position paper: it states what we adopt, what we reject, and why, with transparent citations.
The primary methodological reference is Fairfield & Charman (2022) — the book, not just the 2017 article. This distinction matters: the 2022 book provides a more developed operationalization (verbal probability scales, decibel weight of evidence) than the 2017 article, and this project commits fully to the book’s framework.
The primary audience is threefold: (a) the researcher, as a reference during analysis; (b) external readers and reviewers, to audit the inferential logic; and (c) AI assistants working on this repository, so they do not default to the standard deductive process tracing framework that dominates their training data (see §8).
2. The Core Problem: Theorizing Under Imperfect Theories
O problema central: teorizar com teorias imperfeitas
The available theoretical explanations for the outcome under study are, individually, deficient:
| Deficiency | Definition | Term |
|---|---|---|
| Missing scope | Theory does not cover all relevant causal mechanisms | Insufficient |
| Vague mechanisms | Key causal steps are implied but never formally stated | Underspecified |
| Contradicted premises | Core assumptions do not match the empirical record | Misspecified (following van Fraassen’s empirical inadequacy) |
PT. Insuficiente = escopo incompleto. Subespecificada = mecanismos vagos. Malespecificada = premissas que contradizem a realidade empírica.
Under these conditions, no single theory can be “tested” in the classical sense. The appropriate response is abductive inference: reasoning from systematically organized evidence to the best available explanation, while acknowledging theoretical uncertainty.
3. Process Tracing: Three Variants
The literature identifies three main variants (Beach & Pedersen, 2019; Trampusch & Palier, 2016):
3.1 Theory-Testing (Deductive)
The researcher deduces a causal mechanism from existing theory and tests whether it operated as predicted. Theory must precede evidence. This is the dominant variant in political science training and in AI training data.
3.2 Theory-Building (Inductive)
The researcher begins with intensive empirical investigation to inductively construct a hypothetical causal mechanism. Pure induction is rare — prior theoretical intuitions always shape what the researcher looks for.
3.3 Explaining-Outcome (Abductive) — This Project
The researcher constructs a comprehensive explanation for a particular, complex historical outcome through abduction — a dialectical, iterative combination of deductive and inductive logic. The researcher moves constantly between empirical material and different theoretical frameworks.
This project adopts the explaining-outcome variant. The iterative process between evidence and theory is a methodological feature, not a deviation from rigor.
This is compatible with Bayesian logic. Fairfield & Charman (2022):
“mandates an iterative ‘dialogue with the data,’ which mirrors how process tracing is usually conducted.”
Critical implication: The premise “theory must precede fieldwork” — central to theory-testing PT — does not apply here. The database must not embed this premise.
4. Inference to the Best Explanation (IBE)
The product of the abductive process is an Inference to the Best Explanation:
Which of the candidate explanations — evaluated against the full body of accumulated evidence — best accounts for the observed outcome?
| Feature | Hypothetico-Deductive Testing | IBE |
|---|---|---|
| Starting point | Well-specified theory | Set of imperfect competing explanations |
| Logic | Deduce implications → check if present | Organize evidence → evaluate relative explanatory performance |
| Success criterion | Theory “passes” or “fails” | One explanation performs better than rivals |
| Role of evidence | Confirms/disconfirms predictions | Discriminates between rival explanations |
IBE does not require any single theory to be “correct.” It requires a comparison — systematic, transparent, and auditable. This is what the database is designed to enable.
5. Bayesian-Logical Foundation
Fundamento bayesiano-lógico
Following Fairfield & Charman (2022), this project adopts logical Bayesianism as its epistemological foundation. This project commits fully to the 2022 book’s operationalization, including the use of verbal probability scales and the decibel weight-of-evidence system.
5.1 Bayesianism as Universal Inferential Logic
Bayesian reasoning provides a formal structure for updating confidence in hypotheses as new evidence is encountered:
- Prior probability (prior): initial assessment of each hypothesis’s plausibility, based on background knowledge.
- Bayesian updating: revising confidence in light of new evidence. Key question: how expected is this evidence in the world where H is true, compared to the world where H’ is true?
- Posterior probability: updated confidence after processing evidence. Each posterior becomes the prior for the next round.
Fairfield & Charman (2022) demonstrate that this logic applies to qualitative data (interviews, archives, secondary sources) and unique historical events without requiring a population or frequency distribution.
5.2 Key Premises Adopted
(a) Rival Hypotheses Must Be Concrete and Mutually Exclusive
Sound inference requires comparing specific, substantive hypotheses against each other. Testing H against ¬H (its logical negation) is methodologically inadequate because ¬H is a catchall with no empirical content.
In this database, hypotheses are organized in groups of mutually exclusive rivals sharing a common research question. Each group may contain two or more hypotheses. All hypotheses within a group have equal structural status — there is no “principal” hypothesis. The current implementation uses pairwise groups for tractability, but the design accommodates groups of any size.
PT. Hipóteses rivais devem ser concretas e mutuamente exclusivas. Nunca se testa H contra ¬H. Todas as hipóteses dentro de um grupo têm status estrutural igual.
(b) Weight of Evidence via Likelihood Ratios and Decibels
Each piece of evidence is evaluated by its capacity to discriminate between rival hypotheses. The researcher assigns verbal probability assessments for each hypothesis:
“If H1 were true, how likely would we be to observe this evidence?” “If H2 were true, how likely would we be to observe this evidence?”
This is the exercise of “mentally inhabiting the world of each hypothesis” (Fairfield & Charman, 2022, Ch. 3.6.1).
Verbal Probability Scale
Following Fairfield & Charman (2022), likelihoods are expressed on a 7-point verbal scale mapped to numerical values:
| Verbal label | Value | Meaning |
|---|---|---|
quase_certa |
0.95 | Almost certainly would observe this evidence if H were true |
muito_provavel |
0.80 | Very likely would observe |
provavel |
0.65 | Probably would observe |
cinquenta_e_cinquenta |
0.50 | Could or could not occur — does not discriminate |
improvavel |
0.35 | Unlikely but possible |
muito_improvavel |
0.20 | Very unlikely to observe if H were true |
quase_impossivel |
0.05 | Almost impossible under this hypothesis |
Decibel Scale for Weight of Evidence
The likelihood ratio (LR = P(E|H1) / P(E|H2)) is converted to decibels (dB = 10 × log₁₀(LR)) following Fairfield & Charman (2022). This logarithmic scale has the advantage that weights of evidence from independent pieces of evidence are additive — you can sum dB values across evidence to get the cumulative weight.
| dB range | Category | Interpretation |
|---|---|---|
| ≥ 20 dB | muito_forte_H1 |
LR ≥ 100 — decisive evidence for H1 |
| 10–20 dB | forte_H1 |
LR 10–100 — strong evidence for H1 |
| 5–10 dB | moderado_H1 |
LR 3–10 — moderate evidence for H1 |
| 1–5 dB | fraco_H1 |
LR 1.3–3 — weak evidence for H1 |
| -1 to 1 dB | nao_discrimina |
LR ≈ 1 — does not distinguish H1 from H2 |
| -5 to -1 dB | fraco_H2 |
Weak evidence for H2 |
| -10 to -5 dB | moderado_H2 |
Moderate evidence for H2 |
| -20 to -10 dB | forte_H2 |
Strong evidence for H2 |
| ≤ -20 dB | muito_forte_H2 |
LR < 0.01 — decisive evidence for H2 |
The justification for each likelihood assignment is documented in the justificativa_likelihoods field — this is the most important analytical field in the database.
(c) Qualitative Priors
The justificativa_prior field in the hypothesis tables captures background-knowledge reasoning for each hypothesis’s initial plausibility — following Fairfield & Charman (2022, Ch. 3.1). The verbal probability scale may also be used for priors, but the textual justification is the primary record.
(d) Symmetry of Hypotheses
No hypothesis receives privileged structural treatment within a group. All hypotheses are peers. The database structure does not distinguish “principal” from “alternative.”
5.3 Counteracting Cognitive Bias
Fairfield & Charman (2022) argue that Bayesian discipline mitigates confirmation bias:
- The likelihood assessment for each hypothesis forces the researcher to genuinely inhabit rival worlds — not just ask “does this support my preferred explanation?”
- The decibel scale makes the cumulative weight of evidence transparent and auditable.
- The
justificativa_likelihoodsfield requires the researcher to articulate why each probability was assigned — not just pick a number. - Hypothesis symmetry prevents structural favoritism.
5.4 On the Van Evera Tests: Not Used
This project does not use the Van Evera / Collier typology of process tracing tests (Hoop, Smoking Gun, Straw-in-the-Wind, Doubly Decisive). Fairfield & Charman (2022) argue convincingly that these tests are unnecessary within a Bayesian framework: evidentiary confirmation is a matter of degree (captured by the likelihood ratio), not of type (captured by categorical labels). The verbal probability scale and decibel system subsume and supersede the test typology.
Earlier versions of this database (v1–v3) retained the Van Evera tests as “communicative vocabulary.” This position has been abandoned in favor of full alignment with Fairfield & Charman (2022). See NEWS.md for the history of this decision.
6. The Database as an Audit Trail
The evidence database makes the inferential process public and traceable:
- Any reader can identify exactly where they disagree — at the level of a specific likelihood assignment, its justification, or the prior reasoning for a hypothesis.
- Hypotheses and mechanisms are testable by third parties using the same evidence base.
- Source auditability: every evidence row is traceable to its source material.
- The cumulative weight of evidence (in dB) per hypothesis group can be computed and inspected.
Schema design reflects methodological commitments. See NEWS.md for the full history.
7. Two-Stage Evidence Workflow
The database implements a two-stage process:
Stage 1: Evidence Collection (tb3_evidence_raw)
Raw evidence is registered with: source identification, date, title, full content, summary, and a preliminary relevance assessment per hypothesis. This stage is descriptive — it records what exists without analytical judgment.
Stage 2: Evidence Analysis (tb4_evidence_analisys)
Each piece of raw evidence is analyzed against a specific hypothesis group. The researcher assigns: - Verbal probability for each hypothesis (P(E|H1), P(E|H2), etc.) - Textual justification for the likelihood assignment (justificativa_likelihoods) - Contextual metadata (government, political alignment, episode)
The likelihood ratio, decibel weight, and categorical label are computed automatically.
This separation ensures that the raw evidence corpus exists independently of any specific analytical framework — if the hypothesis structure changes, the raw evidence does not need to be re-collected.
8. What AI Models Typically Get Wrong About Process Tracing
This section exists because this repository is maintained with AI assistance. LLM training data encodes assumptions that conflict with this project’s methodology.
8.1 The Deductive Default
Most training data presents process tracing as deductive theory-testing. AI models tend to prescribe “theory must precede fieldwork” as inviolable and treat iterative evidence-theory movement as a defect.
Correction: In explaining-outcome PT, the iterative dialectic is the method itself (Fairfield & Charman, 2022).
8.2 The Van Evera Tests Default
AI models almost always suggest organizing evidence around Hoop, Smoking Gun, Straw-in-the-Wind, and Doubly Decisive tests.
Correction: This project does not use the Van Evera typology. Evidence is evaluated via likelihood ratios and the decibel scale (Fairfield & Charman, 2022). Do not introduce Van Evera test labels.
8.3 Resistance to Numerical Probabilities in Qualitative Research
Paradoxically, AI models trained on qualitative methods literature often resist the use of numerical probabilities, citing “qualitative research doesn’t use numbers.” This conflates the data (qualitative) with the inferential logic (which can be formalized).
Correction: Fairfield & Charman (2022) demonstrate that verbal probability scales with numerical anchors are both legitimate and useful for qualitative inference. The verbal scale is not about measurement precision — it is about making discriminatory reasoning transparent and auditable.
8.4 The “Principal Hypothesis” Assumption
AI responses typically assume a “main” hypothesis with subordinate “alternatives.”
Correction: All hypotheses within a group have equal structural status. Do not introduce language distinguishing “principal” from “alternatives.”
8.5 Confirmation Bias in Evidence Evaluation
AI models, when asked to evaluate evidence, tend to reproduce the researcher’s framing rather than genuinely inhabiting rival hypotheses.
Correction: When assisting with evidence evaluation, AI assistants should be explicitly asked to inhabit each rival hypothesis and articulate why the evidence might be expected or unexpected under each.
References
Beach, D., & Pedersen, R. B. (2019). Process-Tracing Methods (2nd ed.). University of Michigan Press.
Bennett, A., & Checkel, J. T. (Eds.). (2015). Process Tracing. Cambridge University Press.
Collier, D. (2011). Understanding process tracing. PS: Political Science & Politics, 44(4), 823–830.
Fairfield, T., & Charman, A. E. (2017). Explicit Bayesian analysis for process tracing. Political Analysis, 25(3), 363–380.
Fairfield, T., & Charman, A. E. (2022). Social Inquiry and Bayesian Inference: Rethinking Qualitative Research. Cambridge University Press.
Trampusch, C., & Palier, B. (2016). Between X and Y. New Political Economy, 21(5), 437–454.
van Fraassen, B. C. (1980). The Scientific Image. Oxford University Press.