govern-data-analysis

Data Analysis (govern-data-analysis)

Governance reviewers are comparative-method sophisticated and the journal requires a **Data

Availability Statement** describing whether and how replication materials can be accessed. Analyze as if

a competent reader will follow your inference across countries — because they will. This skill covers

execution and reporting norms; design decisions live in govern-research-design.

When to trigger

• Running main and supporting analyses; building the results section
• A reviewer asked for robustness, heterogeneity, or alternative specifications
• Reconciling pre-specified vs. exploratory analyses (an anonymized pre-analysis plan may be supplied)
• Making the analysis reproducible before drafting the Data Availability Statement

Analysis norms Governance expects

1. Cross-national inference, done carefully. Be explicit about what is identified off within-country

over-time variation vs. cross-country variation, and which the argument needs. Country-year panels

with two-way fixed effects answer a different question than a pure cross-section — say which.

1. Cluster and quantify uncertainty correctly. Cluster at the level of treatment assignment (often

country or reform unit); report confidence/credible intervals and effect magnitudes, not just stars.

1. Robustness that probes, not decorates. Show specifications that could break the result —

alternative governance measures, country/period subsamples, dropping influential cases, alternative

estimators — and say what you learned.

1. Triangulate across methods. Where the design is mixed, show that quantitative and qualitative

estimates corroborate; own and interpret divergence rather than hiding it.

1. Measurement validity for governance indices. Validate the construct; show the result is not an

artifact of one index (V-Dem vs. WGI vs. QoG vs. Bertelsmann) or one calibration; carry index

uncertainty (e.g., V-Dem credible intervals) into the inference where feasible.

1. Pre-specification discipline. Clearly separate pre-specified from exploratory analyses;

if a pre-analysis plan was supplied, reconcile and justify any deviations.

Small-N comparative samples (the recurring Governance problem)

• Few countries/clusters break standard cluster-robust SEs: use wild-cluster bootstrap or

randomization/permutation inference; report the cluster count honestly.

• With a small donor pool, consider synthetic control (and its placebo/leave-one-out checks) rather

than over-claiming from a few-unit panel.

• For set-theoretic (QCA) work, report consistency and coverage and probe robustness to calibration and

threshold choices; do not present a single solution formula as definitive.

• Resist over-fitting: in small samples, a long covariate list and a "clean" table are a warning sign,

not reassurance.

Sensitivity to unobserved confounders

Institutional outcomes are confounded by hard-to-measure history and capacity. Report how strong an

unobserved confounder would have to be to overturn the result (e.g., Oster's δ/bounds, sensemakr-style

robustness values, E-values). State the benchmark covariate you compare against.

Reproducibility while you work (not at the end)

• One master script regenerates every table and figure from the (raw or constructed) data.
• Set and report seeds for bootstrap, permutation inference, simulation, and any stochastic step.
• Pin software/package versions; record the exact governance-index version and download date.
• Keep manuscript table/figure numbers matched to script outputs, ready for the Data Availability Statement.

Execution bridge (StatsPAI / Stata MCP)

Run the battery, don't just enumerate it. Full map:

[execution-with-mcp](../../../shared-resources/empirical-methods/execution-with-mcp.md). Governance is public administration and institutions research — comparative and causal designs on governance reforms; the chain serves its quantitative-causal lane, while comparative-historical / qualitative work uses its own standards.

• Many outcomes / specifications: romano_wolf (step-down FWER) or

benjamini_hochberg — report the adjusted threshold.

• OVB sensitivity: oster_delta / sensemakr.
• Inference: wild_cluster_bootstrap (few clusters), twoway_cluster / conley;

multilevel data → cluster at the right level.

• Re-fit off one handle: audit_result(result_id) lists the missing checks and the

exact suggest_function for each.

• Exhibits: etable / did_summary_to_latex from the handle — no retyped numbers.

Keep the decisive checks in the body and the exhaustive battery in the supplement. See

the executed chain in the [JF execution walkthrough](../../../Journal-of-Finance-Skills/resources/worked-examples/02-execution-walkthrough.md).

Anti-patterns

• Stars-only tables with no effect sizes, intervals, or substantive interpretation across countries
• Standard cluster-robust SEs with a handful of countries (few-cluster bias ignored)
• "Robustness" that reruns near-identical specs to manufacture stability
• Treating one governance index as truth; never checking an alternative measure
• Mining for a significant cross-national interaction and theorizing it post hoc
• A results section whose numbers a reader could not reproduce from the materials

Output format

【Main estimate】magnitude + interval + cross-national substantive meaning
【Inference】clustering level; few-cluster correction if N small
【Measurement】index + version; result holds across alternative measures? [Y/N]
【Robustness】specs that could break it → what held
【Sensitivity】strength of unobserved confounder needed to overturn (δ / RV / E-value)
【Pre-specified vs exploratory】clearly separated?
【Reproducible】master script + seeds + pinned index versions? [Y/N]
【Next】govern-tables-figures

Supplementary resources

• [../../resources/external_tools.md](../../resources/external_tools.md) — estimation, few-cluster inference, synthetic control, QCA, and sensitivity packages
• [../../resources/official-source-map.md](../../resources/official-source-map.md) — Data Availability Statement and pre-analysis-plan policy