pubar-data-analysis

Data Analysis (pubar-data-analysis)

PAR reviewers are methodologically capable public-management scholars, and the journal endorses the

TOP transparency guidelines — so analyses should be reproducible and documented (see

pubar-transparency-and-data). Because PAR articles carry Evidence for Practice, every estimate

that drives a managerial takeaway must be analyzed honestly enough to bear that weight. This skill

covers execution and reporting; design decisions live in pubar-research-design.

When to trigger

• Running main and supporting analyses; building the results section
• A reviewer asked for robustness, heterogeneity, or alternative specifications
• Reconciling preregistered vs. exploratory analyses
• Making the analysis reproducible before deposit

Analysis norms PAR expects

1. Report uncertainty honestly. Confidence/credible intervals, not just stars; the **magnitude and

substantive/managerial meaning** of the estimate, not just its significance. A practitioner needs

effect size, not a p-value.

1. Robustness that probes, not decorates. Show specifications that could break the result

(alternative measures, samples, estimators, fixed effects), and say what you learn.

1. Heterogeneity with discipline. Pre-specify subgroups where possible (agency type, jurisdiction

size, sector); correct for multiple comparisons; don't mine an interaction and theorize it post hoc.

1. Right inference. Cluster at the assignment/sampling level (agency, district); wild-cluster

bootstrap when clusters are few — a common public-management data situation.

1. Preregistration discipline. Clearly separate registered from exploratory analyses;

reconcile and justify deviations.

1. Measurement. Validate constructs (red tape, PSM, performance); report reliability; show results

are not an artifact of a coding/scaling choice — measurement debates are central in PA.

Mixed-methods integration

• State explicitly where the qualitative evidence corroborates, refines, or contradicts the

quantitative estimate; do not present them in parallel silos with no integration.

Reproducibility while you work (not at the end)

• One master script regenerates every table and figure from the (raw or constructed) data.
• Set and report seeds for bootstrap, randomization inference, simulation, any stochastic step.
• Pin software/package versions (renv.lock, requirements.txt, recorded ssc/net installs).
• Keep table/figure numbers matched to script outputs; document design/prep decisions in the

supplementary document PAR recommends.

Execution bridge (StatsPAI / Stata MCP)

Run the battery, don't just enumerate it. Full map:

[execution-with-mcp](../../../shared-resources/empirical-methods/execution-with-mcp.md). PAR is public administration — survey/observational and some experimental work; identification + clustered/multilevel inference, magnitude for practice.

• Many outcomes / specifications: romano_wolf (step-down FWER) or benjamini_hochberg.
• OVB sensitivity: oster_delta / sensemakr.
• Inference: wild_cluster_bootstrap (few clusters), twoway_cluster / conley.
• Re-fit off one handle: audit_result(result_id) lists missing checks + the exact

suggest_function for each.

• Exhibits: etable / did_summary_to_latex from the handle — no retyped numbers.

Decisive checks in the body, exhaustive battery in the appendix.

[JF execution walkthrough](../../../Journal-of-Finance-Skills/resources/worked-examples/02-execution-walkthrough.md).

Anti-patterns

• Stars-only tables with no effect sizes or intervals (a practitioner can't act on stars)
• "Robustness" that only reruns near-identical specs to manufacture stability
• p-hacking / fishing for a significant interaction; HARKing exploratory results into hypotheses
• Clustering at the wrong level or ignoring few-cluster problems
• An Evidence-for-Practice point that the analysis does not actually support

Output format

【Main estimate】magnitude + interval + managerial meaning
【Identification check】(per research-design) result
【Robustness】specs that could break it → what held
【Heterogeneity】pre-specified? MHT-adjusted?
【Registered vs exploratory】clearly separated?
【Reproducible】master script + seeds + pinned versions? [Y/N]
【Next】pubar-tables-figures

What PAR reviewers probe, by analytic tradition

| Analytic tradition | The check a PAR referee runs first | The fix that earns the benefit of the doubt |

|--------------------|------------------------------------|---------------------------------------------|

| Survey / managerial experiment | Is inference randomization-based and pre-registered? | Randomization inference, pre-registered estimand, MDE reported |

| Observational causal (reform) | Is the "causal" word (and the policy advice) doing more than the design licenses? | State estimand + assumption; sensitivity to an unobserved confounder |

| Performance / administrative data | Are measures validated, and is gaming/selection ruled out? | Construct validation, reliability, selection checks |

| Mixed methods | Do quant and qual estimates actually corroborate? | Show where they agree, and own where they diverge |

Worked micro-example (illustrative numbers)

A hypothetical PAR survey experiment tests whether a performance-feedback framing raises frontline

managers' willingness to adopt a new reporting tool. The pre-registered ATE is **+7.4 points (95% CI

3.0 to 11.8)** on a 0–100 willingness scale, randomization-inference p = 0.006. An exploratory

subgroup ("low-tenure managers") shows +13 points, but it was not pre-registered and after a

Bonferroni adjustment across five exploratory subgroups its interval crosses zero. The disciplined

write-up reports the +7.4 confirmatory effect with its interval and a managerial interpretation, flags

the +13 figure as exploratory and not multiplicity-robust, and frames it as a hypothesis — so the

Evidence-for-Practice point rests on the confirmatory estimate only. (All numbers illustrative.)

Calibration anchors (hedged)

• The bar is field-wide PA significance plus honest practice relevance; an effect only a specialist

values, or a takeaway the data can't support, rarely clears PAR review.

• PAR practices methodological breadth — a rigorous mixed-methods or case analysis is not second-class

to a regression. Match the inference standard to the design.

• TOP transparency expectations evolve; confirm the current data-policy wording on the journal's page

(检索于 2026-06；以官网为准).

Supplementary resources

• [../../resources/external_tools.md](../../resources/external_tools.md) — estimation, inference, and survey packages
• [../../resources/official-source-map.md](../../resources/official-source-map.md) — TOP transparency policy