jama-statistics

Statistics & Statistical Review (jama-statistics)

When to trigger

• Results report p-values without effect sizes or confidence intervals
• Many comparisons but no multiplicity plan
• The analysis was not pre-specified, or outcomes drifted from the protocol
• Preparing for JAMA's dedicated statistical review of accepted-pending manuscripts

Core reporting rules at JAMA

1. Effect sizes with 95% CIs, not p-values alone. Report the estimate (mean difference, risk/hazard/odds ratio, absolute risk difference) with its 95% CI. P-values supplement; they do not substitute. Treat "significance" as a statement about the interval, not a 0.05 threshold ritual.
2. Pre-specified outcomes. State the single primary outcome and the secondary outcomes exactly as registered/protocoled. Flag any post hoc analysis as exploratory.
3. Intention-to-treat for RCTs. Primary analysis is ITT; per-protocol and as-treated are secondary/sensitivity.
4. Multiplicity. With multiple outcomes, subgroups, or time points, pre-specify the testing hierarchy and the correction (e.g., hierarchical testing, Bonferroni/Holm, gatekeeping). Subgroups without a multiplicity plan are hypothesis-generating only.
5. Missing data. State the mechanism assumed and the method (e.g., multiple imputation); report a sensitivity analysis. Complete-case-only needs justification.
6. Model assumptions. Justify the model (proportional hazards, linearity, clustering, repeated measures) and report how assumptions were checked.
7. Absolute and relative effects. Pair relative measures (RR, OR, HR) with absolute measures (risk difference, NNT) so clinicians grasp magnitude.

Reporting conventions to follow

• Report exact p-values (e.g., P = .03), not "P < .05"; very small as "P < .001"
• Round sensibly; do not imply false precision
• Give denominators with every percentage; report n/N
• Pre-register and report the analysis population for each estimate
• Describe software and key procedures so the analysis is reproducible
• For meta-analysis: report heterogeneity (e.g., I²), the model (fixed/random), and sensitivity analyses

Audit table

| Symptom | Fix |

|--------------------------------------------------|------------------------------------------------------|

| "P < 0.05" with no estimate | Add point estimate + 95% CI |

| Five "key" secondary outcomes, all "significant" | Pre-specify hierarchy; correct for multiplicity |

| Subgroup result drives the conclusion | Label exploratory; report interaction test |

| RCT analyzed per-protocol as primary | Re-run ITT as primary |

| 20% dropout, complete-case only | Multiple imputation + sensitivity analysis |

| Only relative risk reported | Add absolute risk difference / NNT |

Surviving JAMA's independent statistical review

The Journal of the American Medical Association applies a dedicated statistical review to manuscripts under serious consideration — distinguishing it among general medical journals. Independent statisticians re-interrogate the analysis, so pre-empt the standard queries: a pre-specified primary outcome matching the registry; intention-to-treat as primary; pre-specified multiplicity control; missing data handled by imputation with sensitivity analysis; and an absolute effect (risk difference / NNT) beside every relative measure.

Worked example: a primary-outcome readout (illustrative)

Vignette (illustrative): a multicenter, double-blind randomized clinical trial, N = 5,000 adults with type 2 diabetes and cardiovascular disease, new agent vs placebo; pre-specified primary outcome major adverse cardiovascular events (MACE) over a median 3.1 years.

• ITT result: 11.2% vs 13.6%; absolute risk difference -2.4 percentage points (95% CI, -4.3 to -0.5); hazard ratio 0.82 (95% CI, 0.70-0.96); P = .01.
• Correct readout: relative (HR with CI) and absolute (risk difference, ~42 NNT) effects both given, ITT, primary matches the registry; a "significant" secondary subgroup stays labeled exploratory with its interaction test.

Reviewer pushback and the JAMA fix

• "Primary outcome changed post hoc." Fix: restore the registered primary; demote the swap to labeled exploratory.
• "Subgroup result drives the conclusion." Fix: report the interaction test, label it hypothesis-generating, re-anchor on the primary.

Calibration anchors (hedge where uncertain): estimate-plus-95%-CI over bare p-values, ITT-as-primary, pre-specification, and absolute-with-relative reporting are durable; notation rules track the AMA Manual of Style — confirm against current author guidelines.

Checklist

• [ ] Every primary/secondary estimate has a 95% CI
• [ ] Single pre-specified primary outcome; secondaries labeled
• [ ] Multiplicity handled with a pre-specified plan
• [ ] ITT is the primary analysis for the RCT
• [ ] Missing-data method stated + sensitivity analysis
• [ ] Absolute and relative effects both reported
• [ ] Model assumptions justified and checked
• [ ] Analysis is reproducible (software, code/procedures described)

Anti-patterns

• Reporting p-values without effect sizes or CIs
• Switching or under-reporting pre-specified outcomes
• Mining subgroups and reporting only the "significant" one
• Claiming "no difference" from an underpowered study (absence of evidence ≠ evidence of absence)
• Per-protocol-as-primary to make an RCT look better
• Conclusions built on a secondary outcome while the primary was null

Output format

【Primary outcome estimate + 95% CI】...
【All estimates have CIs】yes / no
【Multiplicity plan】... 
【ITT primary (RCT)】yes / no / n.a.
【Missing-data handling】...
【Absolute + relative effects reported】yes / no
【Stat-review risks remaining】...
【Next skill】jama-figures-tables