hpca-reproducibility

HPCA Reproducibility

Use this to make an HPCA paper's results reconstructable — by a reviewer during the

window, by an artifact evaluator after acceptance, and by you six months later when

the environment has drifted. HPCA's artifact badges are **IEEE reproducibility

badges** earned on a separate AE HotCRP, so build toward that pipeline, not the ACM

one.

Pin the result chain

Every figure in the paper should trace back through an unbroken chain:

simulator commit (or machine + firmware)
  -> configuration file (core, cache, DRAM, interconnect)
    -> workload + input + sampled region
      -> raw output logs
        -> statistics script
          -> the figure/table in the PDF

If any link is unpinned — a simulator built from "latest," a config edited by hand, a

region chosen ad hoc — the result is not reproducible, only re-runnable by you.

One manifest per figure

Give each headline figure a manifest that names its commit, config, workload, region,

seed/trial count, and the exact command that regenerates it. A regen.sh per figure

that reproduces the artifact from raw logs turns "trust me" into "run this."

| Chain link | What to pin | Failure if unpinned |

|---|---|---|

| Simulator | Commit hash + local-patch diff | Silent behavior drift between builds |

| Config | Versioned file, not manual edits | Unreproducible operating point |

| Workload/region | Suite version + sampling recipe | Cherry-picked region suspicion |

| Statistics | Scripted, seeded | Numbers that cannot be regenerated |

| Silicon runs | Governor/turbo/SMT/NUMA + kernel/firmware | Noise mistaken for signal |

Record machine state for silicon

A real-hardware result is only reproducible with its host state captured: frequency

governor, turbo, SMT, NUMA policy, kernel and firmware versions, and the trial count

with dispersion. Archive it next to the logs, not in your memory.

Handle non-redistributable workloads

SPEC-class and other licensed suites cannot ship in the artifact. Provide the

recipe and checksums to reconstruct them, plus at least one freely available

workload that exercises the full pipeline so an evaluator without a license can still

reproduce a headline result.

Keep the environment resurrectable

Between submission and the badge round, environments rot. Containerize or script the

build, pin dependency versions, and do a cold-start resurrection drill on a clean

machine before you promise a reviewer or evaluator anything. The single

rebuttal/revision window is not the time to discover your simulator no longer builds.

Reproducibility checklist

[Result chain] every figure pinned commit->config->workload->region->stats? (Y/N)
[Manifests] one per headline figure with a regen script? (Y/N)
[Silicon state] governor/turbo/SMT/NUMA + kernel/firmware + trials archived? (Y/N/NA)
[Licensed workloads] recipe+checksums + one free workload provided? (Y/N/NA)
[Resurrection] cold-start build drill passed on a clean machine? (Y/N)
[Badge readiness] packaged for the IEEE AE HotCRP? (Y/N)
[Top gaps] <ordered>

Reopen the current artifact-evaluation page for the badge set and calendar before

relying on any AE detail here — the AE mechanics are per-edition.