sigcomm-artifact-evaluation

SIGCOMM Artifact Evaluation

Use this for the post-acceptance artifact submission. SIGCOMM helped establish artifact

evaluation in networking, and the venue's "runnable papers" culture sets a high bar: a

committee attempts to build and, where a badge requires it, reproduce your headline result.

Reopen the current Call for Artifacts for this edition's exact badge set and dates (待核实 for

2026).

Badge selection is claim calibration

Choose the badges you can honestly support; the badge is a promise about what a stranger can

verify, not a participation ribbon.

| ACM badge | What it promises | What the AEC will do |

|---|---|---|

| Artifacts Available | Permanent public retrieval via an archival DOI | Confirm the archive resolves and is complete |

| Artifacts Evaluated (Functional) | The artifact builds and runs as documented | Follow your README end to end on their setup |

| Results Reproduced | The paper's main results follow from the artifact | Re-run to obtain the headline numbers within tolerance |

Aim the highest badge at the result you most want believed, and be candid where hardware or

scale makes full reproduction infeasible.

The networking-specific hard part

Generic artifact tooling cannot judge what SIGCOMM evaluators care about most: a result

measured on a rack of switches or a wide-area testbed rarely rebuilds on a laptop. Plan for

the gap:

• Downscale the topology. Ship a small emulated or Mininet-style topology that reproduces

the mechanism's behavior, with a documented map from the downscaled run to the paper's

full-scale figure.

• Substitute the trace legally. If the production trace cannot ship, provide a synthetic

or public-trace substitute with the same statistical character, and state exactly which

figures used which input.

• Pin the environment. Kernel and NIC features, switch firmware or programmable-data-plane

toolchain versions, and clock/timestamp sources all move results; record them.

• Expose the run count. Every reported percentile needs its replication count and the

seed or workload driver, or the tail cannot be reproduced.

What evaluators open first

artifact/
  README            # 5-minute smoke run FIRST, then the full path
  smoke/            # one command -> one small figure that proves it runs
  topology/         # downscaled + a map to the paper's full-scale setup
  traces/           # substitute inputs + provenance + which-figure-uses-what
  configs/          # exact parameters per experiment
  scripts/          # regenerate each figure from logged results
  results/          # logged raw outputs the plots are built from
  ENVIRONMENT.md    # kernel/NIC/switch/toolchain versions, hardware

Assume the evaluator runs the smoke path first and stops if it fails, so make one command

produce one convincing small figure before anything else is polished.

Vignette: packaging a data-center transport result

A paper reports a tail-FCT win on a 128-server testbed. Turnkey plan: a Mininet topology that

reproduces the incast dynamic at small scale; a synthetic RPC workload matching the trace's

flow-size distribution; a driver that emits the FCT distribution over 20 replays; and a script

that plots the 99th percentile directly from logged results so the artifact figure and the

paper figure cannot diverge. The README states plainly which numbers are full-scale (paper)

and which are downscaled (artifact).

Output format

[Badges sought] available / functional / reproduced — each justified
[Smoke path] one command -> one figure? yes/no
[Topology] full-scale -> downscaled map documented
[Trace handling] shipped / substituted (+ provenance) / described
[Environment] kernel/NIC/switch/toolchain pinned
[Fixes before AEC upload] <ordered list>