conext-artifact-evaluation

CoNEXT Artifact Evaluation

Turn the accepted paper's package into optional ACM reproducibility badges, administered by

CoNEXT's reproducibility committee. This is a distinct process from the paper review, with its

own timing: the decisive move — opting in before the paper submission deadline — happens long

before you have an accept. Treat badging as a track you enter at submission and complete after

acceptance.

The badge timeline (opt-in is the gate)

[Before submission deadline]  OPT IN for ACM badging  <-- miss this and you are ineligible this cycle
[Within ~1 week of accept]    send a ONE-PAGE artifact description + pointers to the committee
[Post-accept evaluation]      committee reviews the artifact against the ACM badge criteria
[Camera-ready]                approved badges appear on the PACMNET article (see conext-camera-ready)

The ACM badges (what each requires)

CoNEXT uses the ACM Artifact Review and Badging scheme. In practice the ladder is:

| Badge | What it certifies | What you must provide |

|---|---|---|

| Artifacts Available | The artifact is publicly archived | A DOI-issuing archive (Zenodo/figshare/Software Heritage) with an open license |

| Artifacts Evaluated — Functional | It is documented, complete, and runs | Clear docs, a runnable package, and a "getting started" path that works |

| Artifacts Evaluated — Reusable | It is functional and others can build on it | Reusability docs, clean structure, parameterization, and a license that permits reuse |

| Results Reproduced | Key results were regenerated by evaluators | A scripted path that regenerates the paper's central figures/tables from the inputs |

Confirm the exact badge set and names offered by the current CoNEXT reproducibility committee — the

scheme is stable but wording is cycle-volatile (待核实).

The one-page artifact description

The committee starts from your one-pager, so make it do work:

• Inventory: code, traces, configs, testbed scripts — with a pointer to each.
• Requirements: the hardware/software an evaluator needs. Networking artifacts often need

specific hardware (a particular switch ASIC, NIC, or SDR); if evaluators need access to

hardware you must provide, flag it prominently and describe how you will give access.

• Claims supported: which paper results the artifact reproduces, and which it cannot (be honest

about hardware-bound results).

• Getting started: the shortest path to a first successful run.

Networking-specific evaluation hurdles

• Hardware dependence. A result that needs a specific switch or NIC is hard for a remote

evaluator; provide a testbed reservation, a remote-access path, or a faithful emulation for the

parts that can run in software, and clearly mark what is hardware-bound.

• Trace size and privacy. Large or sensitive traces may not be fully releasable; ship a

representative or synthetic subset plus the pipeline, and document the restriction.

• Environment drift. Pin firmware/OS/kernel versions and provide images; "works on our testbed"

is not reproducible.

• Live dependencies. Cache anything that would otherwise require a live API/network call so the

evaluator's run is deterministic.

Reusability that survives an evaluator

• A README that assumes nothing about your lab's setup; every path and dependency explicit.
• A single command (or a short script) for the "getting started" run, and a documented full-reproduction

path for the central results.

• An open license present and consistent across the archive.
• A mapping from paper claim → artifact step → expected output, so an evaluator can check

"Results Reproduced" without guessing.

Pre-evaluation checklist

[Opt-in]        confirmed before the submission deadline? yes/no
[Archive]       DOI-issuing archive + open license? yes/no
[One-pager]     inventory + requirements + hardware-access plan + claims map? yes/no
[Functional]    fresh-environment run succeeds from the docs alone? yes/no
[Reusable]      structure + docs let a third party adapt it? yes/no
[Reproduced]    central figures/tables regenerate by script? yes/no
[Hardware]      hardware-bound results flagged; access path provided? yes/no

Output format

[Badge target] Available / Functional / Reusable / Reproduced (which are in scope)
[Opt-in status] done before submission? yes/no
[Artifact readiness] <per-badge gaps>
[Hardware plan] <how remote evaluators reach any required hardware>
[One-pager] <drafted / sent to committee>