conext-experiments

CoNEXT Experiments

Build the evaluation a networking reviewer will actually interrogate. CoNEXT's evidence culture is

systems-and-measurement: claims are backed on the real target platform — a testbed,

deployment, or trace — with honest baselines and reported uncertainty, not simulation standing in

for hardware or a single number with no variance. Because a one-shot major revision is decided on a

list of minimum necessary changes, an evaluation gap you leave now often becomes a mandatory fix

under a tight window later.

Match evidence to claim shape

| Claim shape | Evidence CoNEXT expects |

|---|---|

| A mechanism is faster/cheaper on real hardware | A run on the real target (switch, NIC, kernel, testbed) under its real constraints, vs. a tuned baseline, with effect sizes |

| A phenomenon exists in the wild | A measurement campaign with documented vantage points, capture dates, and a reproducible extraction methodology |

| An architecture scales | Scalability evidence (real deployment or faithful emulation) across the relevant range, not a point claim |

| An operator intervention helps | Evidence at operationally relevant scale, with the counterfactual measured or bounded |

| A learned component adds value | An ablation isolating the learned part from the mechanism, plus a contamination check |

Real testbeds and deployments over simulation

• Evaluate on the platform the paper is about. If the claim is about switch behavior, run it on

the switch (or a faithful hardware testbed); if about a protocol on real paths, use a testbed or

trace-driven replay. Simulation-only systems claims are a classic CoNEXT revision risk.

• Describe the testbed so it can be rebuilt: topology, hardware models, firmware/OS versions,

link rates, buffer depths, and background traffic. A reviewer who cannot picture the setup cannot

trust the numbers.

• State what is emulated vs. real, and why the emulation is faithful for the claim.

Honest, tuned baselines

• Compare against the strongest reasonable baseline, tuned with an equal, documented budget —

not a strawman default. "The baseline is not tuned/fair" is one of the most common CoNEXT

push-backs.

• Report the baseline's configuration alongside yours so the comparison is auditable.
• Where a fair baseline does not exist, say so and justify the comparison you made.

Measurement statistics and uncertainty

• Report multiple runs and variability (confidence intervals, percentiles/CDFs for latency,

not just means). Tail behavior is often the point in networking.

• Use corrected comparisons when testing many conditions; a measurement reviewer notices

uncorrected multiple testing.

• Prefer effect sizes and distributions over a single bar; a CDF that shows the whole

distribution pre-empts questions a mean hides.

Trace and config provenance (pin it now)

• Record capture vantage points and dates, the extraction/anonymization methodology, and the

exact configs used. These cannot be reconstructed after the fact — pin them at collection time.

• For anything derived from operator or third-party data, document the access terms and what can

be released (this interacts with double-anonymity and reproducibility).

• Keep the raw-to-result pipeline scripted so the numbers in the paper regenerate from the trace.

Contamination-aware ML-for-networking ablations

If the paper uses a learner or LLM on networking data:

• Isolate the learned component with an ablation (e.g., replace it with a heuristic) so the

reviewer sees its marginal value over the mechanism.

• Guard against contamination/leakage: ensure test paths/traces/time windows are disjoint from

training, and that a temporal split reflects real deployment.

• Pin model identifiers and dates and cache raw model outputs; a package that needs live API

calls re-samples rather than reproduces (see [conext-reproducibility](../conext-reproducibility/SKILL.md)).

• Apply the model-swap test: if the networking lesson does not survive swapping the model, the

paper may belong at an ML venue (see [conext-topic-selection](../conext-topic-selection/SKILL.md)).

Pre-submission evaluation audit

[Claim coverage]   every claim has a matching measurement on the real target? yes/no
[Platform realism] real hardware/testbed/trace, or simulation standing in? note each
[Baselines]        strongest reasonable, equally tuned, config reported? yes/no
[Uncertainty]      multiple runs, CIs/CDFs, corrected comparisons? yes/no
[Provenance]       vantage points, dates, configs, firmware/OS pinned? yes/no
[ML checks]        ablation + contamination guard + model-swap survives? n/a or yes/no

Output format

[Evaluation status] solid / gaps
[Claim-evidence matrix] <claim -> measurement + platform + baseline + uncertainty>
[Platform] real target used? emulation justified?
[Provenance] traces/configs/firmware pinned for reproducibility
[Revision risk] <the gap most likely to become a minimum-necessary change>