colt-artifact-evaluation

COLT Artifact Evaluation

First, the honest framing: **COLT has no artifact-evaluation track, no artifact

badges, and no code-submission requirement.** Nothing of the kind appears in the COLT

2026 Call for Papers or conference pages (checked 2026-07-08). Do not import

NeurIPS/ICML-style artifact checklists into a COLT plan; they answer questions this

venue does not ask. What replaces artifact review at COLT is the referee's line-by-line

reading of your proofs — so this skill treats the proof appendix as the artifact and

tells you how to make it verifiable.

What replaces the artifact track

| At an artifact-track venue | The COLT counterpart | Who "evaluates" it |

|---|---|---|

| Code repository with run scripts | Complete proofs in the same PDF | Reviewers, line by line |

| Reproducibility checklist form | Explicit assumption and constant bookkeeping | Reviewers and the AC |

| Artifact badge | Community trust that the argument closes | Readers citing the bound |

| Dockerized environment | Self-contained lemma dependencies | Anyone re-deriving the result |

| Benchmark seeds and configs | Scripts for illustrative numerics, if any | Discretionary reader interest |

The proof appendix as artifact

A COLT artifact plan is a verification plan:

• Map every theorem to its full proof location and confirm no step says "similarly" or

"standard" where the step is neither.

• Track constants explicitly through the proof chain; a constant that changes value

between Lemma 4 and Theorem 1 is the theory equivalent of a failing test.

• List every external result invoked (concentration inequalities, minimax theorems,

reductions) with a precise citation including theorem number, and check that your

setting satisfies that result's hypotheses — the most common hidden bug.

• Have a coauthor who did not write a proof verify it cold, logging time-to-verify.

Sections that take a fresh expert more than an hour per page need restructuring

before a time-pressed referee sees them.

A verification ledger you can actually run

Keep a machine-checkable inventory next to the LaTeX source:

# proof-ledger.txt — one line per claim, updated at every freeze
THM1  full proof App.B  | deps: LEM3, LEM4, [BLM13 Thm 6.2]  | verified: RG 2026-01-12
THM2  full proof App.C  | deps: THM1, LEM5                   | verified: none  <-- BLOCKER
LEM3  full proof App.B.1| deps: none                          | verified: SW 2026-01-08
LEM4  sketch only       | deps: [SSBD14 Lemma 26.8]           | verified: n/a   <-- expand
COR1  two lines in body | deps: THM2                          | verified: with THM2

The rule: no submission while any load-bearing line reads verified: none. This

ledger, not a badge, is COLT-grade artifact discipline.

When code exists anyway

Many COLT papers include small numerical illustrations, and some results come with

reference implementations. Norms for those:

• The 2026 CFP imposed no code-upload mechanism; if you want reviewers to see code,

it must be referenced in the PDF via an anonymized link or included as appendix

pseudocode. Anonymize repository history and hosting if you link during review.

• Keep illustration scripts tiny and dependency-light: one file that regenerates the

figure from a fixed seed is worth more than a framework.

• Formal-verification companions (Lean or Coq mechanizations of a key lemma) are an

emerging but optional credibility signal in the theory community — mention one if it

exists, never fake partial formalization as full.

• After acceptance, publish scripts in a citable repository and link them from the

arXiv version, since the PMLR PDF is fixed at camera-ready.

Worked vignette: packaging a bandit paper's evidence

A hypothetical submission proves a $O(\sqrt{TK})$ regret upper bound and a matching

lower bound, and includes one figure simulating both on the lower-bound instance.

Its artifact plan under COLT rules:

• The upper-bound proof (App. B) and lower-bound construction (App. C) are the

artifacts; both enter the ledger with named verifiers and dates.

• The external-results audit lists the two concentration inequalities used, each with

source theorem number and a one-line check that boundedness hypotheses hold.

• The figure's generating script (40 lines, numpy only) is described procedurally in

App. D — instance parameters, seed, 100 replications — because no upload channel

exists; the script itself is queued for a public repository at acceptance.

• Nothing else ships. No environment files, no hardware table, no checklist — those

belong to venues that ask for them.

Post-acceptance release checklist

When code or mechanization exists, release it once anonymity ends:

| Item | Standard | Why it matters at COLT |

|---|---|---|

| Repository | Public, licensed, tagged at camera-ready state | The PMLR PDF is frozen; the repo carries updates |

| Link placement | arXiv version + PMLR "code" field if offered | Theory readers find papers via arXiv first |

| Script scope | Regenerates each figure from a fixed seed | Matches the illustration claims exactly |

| Mechanized proofs | State coverage precisely (which lemmas, which axioms) | Partial formalization overstated reads as spin |

What not to do

• Do not pad a COLT submission with an "artifact appendix" of configs and hardware

tables for a paper whose claims are theorems; reviewers read it as misunderstanding

the venue.

• Do not promise a code release as a substitute for a complete proof — no reviewer

here will trade executable evidence for a gap.

• Do not cite artifact badges from other venues as evidence of correctness.

Cycle-volatility warnings

• If a future COLT cycle introduces any code or artifact mechanism, the CFP will say

so explicitly; absence in 2026 does not bind later years (待核实 each cycle).

• Anonymous-linking rules during review follow the current anonymity policy, which is

re-issued yearly.

Output format

[Artifact reality check] proofs are the artifact / paper also ships numerics
[Verification ledger] complete / gaps at <claims>
[External results audit] <result -> hypotheses checked?>
[Code companion] none needed / anonymized link / post-acceptance release
[Blocking item] <the one unverified load-bearing claim>