eccv-experiments

ECCV Experiments

Use this while the experimental plan is still changeable. ECCV's calendar

shapes the evidence problem: results freeze in early March, reviews weigh

them in May against everything published since, and the field first reads

the paper at a September conference — the numbers must still look

current six months after the freeze.

The six-month-staleness test

For each headline table, ask: *if the strongest lab in this niche publishes

their CVPR camera-ready in June, does this table still support the claim in

September?* Evidence that passes: mechanism-isolating ablations, efficiency

frontiers (accuracy vs compute), and generality sweeps across datasets.

Evidence that fails: a raw leaderboard number 0.2 points above a moving SOTA.

Build the paper's claim on the first kind and let the leaderboard row be

corroboration, not the thesis.

Matched-substrate fairness

The first thing a 2026-era vision reviewer checks is whether wins come from

the method or from what it was fed:

| Axis to match | Unfair pattern | Fair protocol |

|---|---|---|

| Backbone / pretraining | Your ViT-L vs their ResNet-50 numbers | Re-run the top baselines on your backbone, or add a matched-backbone row |

| Training data | Extra pseudo-labeled or web data only on your side | A same-data row, with the extra-data row labeled as such |

| Input resolution / TTA | Higher test resolution quoted against lower | State resolution and TTA per row |

| Compute / epochs | 4x schedule vs baselines' 1x | Report schedule; add an equal-budget row |

| Foundation-model access | API model in your pipeline, none in baselines | Give baselines the same tool or ablate it out |

One honest matched row protects the paper better than three inflated rows —

the mismatched-substrate objection is the most common substantive ECCV

review attack and cannot be answered in a one-page rebuttal without a

matched number already in hand.

Ablations that isolate, not decorate

• Each claimed component gets exactly one toggle row; combinatorial grids go

to the supplement.

• Include the "replace with the dumb version" row (attention → average,

learned prior → uniform): it distinguishes mechanism from capacity.

• Ablate on the mid-sized benchmark, not the smallest one, so effects clear

seed noise (eccv-reproducibility for the variance bar).

Qualitative evidence discipline

Vision panels weigh pixels. Ship, in body or supplement: same-scene

comparisons against the two strongest baselines; a random-sample grid (not

curated) for at least one dataset; and a failure panel tied to the

limitations paragraph. A paper with only curated successes reads as hiding

something — the failure panel is credibility infrastructure.

Run sequencing toward March 5

T-10 weeks:  falsifier first — the experiment most likely to kill the
             claim (matched-substrate row on the main benchmark)
T-8:         main-table runs launched; seeds x3 on deciding rows
T-6:         ablation toggles; efficiency/frontier measurements
T-4:         cross-dataset generality; qualitative harvesting begins
T-2:         freeze new runs; regenerate all tables from logged results
T-1:         random-sample grids, failure panel, supplement tables
T-0 (Mar 5): body tables locked; supplement week polishes, never adds

Launching the falsifier first is the ECCV-specific discipline: with a

biennial venue, discovering at T-2 that the matched row erases the win

wastes not a cycle but two years.

Output format

[Evidence verdict] mechanism-backed / leaderboard-fragile / incomplete
[Staleness test] <headline table -> survives September? why>
[Substrate audit] <axis -> matched / mismatched -> repair row needed>
[Ablation map] <claimed component -> isolating toggle present?>
[Run queue] <next runs in falsifier-first order with weeks-to-freeze>