uist-artifact-evaluation

UIST Artifact Evaluation

UIST has no artifact-evaluation committee or badge track (none was found for the

2026 cycle — 待核实 each year); the CFP-level instrument of proof is the video

figure. That absence raises rather than lowers the packaging bar: your artifacts are

judged twice, informally — at review time as *evidence the system exists as

claimed, and after publication as infrastructure other builders adopt*. Nobody

will certify either; both simply succeed or fail.

What counts as the artifact, by paper type

| Paper type | Review-time artifact | Reuse-time artifact |

|---|---|---|

| Interaction technique | Reference implementation + demo scene | Portable library with the technique isolated |

| Toolkit / authoring system | Runnable toolkit + the example apps from the paper | Documented API, tutorials, package registry entry |

| Hardware / fabrication | Design files, firmware, BOM, assembly photos | Fab-ready files + sourcing notes + calibration guide |

| Sensing / recognition pipeline | Trained models + capture data + eval harness | Dataset with collection protocol + retraining scripts |

| Human-AI / LLM system | Prompts, orchestration code, pinned model IDs, logged transcripts | Same, plus cost and drift notes |

Review-time packaging: the five-minute skeptic

A reviewer gives your supplement five minutes, anonymously, on a machine you don't

control. Optimize for that reader:

• One README at the archive root: what this is, which paper section each

directory backs, and one command (or one video) per claim.

• Prefer a recorded run alongside the code for anything with hardware, drivers,

or GPU dependencies — reviewers cannot rebuild your rig, so show the harness

producing the paper's numbers.

• Pin everything (lockfiles, container image digests, model checkpoints); "latest"

is a broken artifact by review week.

• Anonymize as strictly as the PDF: repository history, notebook authorship cells,

hardcoded home paths, calibration files named after lab members (see

uist-submission for the sweep).

supplement.zip
├── README.md              # claim → artifact map; 5-minute quickstart
├── technique/             # core implementation, pinned deps
├── hardware/              # schematics, PCB, STL/STEP, BOM.csv, firmware/
├── eval/                  # harness + raw logs behind Tables 1-2
│   └── rerun.sh           # regenerates the paper's numbers from logs
├── media/                 # per-claim capture clips (beyond the video figure)
└── LICENSES.md            # third-party components and their terms

Release-time packaging: engineering for strangers

At camera-ready (see uist-camera-ready), the audience flips from three skeptics

to an open-ended stream of builders:

1. De-anonymize deliberately — publish to the real org, restore attribution,

add the paper citation and BibTeX to the README.

1. Cut a release tag matching the camera-ready ("as-published") so later

development never orphans the paper's claims.

1. Choose licenses by artifact class: code (e.g. MIT/Apache-2.0), hardware

designs (e.g. CERN-OHL), data (e.g. CC-BY) — one archive often needs all three,

and institutional tech-transfer rules for hardware are worth checking early.

1. Archive beyond the repo: deposit the tagged release with a DOI service so

the URL in the proceedings outlives your hosting choices.

1. State the support posture honestly in the README — "research prototype,

issues welcome, no maintenance promised" is respectable; silence is not.

What the informal evaluators open first

Order the package for actual reading behavior:

1. README, thirty seconds. If the claim → artifact map is not visible without

scrolling, the evaluation is over.

1. The media directory, two minutes. Clips of the harness producing the

paper's numbers get watched; they are the highest-credibility artifact per

byte, especially for hardware.

1. One quickstart command, two minutes. Whatever you name in the README as

"run this" will be run in a fresh environment; test it in a container or a

colleague's clean machine, not your dev box.

1. Source spot-checks. Reviewers grep for the mechanism the paper claims is

novel; if the "self-calibrating controller" is a 30-line stub, the paper's

credibility inverts. Never ship scaffolding that contradicts the prose.

Toolkit papers: adoption is the long evaluation

For toolkit and authoring-system contributions, the release is the deferred

evaluation, and small engineering choices compound:

• Publish to the ecosystem's registry (pip/npm/crates/Arduino library manager) —

installability is adoption's first filter.

• Ship the paper's example applications as runnable starters; they are the

tutorials people actually read.

• Keep the API surface documented at the level of the paper's abstractions, so

citations of the toolkit describe your concepts in your vocabulary.

• Track downstream uses; a "built with X" list is both maintenance motivation

and the evidence base for the retrospective the venue's decade-scale memory

eventually invites.

Hardware honesty

Physical artifacts cannot be uploaded, so their evidence standard is

reconstruction: exact part numbers with sources, tolerances that matter, assembly

sequence photos, firmware flashing instructions, and the calibration procedure with

expected readings. A paper whose device only the authors can build has published a

demo, not a contribution — reviewers from fabrication-heavy labs apply exactly that

test (see uist-reproducibility for the replication ledger).

Output format

[Artifact class] technique / toolkit / hardware / pipeline / hybrid
[Review package] five-minute test passes? claim→artifact map complete?
[Anonymity] archive-level sweep clean?
[Release plan] tag · licenses (code/hardware/data) · DOI deposit · support posture
[Gap list] <artifacts named in the paper but absent from the package>