cikm-artifact-evaluation

CIKM Artifact Evaluation

CIKM has no verified badge-granting artifact track (2026 lineup: full, short,

applied, resource, demo — source map, 2026-07-08), but artifacts do more work here

than at most venues, because two of the five tracks are about artifacts: Resource

papers publish them, and Demonstration papers perform them. This skill covers the

artifact strategy per track and the staging from anonymous review object to public

citable release.

Artifact role by track

| Track | The artifact is... | Bar it must clear |

|---|---|---|

| Full / Short Research | Supporting evidence, cited anonymously | Regenerates headline results; survives a ten-minute inspection |

| Applied Research | Proof of practice | Substantiates the launch/deployment claim the track requires |

| Resource | The contribution itself | Documented, licensed, maintained, adoptable by strangers |

| Demonstration | The thing being demonstrated | Runs live, in front of people, on conference-venue conditions |

Resource-track packaging bar

A CIKM resource paper is graded on whether others will build on the object. The

verified call names data resources, software, evaluation tasks, and open-source

frameworks. Package accordingly:

• Provenance and license first: where the data came from, what consent/terms

cover it, and a license a university lab and a company can both act on. An

unlicensed dataset is unadoptable regardless of quality.

• Schema and statistics: field-level documentation, size, class balance, known

gaps — the datasheet discipline, applied without the buzzword.

• Baselines included: a resource ships with reference numbers and the scripts

that produce them, or every adopter re-invents the evaluation differently.

• Maintenance posture: versioning scheme, issue channel, and an honest

statement of how long the hosting is funded. Reviewers of resource papers have

watched too many datasets rot at dead URLs.

• All of it inside 4 pages (2026) — the paper is the resource's brochure and

datasheet condensed; depth lives in the repository.

KG-specific packaging

Knowledge graphs — a CIKM staple — need artifact fields other objects skip: the

dump date of any upstream KG, the extraction queries or rules that produced the

subgraph, entity-resolution decisions (what merged, what split), and per-relation

counts. Two teams "using Wikidata" are usually using different graphs; the artifact

is where that ambiguity dies.

Demo systems as artifacts

A demo paper's artifact must run where demos actually happen: conference Wi-Fi,

projector resolution, a visitor who types adversarial queries. Package a local

fallback (cached index, canned dataset, offline model), a 90-second scripted tour,

and a reset mechanism that restores clean state between visitors. The 4-page paper

should include the architecture and what a visitor experiences — reviewers accept

demos they can imagine standing in front of.

Licensing quick guide

License choice is an adoption decision, and CIKM's mixed academic/industry audience

makes it consequential:

| Object | Workable defaults | Watch out for |

|---|---|---|

| Code | MIT/Apache-2.0 for maximum reuse | Copyleft blocks industrial adopters — choose deliberately, not accidentally |

| Original data | CC BY (attribution) or CC BY-NC | NC clauses exclude exactly the applied audience CIKM attracts |

| Derived data (from a KG, a crawl, an API) | Whatever the upstream terms force | Upstream terms propagate: a Wikidata extraction, a scraped corpus, and an API dump each carry different obligations |

| Models/checkpoints | Code license or a stated model license | Base-model license inheritance for fine-tuned checkpoints |

Two failure modes recur in resource-track reviews: no license at all (unadoptable),

and a license the authors had no right to grant because upstream terms forbid

redistribution. Resolve the rights question before the paper claims release.

Hosting for the decade, not the deadline

CIKM papers become baselines years later (cikm-reproducibility), so host on that

horizon: an institutional or platform repository under an organization account

rather than a personal one; an archival deposit with a DOI for the exact

proceedings-version snapshot; dataset hosting with a stated funding horizon rather

than a lab NAS URL. The cheapest insurance is the archival snapshot — it survives

graduations, lab moves, and platform pivots, all of which the venue's older

artifact links have visibly failed to.

Anonymous-to-public staging

Review stage:  anonymized hosting, no author-resolving URLs/metadata,
               referenced from the PDF where each claim needs it
Acceptance:    real repository public by camera-ready (13-day window in 2026 —
               prepare during the review wait)
Public stage:  license chosen; versioned release tagged to the proceedings
               numbers; archival DOI (e.g., a Zenodo-class deposit) so citation
               outlives the hosting; README linking the ACM DL record

The GenAI Usage Disclosure obligation follows the artifact too: generated code,

synthetic data, or AI-produced labels inside the package belong in the paper's

disclosure section (cikm-reproducibility).

Naming and versioning conventions

Give the artifact the paper's short name (the mechanism name from

cikm-writing-style), version it semantically with the proceedings snapshot

tagged (e.g., v1.0-cikm2026), and keep the README's first line identical to the

paper title so search engines and readers connect the two objects. Post-

publication improvements go in later versions with a changelog; the tagged

snapshot stays frozen because it is what the paper's numbers mean.

Output format

[Track role] evidence / proof-of-practice / the-contribution / the-performance
[Package state] provenance / license / docs / baselines / maintenance
[KG fields] dump date / extraction / resolution decisions (if applicable)
[Staging] review-anonymous → public-citable, with dates
[Adoption blocker] <the single thing most likely to stop a stranger from using it>