percom-reproducibility

PerCom Reproducibility

Use this before submission and again before camera-ready. In pervasive computing, reproducibility

turns on the sensing data: a PerCom result is only as trustworthy as the dataset behind it, how

it was collected, and whether it generalizes across people. The goal is that a competent reader

could rebuild your pipeline and reach your conclusions — and, where ethics permit, on your actual

data.

Evidence map

• Map each recognition/system claim and reported number to a verifiable location — a paper

section, a table generated from logged data, or a script in the artifact.

• For recognizers, give enough of the features, model, hyperparameters, and evaluation split

(leave-one-subject-out / leave-one-session-out) that a reader could re-run it.

• For datasets, report subjects and their selection, sensors and placement, sampling rates,

labeling protocol and inter-annotator agreement, and preprocessing (filtering, windowing,

normalization).

• Keep the dataset-availability statement truthful and specific: what is shared, where it will

live after acceptance, and — if something cannot be shared — exactly why (privacy, IRB, consent).

• Keep the paper and the dataset consistent: a number in the PDF that no script reproduces from

the released data is the contradiction reviewers read as carelessness.

Dataset-availability statement audit

| Claim in the paper | Weak availability answer | PerCom-ready answer |

|---|---|---|

| "We collected data from N participants" | "Dataset available on request" | De-identified dataset + datasheet, or a documented restricted-access path with the ethics reason |

| "Our recognizer generalizes across users" | "Code will be released" | Runnable pipeline with a LOSO reproduction script and a README demo |

| "We labeled activities" | Nothing about protocol | Labeling protocol, annotator agreement, and the label files |

| "We deployed in a smart space" | "Testbed is proprietary" | Sensor list, placement, and sampling; simulated/sample data if the raw cannot ship |

"Available on request" reads as not available; convert every such line into a concrete,

de-identified dataset or an explicit, justified exception with a request path.

Sensing provenance floor

[Devices]    device models + firmware, sensor types, sampling rates, placement on body/space
[Labels]     labeling protocol, who labeled, inter-annotator agreement, label schema
[Preprocess] filtering, windowing, normalization, resampling -- the exact pipeline, not prose
[Splits]     leave-one-subject-out / session-out defined so a reader reproduces the same folds
[Ethics]     IRB/approval status, consent scope, and the de-identification performed
[Compute]    hardware, training time, number of runs so a reader can size a reproduction
[Randomness] seeds for any stochastic step; say what is and is not deterministic

Degrees of reproducibility (state the one you achieved)

• Turnkey: one documented command regenerates each table/figure (including the LOSO result)

from released data.

• Scripted: scripts exist but require documented manual steps or restricted-data access.
• Descriptive: prose detailed enough that a competent reader could rebuild the pipeline.

For PerCom, aim turnkey for anything a reviewer might rerun quickly (inference on a bundled sample,

a plot from logged features); full raw human-subjects data may stay **scripted with restricted

access** when consent/IRB forbids public release — but say so honestly rather than promising

turnkey behavior that cannot legally run.

Vignette: a wearable HAR study

Consider a study collecting wrist-IMU data from participants doing daily activities. Its

reproducibility spine: the collection protocol and device/sampling details; the de-identified

extracted dataset with a datasheet; the labeling protocol with annotator agreement; the feature and

model code; the leave-one-subject-out evaluation scripts that regenerate the F1 table; and one

honest sentence about the parts (raw video used for labeling, re-identifiable timestamps) that

cannot be shared and why.

Consistency and camera-ready pass

• Before submission: every scored number traces to the artifact; the availability statement matches

reality; the review package is anonymized (no testbed, lab, or owner strings).

• Before camera-ready: swap anonymized links for a permanent, DOI-issuing, de-identified deposit

(IEEE DataPort / Zenodo), and align the statement with what you actually release

(percom-artifact-evaluation).

Output format

[Claim inventory] <claim -> evidence location>
[Dataset availability] concrete / vague / restricted-with-reason / missing
[Provenance gaps] <devices / labels / preprocessing / splits / ethics / seeds / compute>
[Cross-subject reproduction] LOSO script present and matching the paper? yes/no
[Reproducibility level] turnkey / scripted / descriptive, stated honestly
[Paper fixes] <must appear in the PDF>
[Dataset fixes] <additions before release>