issta-review-process

ISSTA Review Process

Use this to reason about review-stage strategy. ISSTA review is double-anonymous and its outcome

model is richer than accept/reject, so plan around the Major-Revision path from the start. Reopen

the current call and dates page before making process claims — the number of deadlines and the exact

mechanics change between editions.

Process model

• Reviewing is double-anonymous: reviewers do not see author identities and authors do not see

reviewer identities.

• Each paper receives at least three PC reviews; chairs solicit more when expertise is thin or

reviewers disagree sharply.

• First-round outcomes are Accept, Major Revision, or Reject. A Major-Revision paper

revises against a fixed later deadline and receives a terminal decision — it is a real second

chance, not a soft reject, and reviewers expect the revision to address their points concretely.

• Earlier editions (e.g. ISSTA 2023, 2024) ran two rolling submission deadlines, where a

first-deadline paper could be sent a major revision to the second deadline while second-deadline

papers got only accept/reject. The multi-round model is genuine ISSTA history; its exact shape is

cycle-specific, so confirm the current one.

• Accepted papers are published in the ACM Digital Library, so final metadata and camera-ready

compliance matter alongside the initial decision.

The named evaluation criteria

| Criterion | What raises it | What sinks it |

|---|---|---|

| Originality | A technique or question the field did not have | A re-parameterized variant of existing work |

| Importance of contribution | A result the testing/analysis community will reuse | A narrow gain with no reuse story |

| Soundness | Claims scoped to what is actually shown | Overclaimed scope; unstated assumptions |

| Evaluation | Real subjects, fair baselines, proper statistics | Toy subjects, mis-configured baselines, single runs |

| Presentation | A clear threat model and evaluation contract | Undefined scope; results without protocol |

| Comparison to related work | Delta stated against the nearest techniques | Missing the closest competitor |

| Verifiability / transparency | Pinned subjects and a runnable artifact | Unshared subjects, unregenerable tables |

The last two — comparison and verifiability — are where testing/analysis papers most often lose

avoidable ground, because the nearest baseline and the shared artifact are both checkable.

Who reviews here

• The PC is specialized in testing and analysis, so a reviewer will know the closest tool, the

standard benchmark, and the usual statistical protocol. Vague baselines and hand-picked subjects

get caught rather than skimmed past.

• Borderline papers usually fall on one of three edges: an evaluation that does not use an

established benchmark, a baseline configured to lose, or a claim broader than the subjects tested.

Stage-by-stage realism

• Initial reviews: read for the criteria the meta-reviewer will weigh, not for reviewer tone.
• Response / discussion: address the decision-critical objection first; an early precise reply beats

a late comprehensive one.

• Major Revision: treat every comment as a tracked change and walk the ledger in the resubmission;

reviewers who see their points addressed in order revise upward.

• Decision: the meta-review synthesizes; one unresolved soundness or evaluation objection outweighs

several resolved presentation complaints.

Output format

[Current stage] submitted / reviews / response / major-revision / decision / camera-ready
[Outcome model] accept / major-revision / reject
[Decision actors] <reviewers / meta-reviewer / chairs>
[Likely leverage] <soundness / evaluation / comparison / verifiability>
[Forbidden moves] <identity leak / unpromised new results in the box>
[Next response move] <one action>