retro

Retro Skill

Overview

Read-only retrospective front door. Two things to read: docs/what-didnt-work.md, the negative-results registry that records which experiments lost; and the routing and review telemetry in learning.db, queried through scripts/learning-db.py. Both are stores this skill reports on — it records nothing itself. Hooks write the telemetry; humans write the registry.

---

Instructions

Parse the user's argument to pick the subcommand. Default to what-didnt-work when no argument is given.

| Argument | Subcommand |

|----------|------------|

| (none), what-didnt-work, negative results | what-didnt-work |

| routing, route health, route stats | routing |

| reviews, review roi, false positives | reviews |

Subcommand: what-didnt-work

Print the negative-results registry, the list of experiments that lost. Read it before re-running an experiment so a known-dead path is not retried.

The registry is a doc, not a DB table: docs/what-didnt-work.md is the capture, store, and query target.

Step 1: Read and print the registry.

Use the Read tool on docs/what-didnt-work.md and present it. Group by the dated ## YYYY-MM-DD headings; show each entry's Decision verdict (rejected / deferred / revisit-if) up front so a scan answers "did we already reject this?".

NEGATIVE RESULTS (docs/what-didnt-work.md)
==========================================

## [date] [experiment]
  Decision: [rejected | deferred | revisit-if <condition>]
  What happened: [one line]
...

If the file is missing, report that no negative results are recorded yet and point the user at the format in CONTRIBUTING.md.

Step 2: To search the registry, grep the doc.

grep -n -i "TERM" docs/what-didnt-work.md

The doc is the single store. Keep a parallel copy nowhere — a second store drifts from the canonical one and answers stale.

Subcommand: routing

Report routing feedback-loop health from the telemetry hooks write.

Key constraint: Present results as readable tables or sections, not raw JSON. Every command here is read-only.

Step 1: Run the health check.

python3 ~/.claude/scripts/learning-db.py route-health

Step 2: Add the dimension the user asked about. --by is required.

python3 ~/.claude/scripts/learning-db.py route-stats --by agent    # or skill, force-route, errors, override, week, day
python3 ~/.claude/scripts/learning-db.py route-weights             # health-aware re-rank input
python3 ~/.claude/scripts/learning-db.py stack-usage               # enhancement skills seen stacked

Step 3: To compare two cohorts before and after a change, name both refs.

python3 ~/.claude/scripts/learning-db.py route-delta --from SHA_OR_DATE --to SHA_OR_DATE [--key agent:skill] [--metric error|tokens]

Step 4: Present the report.

ROUTE HEALTH
============

Outcome basis:   [share scored from explicit signal vs neutral]
Silent success:  [share]
Top routes:      [key — dispatches, error rate]
Weakest routes:  [key — dispatches, error rate]

Read the outcome basis before reading the rates. A rate computed mostly from neutral outcomes describes the scorer, not the router.

Subcommand: reviews

Report reviewer cost and precision.

python3 ~/.claude/scripts/learning-db.py review-roi                       # per-tier cost vs findings
python3 ~/.claude/scripts/learning-db.py review-fps [--limit N]           # false positives by reviewer agent

Present ROI per tier alongside the false-positive count for the same agent — a tier with high findings and high false positives is expensive twice.

---

Examples

Example 1: Check a settled question before re-running an experiment

User says: "/retro what-didnt-work"

Actions: Read docs/what-didnt-work.md, present entries newest first with each Decision verdict up front.

Example 2: Routing health check

User says: "/retro routing"

Actions: Run learning-db.py route-health, then route-stats --by agent, present outcome basis first, then per-route rates.

Example 3: Which reviewer tier earns its cost

User says: "/retro reviews"

Actions: Run learning-db.py review-roi and review-fps, present cost, findings, and false positives per agent in one table.

---

Error Handling

Error: "learning.db not found"

Cause: No routing telemetry recorded yet in this environment.

Solution: Report that no telemetry exists. The routing hooks populate it during normal dispatches; run a session with hooks synced, then re-check.

Error: "route-stats: the following arguments are required: --by"

Cause: route-stats aggregates along one dimension and has no default.

Solution: Re-run with an explicit dimension: --by agent, skill, force-route, errors, override, week, or day.

Error: docs/what-didnt-work.md missing

Cause: The registry has not been created in this checkout.

Solution: Report that no negative results are recorded and point at the six-field format in CONTRIBUTING.md (date, experiment, expectation, what happened, evidence, decision).

---

References

• ~/.claude/scripts/learning-db.py — read-only CLI for routing and review telemetry
• docs/what-didnt-work.md — negative-results registry; the doc is the canonical store
• skills/meta/do/references/routing-telemetry.md — which hook records what, and the route-failure protocol
• hooks/session-context.py — injects the overnight dream payload at session start