Skip to content

Feedback triage

Generated from a canonical source

This page is a read-only projection of docs/handoff-corpus/feedback-triage.md. Edit the canonical file, then run npm --prefix tools/project-knowledge-derive run derive.

What feedback-triage is for

The invariant you must not break: the agent and cron must act ONLY on feedback submitted through the feedback UI, or on an issue an operator has explicitly opted in with the feedback-agent label — and nothing on the money path is ever auto-implemented. Break the first half and the agent could open PRs against arbitrary in-flight development issues, corrupting unrelated work. Break the second half and an agent could ship an unreviewed change to charge, refund, dunning, or payment-method code. The gate is enforced in two independent places, not one: the cron's data scope (feedback_items only) and tools/feedback-gate/check.mjs's label-plus- footer check. (Source: autonomy-gate.ts:45-67, check.mjs::shouldProceed, GH #1884.)

This domain covers three capabilities a recipient will reason about as one loop:

  • In-product feedback capture — a merchant flags a bug or request from the admin without leaving the app; the widget auto-attaches the console-error trail, viewport, route, and build id so the report is actionable (GH #1884)
  • Agentic triage — submitted feedback is classified (bug/enhancement/ question/kudos, priority, and a work-lane) within a cron cycle, and actionable items become tracked GitHub issues automatically (GH #1884)
  • Autonomous fix for low-risk items — a ui/content bug can be implemented by an agent as a PR-to-dev that a human reviews and merges; the agent never merges its own work (GH #1884)

The load-bearing decisions:

  • The autonomy gate lives in code, not the model — autonomy-gate.ts::applyAutonomyGate is authoritative; the judge model's status field is advisory only. Only bug + ui/content may stay ready-for-agent; anything touching the money path (charge, refund, payment, dunning, billing, card, vault — matched over both the item text and the judge's own rationale) is forced to ready-for-human. A hallucinating judge cannot self-escalate. (GH #1884; rally-hq docs/decisions/0005-feedback-triage-loop.md shape — a sibling repo at ~/Workspace/dev/ref/rally-hq/, outside this repo tree, so cited by path rather than a resolvable link; money-path override added in this port.)
  • Provenance is a structural double-gate, not a convention — the cron reads only feedback_items, whose sole writer is the capture route, so ongoing-dev issues are unreachable from the triage side; the implementer workflow additionally requires (label feedback + a feedback-meta footer) OR an operator feedback-agent opt-in label, enforced by the unit-tested tools/feedback-gate/check.mjs. This is the operator's stated constraint — the agent must never touch work it wasn't handed. (GH #1884)
  • Four separated substrates, ported from a proven loop — capture (D1) / judge (cron + LLM) / queue (GitHub issues) / implementer (GHA) are independent, so a failure or a missing secret in one stage degrades gracefully instead of breaking the others (rally-hq ADR-0005 lineage).
  • The PR is the human gate — feedback-agent.yml opens a PR to dev and never merges; combined with the existing operator-gated dev → main release, no agent-authored change reaches main without two human checkpoints. (GH #1884)

Canonical-framing attestation (operator-ratified 2026-07-02). The loop is a four-stage pipeline, each stage on its own substrate, ported from rally-hq's ADR-0005 feedback-triage loop: capture (apps/admin FeedbackWidget → POST /api/v1/feedback → D1 feedback_items) → judge (apps/api/src/crons/feedback-triage.ts, a forced-tool Claude call) → work queue (GitHub issues, filed with a machine-readable feedback-meta footer) → implementer (.github/workflows/feedback-agent.yml, which opens a PR to dev and never merges). There is no competing hand-written or model-authoritative gate; the deciding artifact is GH #1884, and the autonomy gate and provenance gate are both canonical in code, confirmed by reading autonomy-gate.ts (a pure, dependency-free function) and check.mjs::shouldProceed directly — not inferred from the workflow's coarse label if, which is only a pre-filter.

How it actually works

Capture — POST /api/v1/feedback

handleFeedbackPost (routes/feedback.ts:132-229) validates the body (8-4000 chars, optional 140-char title, optional 16KB JSON context, optional base64 screenshot capped at 2MB decoded), then:

  1. Checks a per-store hourly cap (10/hr) with a direct D1 COUNT(*) query — deliberately not the Workers RATE_LIMITER binding, whose fixed 5/hr budget is the wrong shape for this surface.
  2. If a screenshot is present, decodes and size-checks it before the R2 PUT; the FEEDBACK_SCREENSHOTS binding is optional, so a submission still lands without the attachment when the binding is absent (tests, pre-provision deploys).
  3. Derives persona server-side from the verified JWT (derivePersona, routes/feedback.ts:128-130) — the client never asserts who it is.
  4. Inserts the row with status='new'. A same-store, same-body, same-UTC-day resubmission hits the idx_feedback_items_daily_dedup unique index and is acknowledged as {deduplicated: true} with a 200, not an error — so a double-click never surfaces a failure to the widget.

Judge — the triage cron

runFeedbackTriage (crons/feedback-triage.ts:201-310) runs on a 5-minute clock gate registered in worker.ts's scheduled handler, and no-ops immediately if ANTHROPIC_API_KEY is unset or no status='new' rows exist. Per tick:

  1. Pulls up to 10 new rows, oldest first.
  2. Loads the last 50 open items (across ready-for-agent, ready-for-human, needs-info, needs-design, needs-product) to give the judge duplicate-detection context.
  3. Per row, claims it with an atomic conditional UPDATE ... SET status='triaging' WHERE id=?1 AND status='new' — a concurrent tick cannot double-process the same row (claimed.meta.changes === 0 skips it).
  4. Calls the Anthropic Messages API with tool_choice: {type: 'tool', name: 'triage_feedback'} — a forced tool call, so the verdict is schema-validated at the model boundary rather than hand-parsed from free text (callJudge, crons/feedback-triage.ts:137-168).
  5. Passes the raw verdict through applyAutonomyGate — this is the only place the model's ready-for-agent claim can survive or get downgraded.
  6. If the (gated) status is one of ready-for-agent, ready-for-human, needs-design, needs-product and FEEDBACK_GITHUB_TOKEN is set, ensures the label catalog once per run and files a GitHub issue via createIssue, appending the feedback-meta footer (buildFeedbackFooter) that both the loop-back webhook and the provenance gate key on.
  7. Writes the verdict onto the row regardless of whether an issue was filed.
  8. On any error mid-item, releases the claim (triaging → new) so a later tick retries it, and increments summary.errors rather than aborting the batch.

Read the classification decision tree as: not-understandable/vague → needs-info; spam/out-of-scope → wontfix; restates an open item → duplicate; praise-only → wontfix; a bug needing a UX decision → needs-design, else gated by blast radius (ui/content → ready-for-agent, everything else → ready-for-human); a change request not yet agreed worth building → needs-product, else needs-design or ready-for-agent/ready-for-human by mechanical-vs-non-trivial. The system prompt states the money-path exclusion explicitly to the model (crons/feedback-triage.ts:70) — but that instruction is advice to the model, not the enforcement mechanism; applyAutonomyGate is the enforcement mechanism, applied unconditionally after the call returns.

Implementer — the GitHub Actions agent

.github/workflows/feedback-agent.yml triggers on issues: labeled and gates on github.event.label.name == 'ready-for-agent'. That if is a coarse pre-filter only; the real gate is its Provenance gate step, which shells out to tools/feedback-gate/check.mjs against the live issue's labels and body. shouldProceed (tools/feedback-gate/check.mjs:18-24) returns true only when the issue carries the feedback label and a <!-- feedback-meta footer, or carries an operator-applied feedback-agent label — exit 0 proceeds, exit 1 is a no-op the workflow still treats as a successful run. Only if that passes does the claude-code-action@v1 step run, and its prompt explicitly instructs the agent to branch off origin/dev, implement only the brief's stated scope, never touch charge/refund/payment/dunning/billing/card/vault code, and never merge — opening a PR whose body (not commit subject) carries `Fixes

N`.

Loop-back — POST /webhooks/github

handleGithubFeedbackWebhook (routes/feedback.ts:309-372) verifies X-Hub-Signature-256 with a timing-safe HMAC-SHA256 compare (verifyGithubSignature) before touching the payload, ignores any event that isn't issues, and matches the issue back to a feedback_items row by parsing the feedback-meta footer out of the issue body (parseFeedbackMeta) — never by issue number alone, so an unrelated repo issue can never flip a feedback row. deriveStatusFromIssueEvent (routes/feedback.ts:291-307) maps a closed action to resolved or wontfix (by state_reason), and a labeled/unlabeled/reopened action to whichever status label is present, in a fixed precedence order.

No dedicated sequence diagram exists for this domain yet — Input-B records this explicitly rather than a diagram being omitted by oversight. A capture → judge → issue → implementer → loop-back sequence is the obvious candidate for docs/architecture/sequence-diagrams.md; until it's authored there, this section's prose is the only mechanism reference, and the citations above point at the actual functions rather than a picture of them.

Where intent and reality diverge

Four typed deltas, each cited to its source:

  • Named-deferred — the entire pipeline is dormant. No stage has run live: no cron tick has processed a row, no issue has been filed by the judge, and no agent PR has opened. The code is merged to dev and unit-verified, but inert until docs/runbooks/feedback-assistant-activation.md runs — the triage cron no-ops without ANTHROPIC_API_KEY on the subs-api worker, issue-filing no-ops without FEEDBACK_GITHUB_TOKEN, the webhook returns 502 without FEEDBACK_GITHUB_WEBHOOK_SECRET, and feedback-agent.yml needs the ANTHROPIC_API_KEY Actions secret. Activation is operator-gated: subs-api lives on a personal Cloudflare account reachable only via a GH secret the local agent's tokens can't read, and Anthropic key creation is Console-only.
  • Verified-but-incomplete — the capture API accepts and stores an opt-in screenshot to R2 (handleFeedbackPost, base64 → decode → size-cap → FEEDBACK_SCREENSHOTS.put), but the admin widget (apps/admin/src/components/Feedback/) does not yet capture one — the backend supports the affordance the UI doesn't yet expose.
  • Built-but-untrodden — the capture route writes exactly one feedback_items row per submission, and the webhook matches rows only via the feedback-meta footer; there is deliberately no bulk or mass-mutation surface. A recipient should not look for one, and adding one would need a new gate of its own — the current gate design assumes one row, one issue, one footer.
  • Contract-verified, not live-verified — the webhook loop-back (handleGithubFeedbackWebhook, timing-safe HMAC over the raw body) is unit-tested for signature validity, invalidity, and the unconfigured case (feedback.test.ts:207, :215), but has never received a real GitHub delivery. It returns 502, not 503 — the typed-error system has no 503 variant, and upstreamFailure (which both this route and the sibling /webhooks/stripe route call) is hardcoded to 502 in errors/translate.ts:26. This matches Input-B's claim exactly, and directly contradicts a comment at the top of routes/feedback.ts claiming "503... mirrors /webhooks/stripe" — see Confidence notes below.

Live-state attestation (operator-ratified 2026-07-02). As of this trace, no feedback_items row has been written by a deployed worker, no issue has been filed, and no agent PR has been opened. apps/api has no own wrangler.toml — its config is infra/cloudflare/wrangler.toml (worker name = subs-api) — and the CF account the repo's local wrangler token authenticates against is not the account subs-api actually runs on; the agent cannot enumerate that worker's live secret set from this environment. deploy-api.yml only runs on tag or manual dispatch, and no deploy carrying migration 0048 has been dispatched. The no-op guards are confirmed by reading the code directly: feedback-triage.ts reads env.ANTHROPIC_API_KEY and github.ts's caller reads env.FEEDBACK_GITHUB_TOKEN; both are optional fields on Env (types.ts).

How to operate & extend

  • Activate the loop: follow docs/runbooks/feedback-assistant-activation.md in order — it is operator-gated end to end (Console-only Anthropic key creation, personal-CF-account worker secrets, Actions-secret write access none of the local agent tokens have). Feedback capture itself works as soon as migration 0048 is applied; screenshots additionally need the subs-feedback-screenshots R2 bucket to exist first, or the deploy step fails on the binding.
  • Widen the auto-PR lane: AUTO_PR_SUBTYPES (autonomy-gate.ts:45) is the single set controlling which bug subtypes may stay ready-for-agent. Adding a subtype here is a real autonomy expansion — it is not gated by anything else once the money-path regex doesn't match, so treat it as a decision, not a config tweak.
  • Extend the money-path exclusion: MONEY_PATH_RE (autonomy-gate.ts:48) is the one pattern that forces ready-for-human regardless of subtype. It matches over both the raw item text and the judge's own rationale — extending the vocabulary (a new payment-adjacent term) is a one-line, low-risk change; narrowing it is not.
  • The invariant you must not break: the cron's data scope (feedback_items only) and tools/feedback-gate/check.mjs's label-plus-footer check are the two independent enforcement points for "the agent only touches feedback-loop-owned work." A change to either one needs the other re-verified alongside it — they're independent by design, not redundant.
  • Extension seam — the judge model: DEFAULT_MODEL and the system prompt live in crons/feedback-triage.ts; the structured-output contract (TRIAGE_TOOL) is the only surface the model can affect, since applyAutonomyGate re-derives the actionable decision regardless of what the model claims for status.
  • Extension seam — new issue-state labels: STATE_LABELS_BY_PRECEDENCE (routes/feedback.ts:272-278) is the fixed precedence list the loop-back webhook reads; a new decision lane needs an entry here and in LABEL_CATALOG (feedback/github.ts:39-56).

Confidence notes

  • A code comment in routes/feedback.ts is factually wrong about the webhook's unconfigured-secret status code. The file header (routes/feedback.ts:17-18) says the webhook route returns "503 when unconfigured... mirrors /webhooks/stripe." Tracing the actual call (upstreamFailure('github', 'feedback_github_webhook_not_configured')) through errors/translate.ts:26 shows upstream_failure maps to 502, and a direct test assertion at feedback.test.ts:207 confirms 502. The sibling Stripe webhook (routes/webhooks-stripe.ts:268) calls the same upstreamFailure helper, so the behavior genuinely mirrors Stripe's — the comment's specific number is stale. This matches Input-B's typed delta exactly (which already states 502, not 503) and is a clean example of why this corpus traces code instead of trusting a header comment.
  • The traced unit-test count does not match Input-B's cited "47/47." Input-B's live-state attestation cites "applyAutonomyGate + capture route 47/47 (autonomy-gate.test.ts, feedback.test.ts)." Counting it( blocks directly in this tree: autonomy-gate.test.ts has 13, and feedback.test.ts has 19 — 32 total, not 47. The check.mjs::check.test.mjs 6/6 figure Input-B cites for the provenance gate does match (6 it( blocks counted). I could not run the suites in this worktree to get a live pass/fail count — apps/api's vitest config fails to resolve @cloudflare/vitest-pool-workers here (a known worktree-npm-install gap, not specific to this domain) — so this is a static block-count discrepancy, not a proof that fewer assertions exist (a single it can carry more than one expect, and the 47 could be an assertion count rather than a block count). Flagging as a contradiction worth a human look, not resolving it here.
  • No sequence diagram exists for this domain. Input-B records this explicitly (## Visual aids in the Input-B file says "none yet"), and I confirmed no feedback-specific entry exists in docs/architecture/sequence-diagrams.md's table of contents. This page therefore has no transcluded diagram in Move 2 by design, not by omission — the obvious candidate (capture → judge → issue → implementer → loop-back) is noted as future work, not fabricated here.