Agents Builders

Result distillation policy

Archived
result-distillation-policy

Created

Jul 06, 20:55

Started

Jul 06, 20:59

Completed

Jul 06, 23:26

DevOps handoff

Type

Docs

Shape

backend

Worktree Slug

result-distillation-policy

Repositories

mcritchie-studio

Release Slug

Branch

feat/result-distillation-policy

Local URL

QA URL

Production URL

agent-ops observability

Acceptance Criteria

  • New module documents the result-distillation policy
  • Policy defines validate resolve finding per result
  • Two grains: action findings and activity verdicts

Expected Test Plan

  • [full-suite-bypass] markdown-only operating-model module; no runtime surface, no doc-consistency test covers it

Checks Run

  • [full-suite-bypass] markdown-only operating-model module; no runtime surface, no doc-consistency test covers it

Agent Context

Operating-model POLICY (docs) -- the OPSD Distill grain applied to tool results. PRINCIPLE: a raw tool result is DATA; a distilled finding is DATA + judgment; the trajectory should record FINDINGS not raw operations. POLICY: when a tool result is DECISION-BEARING (a diff, dor-check, a test run, a query that informs a verdict/next step), the agent records a DISTILLED FINDING -- a one-line validate->resolve, e.g. 'checked X, found Y, therefore Z' -- via bin/agent-activity action --summary, instead of leaving a raw re-read as the record. Routine paging re-reads are suppressed (the suppress-tool-result-paging-reads task). TWO GRAINS, nested: ACTION-level = the per-result finding; ACTIVITY-level = the outcome/verdict that synthesizes the findings (already exists). Progressive roll-up: raw result -> action finding -> activity verdict -> learning-loop insight. Agent-driven (only reasoning distills) -> a behavioral policy on the honor system, reinforced in the review SOPs. Deliverable: a NEW module doc docs/agents/modules/result-distillation.md (stands alone, deterministic) + register it in docs/agents/index.md module table (ONE line, keep index edits minimal -- the harden-review-lane task may also touch index, expect a possible rebase). A short reference from the pr-review role SOPs is OPTIONAL and owned by harden-review-lane (do not co-edit those SOPs here). Docs-only: use a [full-suite-bypass] justification or a doc-consistency test.

Stage Timeline

Who handled each stage, the time it took (measured), and the model / tokens / cost reported (best-effort) — plus who's on it right now. means the agent didn't report that metric.

Sizing Avi · PO SMALL Dev SMALL Actual XL ≠ forecast
  1. Created Designed
    L Lugia
    Lugia
    Model
    Duration
    Tokens
    Cost
    Completed Jul 06, 20:55 · 1 day ago
    api
  2. Designed Building
    L Lugia
    Lugia
    Model
    claude-opus-4-8
    Duration
    3 minutes
    Tokens
    9,712,264
    Cost
    ~$7.79
    Started Jul 06, 20:55
    Completed Jul 06, 20:59 · 1 day ago
    cli
  3. Building Submitted
    L Lugia
    Lugia
    Model
    claude-opus-4-8
    Duration
    8 minutes
    Tokens
    Cost
    Started Jul 06, 20:59
    Completed Jul 06, 21:07 · 1 day ago
    cli
  4. Submitted Blocked
    A Avi
    Avi
    Model
    claude-opus-4-8
    Duration
    14 minutes
    Tokens
    385,323
    Cost
    ~$0.30
    Started Jul 06, 21:07
    Completed Jul 06, 21:21 · 1 day ago
    cli
  5. Blocked Submitted
    L Lugia
    Lugia
    Model
    claude-opus-4-8
    Duration
    4 minutes
    Tokens
    15,778,688
    Cost
    ~$8.77
    Started Jul 06, 21:21
    Completed Jul 06, 21:25 · 1 day ago
    cli
  6. Submitted Reviewed Review Activity
    C Carl
    Carl primary
    J Jasper
    Jasper light
    Model
    claude-opus-4-8
    Duration
    2 minutes
    Tokens
    Cost
    Started Jul 06, 21:25
    Completed Jul 06, 21:26 · 1 day ago
    cli
  7. Reviewed Assembled
    S Steffon
    Steffon
    Model
    claude-opus-4-8
    Duration
    about 1 hour
    Tokens
    17,372,951
    Cost
    ~$18.87
    Started Jul 06, 21:26
    Completed Jul 06, 22:15 · 1 day ago
  8. Assembled Shipped
    A Avi
    Avi
    Model
    claude-opus-4-8
    Duration
    about 1 hour
    Tokens
    Cost
    Started Jul 06, 22:15
    Completed Jul 06, 23:26 · 1 day ago
  9. Shipped Archived
    Model
    Duration
    about 6 hours
    Tokens
    Cost
    Started Jul 06, 23:26
    Completed Jul 07, 05:30 · 1 day ago

Conversation

QA review feedback, agent handoffs, and follow-up notes for this task.

Comment alex 1 day ago

Reconciled kind feature->docs: deliverable is a pure markdown operating-model module (docs/agents/modules/result-distillation.md) + one index.md line, no code surface. shape=backend was a mis-fit; dor-check now passes via the designed EXEMPT-kind path. checks_run carries a [full-suite-bypass] record; the full Rails suite was not run (no runtime surface to certify).

Comment jasper 1 day ago

Scout report: merge-ready - Light review: all 3 acceptance criteria met; coupled-pair #431 verbs resolve; 2 non-blocking nits Reporter: jasper Outcome: merge-ready Findings: - AC1-3 met: result-distillation.md documents policy, validate->resolve->finding shape, two nested grains (action finding + activity verdict) - Coupled-pair OK: bin/agent-activity loads atomic-event so 'action --summary' resolves to the verb #431 extends; #431 drop rule (tool-results/ dir Reads dropped, all else kept) matches doc's suppression claim - NIT: doc illustration 'reopening a file to see the next chunk' is broader than #431 (only drops spilled tool-results/ reads, keeps real-file chunk-paging) - NIT: doc examples omit --finding, so recorded findings land kind=bash not #431's queryable 'finding' verb Questions: - none Checks: - gh pr diff 430/431 read-only; referenced atomic-capture-hook.md exists; index link convention matches siblings

Comment carl 1 day ago

Scout report: request-changes - Policy doc is clear, standalone, meets all 3 acceptance criteria; but its finding-recording command omits #431's --finding flag, so agents following it stamp kind=bash not kind=finding — a doc/code divergence in the coupled pair. Reporter: carl Outcome: request-changes Findings: - BLOCKER coupled-pair divergence: result-distillation.md:38 and :58 tell agents to record findings with 'bin/agent-activity action --summary ...' WITHOUT --finding; #431 adds --finding specifically to stamp kind=finding and render 'FINDING · ...' queryable as one class. Following the doc verbatim yields kind=bash rows, defeating #431's atomic-event half. Fix: add --finding to the command examples. - NIT clarity: result-distillation.md:46 describes suppressed paging as 'reopening a file only to see the next chunk'; #431 only drops Reads under a tool-results/ dir (the agent's OWN spilled output), and explicitly KEEPS re-reads of real source files. Tighten to 'its own spilled tool-output file' to match the code. - PASS acceptance: (1) new module documents the policy result-distillation.md:1; (2) validate->resolve->finding defined :29-38; (3) two grains action-level + activity-level table :56-59. Questions: - none Checks: - gh pr diff 430 read; gh pr diff 431 cross-checked; bin/agent-activity confirmed = bin/atomic-event shim; link targets exist

QA Feedback avi 1 day ago

Coupled-pair doc/code divergence (Carl primary: REQUEST-CHANGES; Jasper light flagged the same as a coherence nit). The policy tells agents to record findings with 'bin/agent-activity action --summary "…"' WITHOUT the --finding flag (docs/agents/modules/result-distillation.md — the validate->resolve->finding example ~:38, the grain table ~:58, and the 'How to record a finding' section). But the coupled PR #431 ships a bare --finding verb (FINDING_KIND="finding" in bin/atomic-event) whose entire purpose is to render 'FINDING · <conclusion>' / kind=finding rows and keep distilled findings queryable as one class; without --finding an 'action' defaults to kind=bash. So an agent following this doc verbatim produces kind=bash rows, not the finding rows #431 exists to create — the governing policy never instructs the flag its coupled code adds. FIX: add --finding to the command examples (:38, :58, and 'How to record a finding'). ALSO tighten the paging phrasing at ~:46 ('reopening a file only to see the next chunk') to something like 'reopening its own spilled tool-output file' — #431 suppresses ONLY re-reads under a tool-results/ dir and explicitly KEEPS real-source-file paging, so the current wording can read as claiming all chunk-paging is dropped. All 3 acceptance criteria otherwise PASS and CI is green — this is a light, trivial doc fix to make the pair consistent; resubmit after and post a note --resolves-feedback.

Handoff alex 1 day ago

Added --finding to the command examples so the policy matches the coupled #431 verb; tightened the paging line to specify the harness tool-results/ spill files (real source paging kept).

Handoff avi 1 day ago

Rework addressed Carl+Jasper's coupled-pair finding: added --finding to all command examples (aligns with #431's verb) + scoped the paging line to tool-results/ spill files. dor-check green (docs-exempt), CI green. Advancing to reviewed.

Sealed-bid sizing

Edit →

Alex (PM)

Avi (PO)

SMALL

Dev

SMALL

Actual

XL