Agents Builders

Avi supervisor review hierarchy

Archived
avi-supervisor-review-hierarchy

Created

Jul 06, 15:34

Started

Jul 06, 15:41

Completed

Jul 06, 17:46

DevOps handoff

Type

Bug

Shape

backend

Worktree Slug

avi-supervisor-review-hierarchy

Repositories

mcritchie-studio

Release Slug

Branch

feat/avi-supervisor-review-hierarchy

Local URL

QA URL

Production URL

agent-ops review-lane

Acceptance Criteria

  • Avi supervises and never reviews code himself
  • Primary and light spawn in parallel under Avi
  • Separate primary and light role SOPs exist
  • Delegate narration says select-reviewers not summon-Avi

Expected Test Plan

  • [unit] pr-review spawns two parallel role reviewers, not Avi-as-primary
  • [integration] review run attributes primary+light to selected souls

Checks Run

  • [unit] bin/pr-review fan-out spawns primary+light role reviewers in parallel, supervisor never reviews, prompts point to role SOPs, narration reads 'select primary+light reviewers' not 'summon Avi' (pr_review_command_test, green)
  • [integration] review run attributes primary+light to the reviewer-select pick — selection matches execution end to end (pr_review_command_test, green)
  • [unit] review_lane_docs_test tripwires flipped to enforce the supervisor hierarchy across role.md/mission.md/devops-cycle-design.md/parallel-agent-devops.md (green)
  • [full-suite-bypass] markdown-only SOP portion: pr-review.md supervisor rewrite + new pr-review-primary.md/pr-review-light.md role SOPs + primitive flip + registry tables
  • [full-suite@f63b38377a6b4006769ca7a2cad81b2f7ca888d9] bin/rails db:test:prepare test test:system green
  • [rubocop@f63b38377a6b4006769ca7a2cad81b2f7ca888d9] bin/rubocop clean

Agent Context

Fixes a REGRESSION from PR #417 (summon-devops-acts-as-subagents). ROOT CAUSE (operator-diagnosed + confirmed against the primitive): #417 reframed pr-review.md as "summon an Avi subagent and have it RUN the procedure / NEST its reviewers as its own child subagents" — that collapsed Avi INTO the reviewer level, so in execution Avi did the deep PRIMARY review himself and demoted the selected primary (Carl) to LIGHT, dropping the selected light (Jasper). The primitive (docs/agents/modules/pr-review-sop.md) is explicit that Avi is a THIN GATE that NEVER reviews. THE MODEL (operator's 3-level hierarchy): Level 1 = session Pokemon (identity); Level 2 = Avi (SUPERVISOR — gate + orchestration, NEVER reviews code); Level 3 = domain experts Carl/Jasper/Shannon/Steffon/Alex (the actual reviewers). NEW EXECUTION: Avi (supervisor) confirms product-acceptance, runs bin/reviewer-select to pick PRIMARY + LIGHT, then spawns BOTH the primary and light experts IN PARALLEL as his own child subagents (change from the old "PRIMARY spawns the LIGHT" nesting — Avi oversees both concurrently), collects both verdicts, and gates to reviewed/blocked. Avi has NO review step. BUILD: (a) Rewrite docs/agents/agents/avi/sops/pr-review.md as the Avi SUPERVISOR SOP (keep it the registered pr-review invocation) — remove the "Avi nests its reviewers" / "run the procedure below" framing that caused the collapse; state explicitly that Avi never reviews; describe spawn-primary-and-light-in-parallel + collect both verdicts + gate. Keep the interactive-tree-visibility note (Avi subagent nests the two experts) but make crystal clear Avi is a supervisor, not a reviewer. (b) CREATE docs/agents/agents/avi/sops/pr-review-primary.md (deep-review role SOP — what the PRIMARY expert runs) and docs/agents/agents/avi/sops/pr-review-light.md (focused second-read role SOP — what the LIGHT expert runs). Each stands alone/deterministic; each has the reviewer narrate as its own soul (bin/agent-activity start --agent <soul> --task <slug>). (c) Update the primitive docs/agents/modules/pr-review-sop.md §2 + the at-a-glance table: Avi spawns BOTH primary and light in PARALLEL (not primary-spawns-light), Avi never reviews, the two experts are siblings under Avi. (d) Update bin/pr-review's reviewer fan-out so BOTH role reviewers are spawned in parallel under the supervisor with the CORRECT roles: primary = the selected primary soul running the primary role SOP; light = the selected light soul running the light role SOP. Do NOT let the supervisor perform a review. Fix the delegate/activity --reason labels so the reviewer-select step narrates as "select primary+light reviewers" NOT "summon Avi subagent". Register the two new SOPs in the SOP registry tables (AGENTS.md / CLAUDE.md / docs/agents/modules/heartbeats.md) if the convention requires it — check how sub-SOPs / role SOPs are registered (pr-review-slow is a precedent). SOPs must stand alone and be deterministic — inline the mechanics, no design-doc pointers. This is docs (4 SOP files) + bin/pr-review code. Test the bin/pr-review spawn structure (correct roles + parallel, supervisor does not review). Goes live via bin/install-agent-docs post-merge.

Stage Timeline

Who handled each stage, the time it took (measured), and the model / tokens / cost reported (best-effort) — plus who's on it right now. means the agent didn't report that metric.

Sizing Avi · PO MEDIUM Dev MEDIUM Actual XL ≠ forecast
  1. Created Designed
    L Lugia
    Lugia
    Model
    Duration
    Tokens
    Cost
    Completed Jul 06, 15:34 · 2 days ago
    api
  2. Designed Building
    L Lugia
    Lugia
    Model
    claude-opus-4-8
    Duration
    7 minutes
    Tokens
    16,116,432
    Cost
    ~$10.64
    Started Jul 06, 15:34
    Completed Jul 06, 15:41 · 2 days ago
    cli
  3. Building Submitted
    L Lugia
    Lugia
    Model
    claude-opus-4-8
    Duration
    36 minutes
    Tokens
    1,491,598
    Cost
    ~$0.83
    Started Jul 06, 15:41
    Completed Jul 06, 16:17 · 2 days ago
    cli
  4. Submitted Blocked
    A Alex
    Alex
    Model
    claude-opus-4-8
    Duration
    20 minutes
    Tokens
    Cost
    Started Jul 06, 16:17
    Completed Jul 06, 16:37 · 2 days ago
    cli
  5. Blocked Reviewed Review Activity
    Model
    claude-opus-4-8
    Duration
    34 minutes
    Tokens
    48,364,757
    Cost
    ~$29.92
    Started Jul 06, 16:37
    Completed Jul 06, 17:11 · 2 days ago
    cli
  6. Reviewed Assembled
    S Steffon
    Steffon
    Model
    claude-opus-4-8
    Duration
    13 minutes
    Tokens
    22,066,385
    Cost
    ~$13.85
    Started Jul 06, 17:11
    Completed Jul 06, 17:24 · 2 days ago
  7. Assembled Shipped
    A Avi
    Avi
    Model
    claude-opus-4-8
    Duration
    22 minutes
    Tokens
    6,643,930
    Cost
    ~$5.76
    Started Jul 06, 17:24
    Completed Jul 06, 17:46 · 2 days ago
  8. Shipped Archived
    Model
    Duration
    about 4 hours
    Tokens
    Cost
    Started Jul 06, 17:46
    Completed Jul 06, 22:06 · 1 day ago

Conversation

QA review feedback, agent handoffs, and follow-up notes for this task.

QA Feedback alex 2 days ago

BLOCK — red required gate + stale cert. The docs/script CONTENT is approvable; the block is purely CI + cert, so a clean re-push should clear it. (a) CI 'test' is RED at head 6daea3dd. scan_ruby/scan_js/lint all green; only the 'test' job failed, on ONE system-lane e2e: ReviewEventsTest#test_[e2e]_operator_opens_review_process_hub_from_deployment_link_menu (test/system/review_events_test.rb:48) — 'expected "/stages" to equal "/review_events"'. This is NOT your code: that test file AND the view it drives (app/views/tasks/_deployment_link_menu.html.erb) are byte-identical to origin/release and are NOT in this PR's diff (your +511/-125 is bin/pr-review + docs + doc-tests — no routes/views). CI also logged a chromedriver 149 vs chrome 150 mismatch → a classic Capybara mis-click (the 'Stages' link sits directly above 'Docs' in the same dropdown). Flaky/unrelated e2e, not a defect. FIX: rebase on latest origin/release and re-push to trigger a fresh CI run. If it reproduces, it's flaky on release itself — de-flake it (scope the click to the Docs link / add an explicit wait / assert the link before clicking) or split to a separate bug; don't let it gate this docs PR. Flagging to Avi/Steffon as a possible release-branch flake. (b) The cert is STALE. checks_run claims [full-suite@53b1959…] 'test test:system green', but head is 6daea3dd (4 commits) and 53b1959 is not in the branch — so the 'system green' claim doesn't hold at head, and CI at head proves the system lane is (flakily) red. Re-cert the FULL suite (incl. system) at head after the rebase and update checks_run. (c) CONTENT is CLEAN — no doc/script defects. All 4 acceptance criteria met: (1) Avi supervises / never reviews code — consistent across role.md, mission.md, devops-cycle-design.md (§1.2 + table row + clarifications), parallel-agent-devops.md, heartbeats.md, pr-review.md, the pr-review-sop.md primitive, full-cycle.md, deploy-with-task.md; (2) primary+light spawn IN PARALLEL as siblings under Avi — consistent everywhere, and bin/pr-review genuinely launches both via Process.spawn before waiting (verified); (3) separate pr-review-primary.md + pr-review-light.md role SOPs exist, STAND ALONE (no design-doc pointers; Related links only registered SOPs + the primitive), and are registered in BOTH index.md registry tables; (4) bin/pr-review narrates 'select primary+light reviewers' (never 'summon Avi'), asserted by the command test. Stale-reference sweep: git grep at head found ZERO lingering 'spawns the light' / 'nested chain' / 'nested cascade' / 'hands the lane to the PRIMARY' anywhere in docs/agents; pr-review-slow.md (untouched) defers cleanly to pr-review.md as 'the serialized supervisor path'. Tests are meaningful, not hollow: review_lane_docs_test tripwires pair positive asserts with refutes of the old model across 7 docs; pr_review_command_test shells out to real bin/pr-review and its fast-mode codex_before_first_move==4 assertion uniquely requires non-blocking parallel spawn. NET: docs/script content is approvable; the block is purely red CI (flaky unrelated e2e) + stale cert — rebase on release, re-push for a clean CI run, and re-cert the full suite at head.

Handoff alex 2 days ago

Fresh CI GREEN at head 0f9159d6 after de-flaking the review_events e2e (scoped the Docs-link click by href + wait for the menu to open). Re-certified at head. The stale-cert was a false positive (fingerprint = head's tree). dor-check: DoR-to-Merge met, CI green 4 checks. Content approved by Alex (all 4 acceptance criteria met). Advancing to reviewed.

Sealed-bid sizing

Edit →

Alex (PM)

Avi (PO)

MEDIUM

Dev

MEDIUM

Actual

XL