McRitchie Studio
Say Hi 👋
Agents Builders
← All Tasks
Sizing

Register QA test commands

Archived
register-qa-test-commands

Created

Jul 04, 04:11

Started

Jul 04, 04:52

Completed

Jul 04, 08:31

DevOps handoff

Type

Chore

Shape

backend

Worktree Slug

register-qa-test-commands

Repositories

mcritchie-studio

Release Slug

—

Branch

feat/register-qa-test-commands

Pull Request

https://github.com/amcritchie/mcritchie-studio/pull/366

Local URL

—

QA URL

—

Production URL

—

devops

Acceptance Criteria

  • qa_test_cmd registered per app in release_repos.yml
  • Steffon pre-QA gate runs registered tiers before deploy
  • Quoted-arg command strings parse correctly or documented

Expected Test Plan

  • [unit] registry qa_test_cmd values + argv parsing pinned both ways
  • [integration] pre-QA gate real git checkout + subprocess run with quoted arg

Checks Run

  • [unit] bin/rails test test/models/release/repos_test.rb — qa_test_cmd registry contract (values, flag-style parse parity, nil for planned apps/gems)
  • [unit] bin/rails test test/lib/release_cli_test.rb — test_cmd_argv pinned both ways (flag-style == split, quoted arg = one element), unbalanced-quote abort, gate+ship argv via stubbed sh, dry-run malformed abort
  • [integration] release_cli_test pre-QA gate vs REAL git sibling (bare origin, main/release) + REAL subprocess — quoted arg arrives intact, main restored via ensure
  • [integration] registered command runnable as the gate invokes it: bin/rails test test/integration green in sibling turf-monster (79 runs) and rolio (77 runs)
  • [full-suite@bffb51c251aebadad3595ee95624ae41729f2802] bin/rails test green
  • [rubocop@bffb51c251aebadad3595ee95624ae41729f2802] bin/rubocop clean

Agent Context

Follow-up filed from the move-release-assembly-to-steffon review (PR #362, activity-729). The new pre-QA gate in bin/release prepare is registry-driven via optional qa_test_cmd per app in config/release_repos.yml and shipped with NO values registered — every app self-gates (prints 'self-gates … skip'), so QA-green currently means deploy+boot+smoke /up only. Register real tier commands (integration + e2e smoke) per app. Known latent limitation flagged by review: the gate's cmd.split can't handle quoted args (mirrors ship's existing test_gate) — either fix the parsing (Shellwords) or document the constraint when registering. Depends on move-release-assembly-to-steffon shipping.

Stage Timeline

Who handled each stage, the time it took (measured), and the model / tokens / cost reported (best-effort) — plus who's on it right now. — means the agent didn't report that metric.

Sizing Avi · PO SMALL Dev SMALL Actual XL ≠ forecast
  1. Created → Designed
    F Fearow
    Fearow
    Model
    —
    Duration
    —
    Tokens
    —
    Cost
    —
    Completed Jul 04, 04:11 · 4 days ago
    api
  2. Designed → Building
    F Fearow
    Fearow
    Model
    claude-fable-5
    Duration
    41 minutes
    Tokens
    10,353,224
    Cost
    ~$17.90
    Started Jul 04, 04:11
    Completed Jul 04, 04:52 · 4 days ago
    cli
  3. Building → Submitted
    F Fearow
    Fearow
    Model
    claude-fable-5
    Duration
    about 3 hours
    Tokens
    131,538
    Cost
    —
    Started Jul 04, 04:52
    Completed Jul 04, 07:40 · 4 days ago
    cli
  4. Submitted → Reviewed Review Activity
    C Carl
    Carl primary
    J Jasper
    Jasper light
    Model
    claude-fable-5
    Duration
    12 minutes
    Tokens
    10,998,034
    Cost
    ~$25.14
    Started Jul 04, 07:40
    Completed Jul 04, 07:52 · 4 days ago
    cli
  5. Reviewed → Assembled
    S Steffon
    Steffon
    Model
    claude-fable-5
    Duration
    10 minutes
    Tokens
    3,078,471
    Cost
    ~$4.58
    Started Jul 04, 07:52
    Completed Jul 04, 08:02 · 4 days ago
  6. Assembled → Shipped
    A Avi
    Avi
    Model
    gpt-5.5
    Duration
    29 minutes
    Tokens
    —
    Cost
    —
    Started Jul 04, 08:02
    Completed Jul 04, 08:31 · 4 days ago
  7. Shipped → Archived
    —
    Model
    —
    Duration
    about 12 hours
    Tokens
    —
    Cost
    —
    Started Jul 04, 08:31
    Completed Jul 04, 20:46 · 4 days ago

Conversation

QA review feedback, agent handoffs, and follow-up notes for this task.

Handoff 4 days ago

PR #366: qa_test_cmd = 'bin/rails test test/integration' registered for mcr/turf/rolio (tax-studio + chain-ops stay self-gating, documented). Quoting FIXED via shared Shellwords test_cmd_argv in both pre-QA gate and ship test_gate — flag-style pinned identical, quoted args survive, unbalanced quote aborts fast (even dry-run). 12 new tests incl. real-git integration run; suite certified 2694/0F at bffb51c.

Comment 4 days ago

Carl review (PRIMARY, review-one) — ALL-CLEAR, moving to reviewed. Non-blocking follow-ups: (1) ship-seam parity pin asymmetry — repos_test pins qa_test_cmd flag-style parse parity at the registry, but no equivalent registry-level pin for ship's test_cmd values (Shellwords eats backslashes silently; current values 'bin/rails test' x2 verified safe + semantics documented in YAML header — nit, ~3-line test in a follow-up). (2) Jasper light: run_cli doc comment 'MUST include --dry-run' (release_cli_test.rb:147-149) now stale — new tests correctly pass --yes with stubbed/hermetic sh. (3) Release::Repos.qa_test_cmd has no runtime consumer yet (CLI reads via app_meta_for) — consistent API surface, fine. Cert verified cryptographically: PR head tree == bffb51c2 fingerprint.

Sealed-bid sizing

Edit →

Alex (PM)

—

Avi (PO)

SMALL

Dev

SMALL

Actual

XL