McRitchie Studio
Say Hi 👋
Agents Builders
← All Tasks
Sizing

Fresh tokens accurate cost

Archived
fresh-tokens-accurate-cost

Created

Jul 01, 16:04

Started

Jul 01, 16:09

Completed

Jul 01, 16:39

DevOps handoff

Type

Feature

Shape

backend

Worktree Slug

fresh-tokens-accurate-cost

Repositories

mcritchie-studio

Release Slug

—

Branch

feat/fresh-tokens-accurate-cost

Pull Request

https://github.com/amcritchie/mcritchie-studio/pull/317

Local URL

—

QA URL

—

Production URL

—

tooling

Acceptance Criteria

  • Tokens shows fresh spend excluding cache reads
  • Cost prices cache tiers at correct rates
  • Cache-read tokens stored separate from fresh
  • Span and session totals stay turn-deduped

Expected Test Plan

  • unit
  • integration

Checks Run

  • [unit] turn_usage splits fresh (input+cache_creation) from cache_read
  • [unit] build_payload stamps cache_read_tokens apart from fresh in/out
  • [unit] cost_for prices cache_read at 0.1x input, sums fresh at full rate
  • [unit] cost_for stays nil for a model with no rate, even for cache_read
  • [unit] heartbeat token display + usage totals exclude cache_read (fresh only)
  • [integration] capture stores cache_read_tokens, prices cache tier into cost
  • [integration] controller permits+coerces cache_read_tokens; cost stays server-derived
  • [integration] usage totals stay turn-deduped by source_turn_uuid
  • [full-suite@d9a68d47eb8bb6dc270d2af17ddf43bef976ed29] bin/rails test green
  • [rubocop@d9a68d47eb8bb6dc270d2af17ddf43bef976ed29] bin/rubocop clean

Agent Context

Refine the just-shipped token/cost capture (capture-action-tokens-and-cost-2, now on prod). PROBLEM verified live: tokens_in = input + cache_creation + cache_read all LUMPED, so on a long session it's ~300K/action dominated by CACHE READS (re-read context), and cost prices ALL of tokens_in at full input rate -> cost overstated ~10x (a real turn showed 309313 tokens / $1.58; true cost ~$0.15). Operator decision: TOKENS column shows FRESH tokens (new spend, exclude cache reads); COST priced with correct cache tiers. CHANGES: (1) Hook bin/atomic-capture-hook : stop lumping. tokens_in = input_tokens + cache_creation_input_tokens (fresh input processed this turn); tokens_out = output_tokens; NEW column cache_read_tokens = cache_read_input_tokens (re-used context, for cost only). (2) Migration: add cache_read_tokens to atomic_actions (nullable default 0); controller permits it; cost stays SERVER-DERIVED (not caller-set). (3) Cost in app/models/atomic_action.rb cost_for: price per tier = tokens_in*in_rate + cache_read_tokens*(in_rate*0.10) + tokens_out*out_rate; keep NULL when no rate for the model. (cache_creation lumped into tokens_in at 1x is acceptable best-effort vs its true 1.25x — note it in a comment; or store separately if easy.) (4) Display: heartbeat TOKENS column + drawer = tokens_in + tokens_out (fresh, EXCLUDES cache_read). heartbeat_usage_totals sums fresh tokens_in/out + cost, STILL deduped by source_turn_uuid (unchanged); cache_read need not be displayed. Update heartbeat_helper + _trajectory_table + _event_table + _drawer as needed. (5) Tests: usage split (fresh vs cache_read), tiered cost (cache_read at 0.1x + null-when-no-rate), fresh-token display, dedupe unchanged. VERIFY against a real transcript: a long-session turn should show fresh ~1-8K tokens + cost ~cents, NOT 300K/$1.58. Files: bin/atomic-capture-hook, app/models/atomic_action.rb, migration, app/controllers/api/v1/atomic_actions_controller.rb, app/helpers/heartbeat_helper.rb, heartbeat views. Part of narrated-events epic — project_opsd_learning_layer, [[feedback_token_capture_fragility]]. NOTE: the hook usage-folding method is turn_usage in bin/atomic-capture-hook (that word was lost to a shell-quoting slip).

Stage Timeline

Who handled each stage, the time it took (measured), and the model / tokens / cost reported (best-effort) — plus who's on it right now. — means the agent didn't report that metric.

Sizing Avi · PO SMALL Dev SMALL Actual XL ≠ forecast
  1. Created → Designed
    C Caterpie
    Caterpie
    Model
    —
    Duration
    —
    Tokens
    —
    Cost
    —
    Completed Jul 01, 16:04 · 7 days ago
    api
  2. Designed → Building
    C Caterpie
    Caterpie
    Model
    claude-opus-4-8
    Duration
    5 minutes
    Tokens
    11,082,577
    Cost
    ~$7.30
    Started Jul 01, 16:04
    Completed Jul 01, 16:09 · 7 days ago
    cli
  3. Building → Submitted
    C Caterpie
    Caterpie
    Model
    claude-opus-4-8
    Duration
    15 minutes
    Tokens
    —
    Cost
    —
    Started Jul 01, 16:09
    Completed Jul 01, 16:24 · 7 days ago
    cli
  4. Submitted → Reviewed Review Activity
    C Carl
    Carl primary
    S Shannon
    Shannon light
    Model
    claude-opus-4-8
    Duration
    under a minute
    Tokens
    1,784,230
    Cost
    ~$1.02
    Started Jul 01, 16:24
    Completed Jul 01, 16:25 · 7 days ago
    cli
  5. Reviewed → Assembled
    S Steffon
    Steffon
    Model
    claude-opus-4-8
    Duration
    under a minute
    Tokens
    1,074,639
    Cost
    ~$0.60
    Started Jul 01, 16:25
    Completed Jul 01, 16:26 · 7 days ago
  6. Assembled → Shipped
    A Avi
    Avi
    Model
    claude-opus-4-8
    Duration
    13 minutes
    Tokens
    5,060,333
    Cost
    ~$2.82
    Started Jul 01, 16:26
    Completed Jul 01, 16:39 · 7 days ago
  7. Shipped → Archived
    —
    Model
    —
    Duration
    2 days
    Tokens
    —
    Cost
    —
    Started Jul 01, 16:39
    Completed Jul 04, 03:23 · 4 days ago

Conversation

QA review feedback, agent handoffs, and follow-up notes for this task.

No task feedback has been recorded yet.

Sealed-bid sizing

Edit →

Alex (PM)

—

Avi (PO)

SMALL

Dev

SMALL

Actual

XL