v1.2.0 - Total Recall
tokenuse 1.2.0, named Total Recall, makes your whole history reachable. Scrollback adds full-text search across every archived session transcript on both surfaces, and the read-only MCP server can now serve the same local numbers to an LLM client over stdio or a loopback HTTP endpoint. Around those two headline features, spend is now broken down by task activity, the desktop app gains session and project drill-downs, the TUI gains its own Coach page, and the parsers get another accuracy pass. Everything is still computed from the usage files already on disk — no network calls, no LLM in the loop.
Highlights
- Scrollback: full-text transcript search across every archived session — a TUI tab (
/from any page) and a desktop sidebar page, backed by an SQLite FTS5 index inarchive.db(schema v7). - MCP server:
tokenuse mcpexposes read-onlystatus,overview,projects, andscrollbacktools over stdio, and can now also serve over a loopback HTTP endpoint so it runs alongside the desktop app. A new desktop Config panel hosts the opt-in HTTP toggle, endpoint, masked bearer token, and a copyableclaude mcp addcommand. - By Activity: every call is classified into one of thirteen deterministic task categories (no LLM), surfaced on the desktop Analytics page, the TUI Deep Dive page,
tokenuse overview, and in every report format. - Drill-downs: Top Sessions rows open the full session view on both surfaces, and the desktop Projects page becomes a two-level index → dedicated project page.
- TUI Coach page (
k): the same grade, findings, and advisory Setup panel as the desktop Coach page. - Scriptable summaries:
tokenuse statusandtokenuse overviewprint totals without the TUI, both honouring the configured currency and taking--json. tokenuse doctor: read-only per-tool diagnostics, mirrored in a desktop Config Data Sources panel.- Another parser accuracy pass across Claude Code, Codex, Copilot, and Cursor, plus a sample dataset that now exercises the whole schema.
Scrollback
Scrollback is full-text search across every archived session transcript, on both surfaces.
- TUI: a Scrollback tab, opened with
/from any page (search input focused). Type a phrase, press Enter; results group per session with role-tagged, term-highlighted snippets and a match count. Opening one drills into the full session view and returning restores the query and results. - Desktop: a Scrollback sidebar page with search-as-you-type plus tool and project filters, the same per-session grouping and drill-in.
- Search honours the active tool and project filters and always reads the live archive, even in sample-data mode.
To power it, all five tool parsers now capture full user and assistant turn text — thinking/reasoning text stays excluded everywhere — into a transcript store with an FTS5 index inside archive.db. Existing archives migrate automatically: already-archived truncated prompts are seeded as prompt-only fallbacks so sessions whose source files are gone stay searchable (badged "prompt only" in results), and surviving sources re-parse once to upgrade to full text.
MCP server
tokenuse mcp is a read-only MCP server for LLM clients. It exposes four tools with the same numbers as the scriptable CLI summaries:
status— a one-line rolling-24-hour and month summary.overview— a month summary with per-tool spend, top models and projects, and a daily table.projects— per-project spend, with names pseudonymised via a locally persisted salt unless started with--real-names.scrollback— full-text transcript search with a requiredqueryplus optionalproject,tool, andlimit; results carry each matching session's tool, project, last timestamp, cost, match count, and plain-text snippets.
stdio remains the default. The server can now also serve over loopback HTTP so it runs while the desktop app is open:
- CLI:
tokenuse mcp --http [--port N] [--real-names](default port 20151). - Desktop: an opt-in MCP Server panel on the Config page. While enabled, the app serves the same four tools on
http://127.0.0.1:<port>/mcp, guarded by a bearer token stored owner-readable-only at<config dir>/tokenuse/mcp-token(delete the file to rotate it). The panel shows the endpoint, running state, and masked token with reveal/copy buttons, plus a copyableclaude mcp addcommand; the token never travels in the background snapshot poll.
The listener binds loopback only, requires the token on every request, rejects non-localhost Host/Origin headers (DNS-rebinding defence), and makes no outbound calls. Project names stay pseudonymised on the desktop endpoint always; --real-names remains CLI-only. See docs/development/mcp-server.md for the tool schemas and security model.
Spend by activity
Every call is classified into one of thirteen deterministic task categories — coding, debugging, feature dev, refactoring, testing, exploration, planning, delegation, git ops, build & deploy, brainstorming, conversation, general — from its tool usage and prompt. No LLM is involved.
- A By Activity panel appears on the desktop Analytics page, the TUI Deep Dive page, and in
tokenuse overview. - Reports carry the same breakdown: JSON gains a
task_categoriesblock, Excel a "By Activity" sheet, and the CSV folder atask_categories.csv, each row with the category label, calls, tokens, and cost.
Desktop drill-downs and layout
- Session drill-down: Top Sessions rows link to the full session view on both surfaces. On the desktop, Analytics and per-tool rows open the session view (click,
Enter, orSpace) and closing it returns to the originating page with its scroll position restored — the back button names that page. In the TUI, clicking a Top Sessions row on Deep Dive opens that session's page. - Projects page: now a two-level drill-down. The page itself is a full, uncapped index of every project in the period (cost, avg/session, sessions, calls, last active, tool mix). Selecting a row — or any project row on Overview, Analytics, or a tool page — opens that project's dedicated page: KPIs, activity pulse, full session list, a per-tool split donut (cost/calls/sessions/avg), top models, By Activity, core tools, shell commands, MCP servers, and the raw per-tool source paths. In the TUI, clicking a By Project row on Deep Dive applies that project as the active filter.
- Tools screen: no longer repeats the per-tool consoles. It is now a rolling 24-hour overview — one compact KPI card per tool with cost, activity pulse, calls/tokens/last-seen, primary limit gauges, and the plan-value line — ordered busiest-first, with the busiest tool's card spanning the row as a spotlight. In the sidebar, Tools moves down to sit above the tool rows it summarises. The TUI Usage page is unchanged.
- Ranked tables on both surfaces drop the leading rank-meter column and paint each row's relative magnitude as a muted blue wash with a thin terminus edge; the reclaimed width goes to project and model names, with exact percentages in tooltips and screen-reader text.
- Equal-height cards: side-by-side panels on every desktop page now stretch to one flush bottom edge instead of ragged content-sized bottoms; capped lists still scroll internally at 480px.
- The Analytics spend-by-tool stack switches to hourly buckets on the 24 Hours period (titled "Hourly spend by tool").
Coach and insight
- TUI Coach page (
k, or Tab-cycle): overall grade, per-group scores, triggered findings with occurrence lines, and the advisory Setup panel — the same data and copy as the desktop Coach page, honouring the active period, tool, and project filters. - Coach Setup panel (desktop): flags unused configured MCP servers, CLAUDE.md files over 200 lines after
@importexpansion, repeated file re-reads inside sessions, and reads under build/dependency directories — each with a heuristic token-savings estimate. Setup findings never affect the practice grade. - New Coach rule Retry Loops (session hygiene, medium): flags periods where a high share of edited files needed a re-edit after a shell run — the change didn't survive its own verification. Detection is per session and file at assistant-round granularity, only for tools that report edited files (Claude Code, Codex, Cursor).
- Plan-value tracking: Usage consoles (TUI and desktop) show what the calendar month's tokens would have cost at API prices against the tool's monthly subscription price, with the resulting value multiple. Prices are configured in the desktop Config page's Plan Value panel (
plan_pricesinconfig.json, USD); detected ChatGPT and Copilot plans get a built-in default until one is set, and org-paid tiers never do. - Fallback-priced models are now surfaced instead of blending in: the Config page (TUI and desktop) warns with the affected
tool · modelpairs and the fix hint, and report metadata carries afallback_priced_modelsrow. The desktop warning is now warning-toned.
Diagnostics and scriptable output
tokenuse doctor: read-only per-tool diagnostics. Shows every location each adapter probes (and whether it exists), the environment overrides in effect, discovered session/limit source counts, and a bounded parse sample, ending in anOK/NOTHING FOUND/ERRORS/DISCOVERY FAILEDverdict with the likely cause.--jsonemits the same report machine-readable.- The same diagnostics reach the desktop: a Data Sources panel on the Config page runs the doctor report on demand (never from the background poll) with per-tool verdict badges, probed locations, env overrides, source counts, and parse-sample results.
tokenuse statusandtokenuse overview: scriptable summaries without the TUI.statusis a one-liner with rolling-24-hour and month totals;overviewis a copy-pasteable month summary. Both respect the configured currency and take--json.- The Config page's Clear Data row on both surfaces now leads with the archive size (for example
Archive 12.3 MiB incl. transcript index) oncearchive.dbexists, so transcript-index growth stays visible.
Parser and ingestion accuracy
- Claude Code sync is now incremental at the byte level: grown session files resume from a stored per-file byte offset (with a prefix probe guarding against rewrites) instead of re-parsing every session. On a ~50MB project a one-file append resyncs in ~0.04s versus ~0.4s before, byte-identical. The sync status line appends
· N tail-resumedwhen it happens. - Claude Code parsing now merges every streamed JSONL line of an assistant message into one call; the previous first-line-wins dedup silently dropped tool calls, shell commands, edited/read files, and response text carried on later lines. Token counts and costs are unchanged, but Core Tools, MCP Servers, shell command counts, and Coach signals become noticeably more complete after the automatic reparse.
- Nested Claude Code workflow subagent transcripts (
subagents/workflows/<workflow>/agent-*.jsonl) and session-scoped subagent transcripts (<project>/<session-id>/subagents/agent-*.jsonl) are now ingested. - Claude Code advisor runs are now counted — each
advisor_messageiteration is a separately billed API call with its own model and token buckets — and 1-hour cache writes now price at Anthropic's published premium (2× base input vs 1.25× for the 5-minute TTL). - Codex: calls are deduplicated by session lineage (content-addressed cumulative totals) instead of file path, so UI-forked rollouts no longer double-count; archives migrate automatically and keep each row's import-time cost. Sessions archived from the Codex UI (
~/.codex/archived_sessions/) are now discovered and ingested. - Copilot: VS Code now reads the Copilot Chat extension's OpenTelemetry span store (
agent-traces.db) — the one VS Code source with real input, output, and cache counts — plus VS Code core chat-session journals; per variant the most authoritative source wins so estimates never double count. VSCodium paths are covered, and legacy Copilot CLI sessions recover real input and cache counts from theirsession.shutdownrollups. - Cursor: input tokens now come from Cursor's own per-conversation context meter (
promptTokenBreakdown) when bubbles carry no explicit counts, credited once per conversation; conversations resolve their project from VS Code workspace storage; and Composer house models (composer-1, retiredcomposer-1.5/composer-2) all price at Cursor's published rates instead of the global fallback.
Sample data
Sample data (shown with no local sessions, or forced with Shift+D) now exercises the whole schema instead of leaving newer panels blank. All five tools carry spend across every period — Cursor and Gemini were previously in the Usage gauges only, so they now appear in the Models table, Model Catalog, provider/tool share, and project tool-mix too. The By Activity panel is populated, drilling into a sample session opens a full per-call ledger that reconciles to the session total, the Claude Code Usage section gains a 5h/weekly gauge, and the Coach sample's project names are anonymised to match the rest of the demo set.
Upgrade notes
- The local archive schema moves to version 7 on first open, adding the transcript store and its FTS5 index. Existing archives migrate automatically; sources still on disk re-parse once to capture full transcript text, so expect the first ingest after upgrading to take longer than usual. Truncated prompts for sessions whose sources are gone are seeded as prompt-only fallbacks.
- Older tokenuse binaries refuse to open a version 7 archive. To downgrade after opening 1.2.0, delete
archive.dband let the older version rebuild it from the source files still present on disk, backing up the database first if archive-only history matters. - Local usage analysis remains local. Network use stays limited to confirmed Config-page downloads, opt-in quota sync, Windows/Linux update checks, and maintainer refresh or release paths. The one new inbound surface — the opt-in MCP HTTP endpoint — is loopback-only, bearer-token-gated, and makes no outbound calls.
- The release tag should be
v1.2.0and must match the root Cargo package, desktop Cargo package, desktoppackage.json, and Tauri config versions.
Full Changelog: https://github.com/russmckendrick/tokenuse/compare/v1.1.1...v1.2.0
Full Changelog: https://github.com/russmckendrick/tokenuse/compare/v1.1.1...v1.2.0