Architecture

tokenuse keeps usage ingestion local: read local session files, append normalized records to its own archive, aggregate in memory, and render a dashboard. The TUI is the default frontend, and the Tauri desktop app is a second frontend over the same Rust core. There is no daemon and no file watcher. Network access is limited to explicit confirmed Config-page downloads, explicit Copilot / Claude.ai / ChatGPT (Codex) quota sync, and maintainer refresh flags. The Claude.ai and ChatGPT quota sync features store a session cookie locally in the OS keychain (via the keyring crate) and are gated behind the quota-sync Cargo feature.

Startup Flow

The durable archive lives at <config dir>/tokenuse/archive.db. If it already has rows, startup loads it immediately and queues an incremental background sync so the dashboard opens without reparsing every source. If the archive is empty, startup imports the legacy ~/.cache/tokenuse/ingest-cache.json snapshot when present, performs one synchronous source sync, then renders from the archive. If the archive cannot be opened or migrated, the app falls back to raw ingest::load() for that run.

Both Config pages can also clear local usage data after confirmation. That path deletes archive.db, recreates the schema, and immediately syncs local tool sources so per-source fingerprints are rebuilt from scratch. Config files, rates, pricing books, limit sidecars, legacy pricing snapshots, and generated reports are kept; archive-only history is lost if the original source files are no longer present.

The startup loader lives in src/runtime.rs so both frontends use the same config, currency, archive, fallback, and background refresh setup. The desktop app stores an App instance behind Tauri managed state and exposes narrow commands for filters, session drill-down, config actions, shortcuts, refresh, reports, desktop settings, and the tray popover. It also runs a small backend monitor that continues calling App::poll_reload() while the webview is hidden, drains queued background usage alerts, and sends native notifications from Rust. See Desktop app usage.

New sessions written while the dashboard is open are visible after archive sync — press r (Dashboard, Usage, or Session pages) to sync on a background thread. The dashboard stays responsive: the status bar shows reloading… while it runs, the next tick of the main loop drains completed results via App::poll_reload, and the status flips to reloaded · N calls. The refresher runs one sync at a time; if several results complete between UI ticks, the latest result wins. Failures or empty sync results keep the prior data unchanged.

Desktop background alerts use the unfiltered live archive totals as their baseline: cost in USD, activity tokens, and call count across all tools/projects. Automatic refresh deltas accumulate until one configured threshold crosses, then an alert is queued, the baseline resets to the new totals, and the cooldown starts. Manual refreshes reset the baseline without alerting. The thresholds live under background_alerts in config.json; sample-only startup data does not trigger alerts. Desktop-only startup preferences live under desktop in config.json and currently control open-at-login plus Dock/taskbar visibility.

Individual adapter discovery or parse errors are skipped so one malformed source does not stop the whole dashboard. If the archive has no calls or limits after sync, the UI shows sample data and a status message. Bundled sample data lives in src/data/sample_data.json and is embedded at build time.

Normalized Record

Every adapter emits ParsedCall from src/tools/types.rs. The important fields are:

Aggregation

The dashboard panels are built from the filtered call set:

• Summary: cost, call count, tool-qualified session count, cache hit rate, input, output, cache reads, and cache writes.
• Daily Activity: cost and calls by local date.
• By Project: projects with cost, average cost per session, and top tool spend mix.
• Top Sessions: sessions keyed by tool:session_id.
• Project Spend by Tool: project/tool rows with cost, calls, session count, and average cost per session.
• By Model: model display name, cost, calls, and cache percentage.
• Core Tools: normalized assistant tool calls.
• Shell Commands: first word of split Bash commands.
• MCP Servers: tool names shaped like mcp__server__tool, grouped by server.

App::sort is a runtime-only SortMode (Spend, Date, Tokens) and defaults to spend on launch. Aggregators carry cost, activity tokens (input + output + cache_creation + cache_read), and latest timestamp until rows are ordered; count-style tables split a call’s cost/tokens evenly across the row occurrences they emit while keeping occurrence counts unchanged. Dashboard views serialize as DashboardData, while Reports build a separate ReportDataset from raw Ingested calls and limits.

Pages And Modals

The TUI is a small state machine over six pages (Overview, Deep Dive, Usage, Insights, Config, Session) plus picker, confirmation, detail, and help modals. The first four pages are reachable through the tab strip via Tab / Shift-Tab or their direct keys; Config and Session are sub-pages opened from any tab. g cycles the global sort mode, and Shift-D toggles the visible data source between live and sample data when live data is available. Shortcut definitions, help groups, and footer hints live in src/keymap/keymap.json; src/keymap/mod.rs validates the embedded JSON and resolves keys to action IDs. src/app.rs applies those actions to state, while rendering is dispatched from src/ui/mod.rs.

• Overview (Page::Overview): default command-center landing page. Compact KPI strip plus a chronological activity pulse, models, project/tool spend, shell commands, and MCP servers. Acts as the at-a-glance landing for everyday use.
• Deep Dive (Page::DeepDive): analysis workbench with every panel listed under Aggregation, including a larger chronological activity trend, top sessions, model efficiency, and core tool counts that are not on Overview.
• Usage (Page::Usage): per-tool 24-hour console with an activity pulse, optional plan-side rate limit gauges, and top-3 models per tool. Built from Ingested::limits over the same ParsedCall set plus LimitSnapshot records. Entering Usage normalizes the visible period to Period::Today, the rolling 24-hour window; project filters are deliberately ignored, while sort mode controls section/model order. See TUI usage.
• Insights (Page::Insights): heuristic recommendations across model right-sizing, cache efficiency, anomalies, and quota pacing. Computed locally by crate::insights::compute_insights over Ingested.calls and Ingested.limits, with rolling 30-day baselines for outlier and z-score detection. Cards carry severity, scope, and an estimated weekly savings figure with the assumption stated inline. Rules live under src/insights/rules/; per-rule copy lives under insights.rules.<id> in src/copy/copy.json. The Tauri snapshot exposes the rendered view at DesktopSnapshot.insights. See Insights.
• Session (Page::Session): drill-down for one tool:session_id. Rendered from SessionDetailView, computed by filtering Ingested.calls by session_key(call) == key and sorting calls with the active sort mode. Live data shows per-call timestamp, model, cost, in/out tokens, cache, tools used, and a 120-char single-line prompt snippet; selecting a call opens a modal with the full stored prompt plus reasoning/web-search counts and bash commands. Sample mode shows a privacy note since per-call records are not bundled.
• Config (Page::Config): currency override, local data refresh actions (rates, pricing books, Claude/Copilot limit sidecars), and clear-data archive rebuild. The desktop frontend adds native-only controls for open-at-login and Dock/taskbar visibility on its Config page without changing the TUI state machine.
• Project picker, Currency picker, Session picker (*Modal structs): each holds options, a typeable query, and a filtered: Vec<usize> mapping; all three share the same case-insensitive substring filter pattern. The project picker pins All regardless of query.
• Report picker (ExportModal): report chooser for format, period, project/all-projects scope, and redaction. It defaults to the current period and project, always includes all tools, and writes HTML, PDF, SVG, PNG, JSON, Excel, or a CSV folder.
• Report folder picker (FolderPickerModal): directory-only picker rooted at the current report folder. Use this folder updates App::export_dir for the running session; Esc cancels without saving to config.json.
• Help (help_open: bool): full keybinding reference rendered from the shared keymap, openable from any page with h or ?. Closes with h, ?, or Esc.

The modal state is checked in priority order in App::handle_key: help, call detail, currency, clear-data confirmation, download confirmation, project, session, report folder picker, then report. The active context is passed to the keymap resolver before App applies the returned action. The folder picker is the only nested modal and sits on top of the report picker. The desktop app uses the same resolver through the handle_shortcut Tauri command, returning frontend effects for Svelte-owned modals and call-detail state.

Terminal graph primitives live in src/ui/graphs.rs. They provide relative block sparklines, ranked bars, and compact gauges for TUI panels without adding another charting dependency. The desktop frontend keeps the same visual language, but renders webview activity charts, tray sparklines, and rank heat strips with D3-backed Svelte SVG components and applies Motion JS actions for panel, modal, gauge, and tray transitions. DashboardData.activity_timeline is the chronological graph source for Overview and Deep Dive in both frontends: 24 Hours and 7 Days use hourly buckets, This Month uses hourly buckets until day 15 of the month and daily buckets after that, and 30 Days/All Time use daily buckets. Period::Today is a rolling last-24-hours filter based on the current time, not a local calendar-day filter. The desktop tray popover requests a dedicated 24-hour snapshot so opening it does not mutate the main window’s selected period, tool, project, or sort. DashboardData.daily remains the sort-aware table source.

Project Identity

Raw project strings come from each tool’s local data. Before display, tokenuse:

1. normalizes path separators and trims trailing slashes
2. folds absolute paths to the nearest existing Git root when one exists
3. groups costs by that identity across tools
4. displays the shortest unique suffix, such as tokens or dvr/tokens

cargo run -- --list-projects syncs the archive, then prints both the compact project label and the raw project value so ingestion mistakes are easier to spot.

Archive And Sync

src/archive.rs owns the SQLite archive. It stores full ParsedCall rows, append-only limit snapshots, and per-source fingerprints in source_state. Calls are unique on (tool, dedup_key), so a changed source can be reparsed safely without duplicating historical calls. Source deletion never removes archive rows; once tokenuse has imported a call, it remains available even if the original tool history is later cleared.

The source fingerprint hook defaults to file metadata for file-backed sources and recursive directory metadata for directory-backed sources. Sources are tagged as session or limit sources. Session sources must parse calls successfully before their fingerprint is advanced; limit sidecars must parse limit snapshots successfully before their fingerprint is advanced. When a source fingerprint has not changed, sync skips parsing it. When it changes, sync parses the source, inserts only new call keys, stores any new limit snapshots, and updates the fingerprint.

Codex imports limit snapshots from the same rollout JSONL files as calls. Claude Code and Copilot import optional local sidecars from <config dir>/tokenuse/limits/: Claude Code reads a status-line JSON capture, while Copilot reads the local copilot.json written by the confirmed Config-page sync action. The opt-in claude_subscription and codex_subscription adapters write claude_subscription.json and codex_subscription.json sidecars from the same directory; they call Claude.ai’s and ChatGPT’s user-facing usage endpoints with a session cookie pulled from the OS keychain, and tag the resulting LimitSnapshot rows with the existing claude-code / codex tool IDs so gauges appear inside those sections.

The old JSON ingest cache is now legacy seed input only. New runs do not write ~/.cache/tokenuse/ingest-cache.json.

Deduplication

A single shared HashSet<String> is passed through every adapter during a run. Each parser creates a stable dedup_key for the call shape it understands:

• Claude Code: message id, falling back to timestamp
• Cursor bubbles: conversation id, timestamp, and token counts
• Cursor Agent KV: request id
• Cursor Agent transcripts: transcript path, conversation id, and turn index
• Codex: rollout path, token event timestamp, and cumulative token totals
• Copilot: session id and message id
• Gemini: session id and message id

Session counts are tool-qualified, so claude-code:s1 and codex:s1 remain separate sessions even if the raw session id text matches.

Pricing

Pricing is embedded as two compile-time books under costs/. At runtime, PriceTable::configured() first looks for local pricing-upstream.json and pricing-overrides.json in the tokenuse config directory, then falls back to the embedded books. A legacy local pricing-snapshot.json is still accepted for older installs.

Canonicalization lowercases model names, drops a vendor prefix such as anthropic/, strips an @pin suffix, and removes trailing -YYYYMMDD date stamps. Aliases such as anthropic-auto and openai-auto resolve through the overrides book; cursor-auto is a direct Cursor Auto pricing row. Tool aliases are scoped, so Copilot display names do not affect Codex/OpenAI/Claude/Gemini calls.

The pricing formula is:

cost = multiplier * (
    input_tokens * input_rate
  + output_tokens * output_rate
  + cache_creation_input_tokens * cache_write_rate
  + cache_read_input_tokens * cache_read_rate
  + web_search_requests * web_search_rate
)

Claude Opus fast mode uses the model row’s fast_multiplier when present. Cache-rate labels in the UI are derived from cache_read_rate / input_rate, not from observed cache-hit percentage. The maintainer CLI refresh command reads pricing-sources.json, fetches configured upstream feeds and official-source tables, then rewrites both checked-in books:

cargo run -- --refresh-prices

The TUI and desktop configuration pages can also download the published pricing books into the local config directory after confirmation and reload pricing in-process. Because the archive stores cost_usd at import time, refreshed pricing applies to newly imported calls; existing historical rows keep their original USD cost. Builds made with --no-default-features compile without these download actions.

See Pricing and cache rates for provider source quotes, current cache-read multipliers, and parser caveats.

Reports

Press e on Dashboard, Usage, or Session pages to open the report picker. Output defaults to the user’s Downloads folder, falling back to ~/Downloads and then <config dir>/tokenuse/reports/ if the platform does not expose a Downloads directory. Press f or b inside the report picker to choose another folder for the current TUI session. Report files never overwrite prior runs: every filename is timestamped with YYYYMMDDTHHMMSS and slugged with the chosen period and project scope.

Reports are built from raw Ingested calls and limits through ReportDataset, not from the visible dashboard snapshot. Scope is period plus project or all projects; tools are always included together. Redaction is off by default and, when enabled, replaces prompts, shell commands, raw paths, session IDs, and dedup keys with report-local placeholders while preserving totals and costs.

The report pipeline depends on plotters for SVG/PNG summaries, fulgur for browserless HTML/CSS-to-PDF rendering, rust_xlsxwriter for Excel, and serde_json for JSON. HTML generation is hand-written, escaped at render time, and uses no external scripts or network dependency. Full raw data lives in JSON, Excel, and CSV outputs rather than the visual deck/summary reports.

Configuration And Currency

Runtime settings live in the platform config directory under tokenuse:

USD is the default display currency. The dashboard still stores calculated spend as cost_usd; aggregation sums USD and formats the final display values through the active currency table.

The clear-data Config action deletes and recreates archive.db, then reimports local tool history immediately. Rebuilt rows are priced with the current configured pricing table. It intentionally does not delete config.json, local exchange-rates.json, legacy rates.json, local pricing books, legacy pricing-snapshot.json, or generated reports.

costs/exchange-rates.json is the embedded fallback snapshot. The TUI and desktop configuration pages can download the latest published copy after confirmation from:

https://raw.githubusercontent.com/russmckendrick/tokenuse/refs/heads/main/costs/exchange-rates.json

That local rates download writes <config dir>/tokenuse/exchange-rates.json and reloads the currency table immediately. Existing local <config dir>/tokenuse/rates.json files are still accepted as a legacy fallback. Builds made with --no-default-features compile without this download action.

The snapshot is generated from Frankfurter’s USD-based v2 rates endpoint, filtered to fiat display currencies, and refreshed by a weekly GitHub Action:

cargo run -- --generate-currency-json