Skip to content

v0.22 Scale Evidence — 2026-08-17

This page records completed credential-free scale runs for the v0.22 admission changes. It is a reviewed evidence summary, not a live-provider throughput claim.

Environment and revision

Both reports were produced from behavioral revision ffd2f1e26da9883ffd73b0f6f0949d41a629df21 on:

  • CPython 3.12.12, Linux x86-64;
  • 32 reported CPUs;
  • report schema 2;
  • package metadata 0.21.0 because the locked plan requires the 0.22 metadata bump only after behavioral, documentation, scale, and packaging gates pass.

The harness records no hostname, username, home path, prompt, output, or raw quota-scope representation.

Full 100k profile

Command actually run:

make scale-100k

The generated report was benchmark-results/scale-100k.json (103,220 bytes, intentionally gitignored), generated at 2026-08-17T19:56:04.858109Z. Its SHA-256 is:

08d6686a5d3ac6a9f45da7cfc490d1793fb92e395e4df649183325e8c737809c

All nine scenarios passed all 115 assertions:

Scenario Items Result
healthy 100,000 Passed
slow_consumer 100,000 Passed
rate_limit_wave 100,000 Passed
transport_retry 100,000 Passed
validation_recovery 100,000 Passed
stop_resume 100,000-source restart cycle Passed
artifact_bench 100,000 SQLite records Passed
progress_overhead 20,000 per mode Passed
token_quota_mixed 100,000 Passed

The artifact scenario appended 100,000 SQLite records at 15,020.0 records/s, truncated its WAL to zero on close, reopened in 0.0015s, completed 2,000 indexed lookups at 23,037.6 lookups/s, and iterated at 59,587.6 records/s. Its JSONL comparison deliberately remained at 20,000 records. These are local fake-data measurements, not storage guarantees on other hardware.

Mixed-token evidence

The new scenario used the real lazy process_stream() surface, concurrency 64, input/result bounds 512/512, two independent quota scopes sharing one provider capacity scope, a real 60-second local quota window, and a 2,307,692 TPM bucket per scope. It completed in 58.654s (1,704.9 items/s):

  • exact bands: 94,000 small, 5,000 medium, 1,000 large;
  • exact digest count 100,000 and modular index sum 4,999,950,000;
  • 100,002 provider calls: 100,000 successes plus one recoverable known-usage retry and one unknown-usage transport retry;
  • 5,502,100 reserved tokens: 3,526,060 estimated input plus 1,976,040 estimated output;
  • 5,719,990 reported/reconciled tokens;
  • 282,060 refunded tokens and 500,000 underestimation-debt tokens;
  • one known-zero attempt and one unknown-usage attempt;
  • 100,002 QUOTA_ADMITTED and 100,002 QUOTA_RECONCILED events;
  • dispositions: 94,002 refunded, 1,000 debt, 4,999 exact, 1 retained unknown;
  • 497.483250 aggregate quota-wait seconds across concurrent attempts and 4.169630s maximum wait; p50/p95/p99 were zero because fewer than one percent of the 100,002 attempts waited;
  • provider peak concurrency 48 within the shared bound 64, with zero simulated provider rate-limit/overload failures;
  • queue high-water marks 512 input and 33 results, both within their bounds;
  • both scopes admitted and reconciled 50,001 attempts; no quota/coordinator background task remained;
  • post-warmup RSS growth 0 bytes in the sampled current-RSS series; task, thread, FD, and store-worker counts returned to baseline.

This proves the local integration's exact aggregate accounting, real waiting, bounded retention, multi-scope behavior, and cleanup for this deterministic workload. It does not prove equivalence with a provider's fixed/rolling window, live-provider throughput, gateway-hidden retry visibility, distributed quota coordination, or active GPU sequence/KV-cache scheduling.

One-million-item post-refactor run

Command actually run:

uv run python -m benchmarks.scale_soak \
  --profile 1m \
  --scenario healthy \
  --output benchmark-results/scale-1m-healthy.json

The generated report was 19,490 bytes, generated at 2026-08-17T20:06:01.915182Z, with SHA-256:

b049dae88ae321cff6b888da2a2370b1f2519b8f0cf096c825ca7a998bd0f3c3

The healthy scenario passed all 11 assertions:

  • 1,000,000 accepted, processed, succeeded, and provider calls; zero failures, retries, cooldowns, or rate-limit hits;
  • exact 10,000,000 reported tokens;
  • 559.552s report wall time and 1,787.1 items/s;
  • provider peak concurrency 64;
  • input/result queue highs 512/64 within 512/512 bounds;
  • SQLite WAL peak 4,416,672 bytes and clean close;
  • 5,963,776 bytes sampled post-warmup current-RSS growth;
  • task/thread/FD/store-worker counts returned to baseline.

Only healthy was run at one million items for this Session C gate. slow_consumer, stop_resume, and artifact_bench remain runnable members of the default 1m profile but were not run here and are not claimed as covered by this evidence.