You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(server): usage summaries no longer wait on a forced pricing refresh (20 s → 2.7 ms median) - #30039
While one client runs a manual usage Refresh, every other ordinary usage summary read on that environment waits for the refresh to finish, even though its cached model rates are still valid. Against a slow or unreachable pricing endpoint, the refresh makes two sequential 10-second attempts (raw GitHub, then the GitHub API fallback), so those reads take about 20 s.
Change
In apps/server/src/usage/UsageService.ts, ensureRates acquired ratesLock before loadRates checked the 24-hour TTL. A non-forced read with a valid table therefore queued behind a forced refresh holding the lock for its HTTP attempts.
Non-forced reads now check the TTL before taking the lock and use the valid table directly. Forced refreshes, expired caches (same strict < TTL boundary), missing caches and the 60 s refresh floor still go through the lock as before.
scanSummary captures the pricing metadata at the same moment it hands rates to the UsageAggregator. Before, it read pricing() at the end of the scan, so a refresh that landed mid-scan (for example during Codex service-tier resolution) labelled old-priced buckets with the new table's fetchedAt. That race already exists on main; the fast path would have made it easier to hit.
scanKey now includes the table's fetch time. The client's Refresh calls refreshUsageRates and then re-reads the summary. Without this, that re-read could join an in-flight scan priced with the old table. That case was possible on main too and is more likely with the fast path.
There are no contract, client or provider changes. Web, desktop and mobile share the same RPCs and gain the same benefit. I did not build stale-while-refresh pricing for cold or expired caches. That would need a pending-cost state in the clients.
Scope and approval
This is a focused performance and reliability fix, which CONTRIBUTING.md lists as most likely to be accepted, and it fits the small, focused bug-fix exception. Ordinary reads waited on an unrelated lock for no reason. The change is three small edits in one service. It adds no feature, changes no product defaults or contracts, and keeps manual Refresh semantics.
Verification
Benchmark: a temporary it.live case (not committed) uses the real UsageService, a 200-record Claude transcript, a seeded valid rates cache, and a local node:http stub that holds every request for 15 s, longer than the service's 10 s per-attempt timeout. Each repeat uses a fresh service. It starts refreshRates, waits until the stub receives the request, then times one ordinary readSummary.
Ordinary read latency
main
this branch
During a forced refresh (7 repeats), median
20,004.8 ms (min 20,002.7, max 20,012.5)
2.7 ms (min 2.3, max 3.6)
No concurrent refresh (21 repeats), median
2.10 ms
2.33 ms (an earlier branch run gave 2.01 ms)
Reads with no concurrent refresh are unchanged within noise. I did not measure how often this convoy happens in real use. It needs a manual Refresh while the pricing endpoint is slow or unreachable, plus another read at the same time.
Tests: vp test run src/usage/UsageService.test.ts -t "valid rates while a forced refresh|missing or has reached|pricing metadata of the rates|in-flight scan": 5/5 pass. All use TestClock and Deferred-gated fake HTTP, with no sleeps.
serves an ordinary read from valid rates while a forced refresh is in flight: on main it hangs until the timeout. On this branch the read returns old prices with old metadata, and the read after the refresh gets the new ones.
waits for rates when the cache is missing or has reached its 24-hour expiry: passes on both. A missing cache waits for HTTP, 24 h − 1 ms uses the cache, and exactly 24 h fetches.
reports the pricing metadata of the rates a summary was priced with: fails on main with mismatched metadata. It also checks that a read after a refresh doesn't join the older in-flight scan; without the scanKey change that read hangs.
vp run check:changed: format, lint, knip and typecheck pass. Two existing tests in UsageService.test.ts ("discovers Pi history in its default home…" and "retains Codex reconciliation…") fail on macOS with the /private/var realpath issue. They fail the same way on unmodified main.
I did not check this in a real client, against the real LiteLLM endpoint, or in a real offline outage.
An independent review (GPT-6.1 Terra via Codex) found the follow-up-read join issue, and it is fixed above. It found no mixed table/metadata publication, no TTL boundary drift, and no interruption or lock leak.
Made with Claude Opus 5.5 via Claude Code in T3 Code.
An ordinary usage read joined the pricing lock even while its cached
rates were valid, so it waited behind a manual Refresh's HTTP attempts.
Valid rates now skip the lock, each summary reports the pricing metadata
of the table its buckets were priced with, and the in-flight scan key
includes the table's fetch time so a manual Refresh's follow-up read
never joins a scan priced before the new table was published.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Macroscope's review found this PR approvable — Usage summaries can use still-valid cached pricing without waiting for a forced refresh, while keeping pricing metadata aligned with the summary. Existing freshness limits and refresh safeguards remain unchanged.
You can add or adjust custom eligibility rules. Learn more.
No actionable comments were generated in the recent review.
Walkthrough
Usage rate reads now use valid cached rates without waiting for a pending refresh. Each scan summary captures the pricing metadata for the rates used during that scan.
Changes: Usage rate reads and summaries
Layer / File(s)
Summary
Rate cache and refresh behavior apps/server/src/usage/UsageService.ts, apps/server/src/usage/UsageService.test.ts
Non-forced reads skip the refresh semaphore when cached rates are younger than 24 hours. Tests cover reads during a forced refresh, missing rates, and the 24-hour expiry boundary.
Pricing metadata per scan apps/server/src/usage/UsageService.ts, apps/server/src/usage/UsageService.test.ts
Each summary returns the pricing snapshot captured before aggregation. Scan keys include the rates publication timestamp. A test checks summary pricing when a scan overlaps a refresh.
No actionable merge-blocking issue is established; merge after normal checks.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: ordinary usage summaries no longer wait on a forced pricing refresh. The changeset matches this. ensureRates now checks the 24-hour TTL before taking the lock, and the…
Description check
The description follows the template sections: Problem, Change, Scope and approval, and Verification. The Problem section gives the symptom, the expected behavior, and the cause. The Change section ex…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the main change: ordinary usage summaries no longer wait on a forced pricing refresh. The changeset matches this. ensureRates now checks the 24-hour TTL before taking the lock, and the title's latency figures match the PR objectives. The title also uses the conventional commit format from the repository template.
Full details: Description check
Explanation
The description follows the template sections: Problem, Change, Scope and approval, and Verification. The Problem section gives the symptom, the expected behavior, and the cause. The Change section explains the three edits and why each is needed. The Scope section links the change to CONTRIBUTING.md and argues the small bug-fix exception. The Verification section gives benchmark data, targeted test results, the check:changed result, and known gaps. It does not link a triaged issue or maintainer approval, so the Scope section relies on the bug-fix exception argument. The description also includes the model and harness, as the template asks.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
CodeRabbit Verify
Run CodeRabbit Verify to test the changes in this pull request.
Reviewing files that changed from the base of the PR and between cefb544 and 705f049.
Files selected for processing: 2
apps/server/src/usage/UsageService.test.ts
apps/server/src/usage/UsageService.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.
verified as-is at 705f04927ecf532ef490628aed47d8fad9ae0b18, after reproducing on github/main (cefb544af3). no source changes.
ran one isolated dev server at a time on linux, using repository Playwright and /usr/bin/google-chrome. added the real repository through the app and completed real Codex gpt-6.1-sol turns through the local proxy on main and the pr. no mocked provider, pricing responses, or transcript fixtures.
warmed pricing from the real LiteLLM endpoint, then temporarily dropped pricing traffic at the network layer. after the refresh floor, client a clicked Refresh usage; client b changed from 30 days to 7 days. these are browser-observed RPC timings, one reproduction per revision, not a benchmark median:
main pr
server.refreshUsageRates 20,019 ms 20,014 ms
client b getUsageSummary 19,168 ms 51 ms
the pr rendered the new date range while the manual refresh remained pending. main kept the previous summary dimmed until the refresh finished. both returned the cached pricing timestamp, 2026-11-03T10:04:05.168Z.
the scenarios use the same real transcript directory; totals grew between revisions because verification itself used real Codex turns.
restored networking and checked a successful real refresh: 150 ms, followed by a 24 ms summary. both reported fresh pricing at 2026-11-03T10:14:49.243Z; the matching before/after summary buckets were identical. token/cost switching, 24-hour and daily date windows, and model details also worked. immediate retries respected the refresh floor.
vp run check:changed cefb544af3c98224874419198c0359b652ca3c94 passed through blacksmith-check: format, lint, knip, graphql, affected typechecks and 55 tests. the three focused pricing tests also passed separately.
Not verified
native desktop/mobile clients, relay/tunnel connections and other providers were not exercised. the exact mid-scan publication race and missing/24-hour-expired cache behavior passed deterministic tests but were not forced in the live app. Codex account limits could not be verified: the proxy returned JSON-RPC -32600 on both main and the pr; transcript-based usage and real turns worked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
size:S10-29 changed lines (additions + deletions).vouch:unvouchedPR author is not yet trusted in the VOUCHED list.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
While one client runs a manual usage Refresh, every other ordinary usage summary read on that environment waits for the refresh to finish, even though its cached model rates are still valid. Against a slow or unreachable pricing endpoint, the refresh makes two sequential 10-second attempts (raw GitHub, then the GitHub API fallback), so those reads take about 20 s.
Change
In
apps/server/src/usage/UsageService.ts,ensureRatesacquiredratesLockbeforeloadRateschecked the 24-hour TTL. A non-forced read with a valid table therefore queued behind a forced refresh holding the lock for its HTTP attempts.<TTL boundary), missing caches and the 60 s refresh floor still go through the lock as before.scanSummarycaptures thepricingmetadata at the same moment it handsratesto theUsageAggregator. Before, it readpricing()at the end of the scan, so a refresh that landed mid-scan (for example during Codex service-tier resolution) labelled old-priced buckets with the new table'sfetchedAt. That race already exists onmain; the fast path would have made it easier to hit.scanKeynow includes the table's fetch time. The client's Refresh callsrefreshUsageRatesand then re-reads the summary. Without this, that re-read could join an in-flight scan priced with the old table. That case was possible onmaintoo and is more likely with the fast path.There are no contract, client or provider changes. Web, desktop and mobile share the same RPCs and gain the same benefit. I did not build stale-while-refresh pricing for cold or expired caches. That would need a pending-cost state in the clients.
Scope and approval
This is a focused performance and reliability fix, which CONTRIBUTING.md lists as most likely to be accepted, and it fits the small, focused bug-fix exception. Ordinary reads waited on an unrelated lock for no reason. The change is three small edits in one service. It adds no feature, changes no product defaults or contracts, and keeps manual Refresh semantics.
Verification
Benchmark: a temporary
it.livecase (not committed) uses the realUsageService, a 200-record Claude transcript, a seeded valid rates cache, and a localnode:httpstub that holds every request for 15 s, longer than the service's 10 s per-attempt timeout. Each repeat uses a fresh service. It startsrefreshRates, waits until the stub receives the request, then times one ordinaryreadSummary.mainReads with no concurrent refresh are unchanged within noise. I did not measure how often this convoy happens in real use. It needs a manual Refresh while the pricing endpoint is slow or unreachable, plus another read at the same time.
Tests:
vp test run src/usage/UsageService.test.ts -t "valid rates while a forced refresh|missing or has reached|pricing metadata of the rates|in-flight scan": 5/5 pass. All use TestClock and Deferred-gated fake HTTP, with no sleeps.serves an ordinary read from valid rates while a forced refresh is in flight: onmainit hangs until the timeout. On this branch the read returns old prices with old metadata, and the read after the refresh gets the new ones.waits for rates when the cache is missing or has reached its 24-hour expiry: passes on both. A missing cache waits for HTTP, 24 h − 1 ms uses the cache, and exactly 24 h fetches.reports the pricing metadata of the rates a summary was priced with: fails onmainwith mismatched metadata. It also checks that a read after a refresh doesn't join the older in-flight scan; without thescanKeychange that read hangs.vp run check:changed: format, lint, knip and typecheck pass. Two existing tests inUsageService.test.ts("discovers Pi history in its default home…" and "retains Codex reconciliation…") fail on macOS with the/private/varrealpath issue. They fail the same way on unmodifiedmain.I did not check this in a real client, against the real LiteLLM endpoint, or in a real offline outage.
An independent review (GPT-6.1 Terra via Codex) found the follow-up-read join issue, and it is fixed above. It found no mixed table/metadata publication, no TTL boundary drift, and no interruption or lock leak.
Made with Claude Opus 5.5 via Claude Code in T3 Code.
🤖 Generated with Claude Code