[TheSentinel454]

Summary

Newest-first events page reads are now split at the start of the previous UTC month, so Postgres can prune the older partitions whenever the newest ones fill the page. Results are unchanged.

Why. events is range-partitioned by month, and every partition carries the full index set. A page read with no created_at lower bound can't prune, so Postgres plans and locks every partition and every partition index: about 200 relations per read with 19 partitions. A backend has 16 fast-path lock slots, and every lock past those goes through the shared lock table. On a production-shaped load rig at pool 200, active backends spent 36% of their time waiting on LWLock:LockManager and 1.6% on disk.

What changes. query_events_on (REQ history, bridge /query, mentions, rosters, notices) and get_channel_window_on (channel windows) now read in up to two passes on the same connection, through one helper (store/recent_pass.rs):

1. Recent pass: the same read plus created_at >= <floor>, where the floor is the start of the previous UTC month. When it returns a full page, that page is the answer, because every row at or above the floor sorts before every row below it.
2. Older pass, only when the recent pass comes up short: created_at < <floor>, limited to the rows still missing, appended after the recent rows. The ranges don't overlap, so no row is read twice and the page is exact. For channel windows the limit includes the +1 probe row, so has_more stays exact.

One snapshot, one deadline. The two passes run in one REPEATABLE READ, READ ONLY transaction (ReadTransaction), so both read the same snapshot. Under READ COMMITTED, a replacement committing between the passes (a replaceable event, an artifact head) could move the live row from below the floor to above it, and it would land in neither pass. Inside a transaction the passes run in a savepoint instead; the only callers that read inside a transaction are routed replica reads, which are already REPEATABLE READ. Each pass after the first runs under whatever is left of the 20 s read deadline, so a split read never gets more time than an unsplit one. Single-statement reads (COUNT, unsplit pages) keep their old path, now through the same type. Channel windows now take the read deadline too. On a routed reader that is the reader's session deadline. On the writer they used to run under the writer's own session timeout (2 minutes), and now get the 20 s read deadline, like every other event read there.

The floor is a SQL literal, not a bind parameter, so Postgres prunes when it plans the statement, cached generic plans included. It changes once a month, so the statement cache holds one statement per pass and shape. #p reads put the same bound on event_mentions.event_created_at.

These reads are not split:

• Reads whose own bounds already prune (since at or after the floor) or fall entirely before the floor (until before it).
• OFFSET reads, since the offset spans both sides of the floor.
• Lookups that match a fixed set of rows wherever they sit: ids, #e, #d, artifact d tags.
• Fetch-everything reads (max_limit, the COUNT fallback).
• Reads limited to replaceable or addressable kinds. Those are latest-state rows, often old and few, so a recent pass would rarely fill the page.

No protocol, schema, index or client change.

Not covered: COUNT and the per-row aux #e reads, neither of which has a page to fill. The recent pass also has no upper bound, because rows dated in the future must stay visible. So partitions ahead of the current month (the manager keeps 6) and the catch-all are still planned.

Evidence

A production-shaped load rig (6,600 desktop clients connecting cold at 100/s, against a 19-partition events table), comparing main with this branch. Each row is one arm, so expect run-to-run noise of about ±10%.

Arm	Postgres exec time (top statements)	Errors	Relation locks outside fast path (avg)
Pool 200 + channel-first index, main	13,217 s	4.75%	9,481
Pool 200 + channel-first index, this PR	10,931 s	3.68%	4,988
Default pool, no extra index, main	6,705 s	5.96%	—
Default pool, no extra index, this PR	6,145 s	5.15%	—
Default pool + index, main (repeat)	1,070 s	0.99%	—
Default pool + index, this PR (repeat)	1,013 s	0.93%	—

Pages that fill above the floor stop planning old partitions, and that halves the locks taken outside the fast path. LWLock:LockManager stays at about 41% of active backend samples at pool 200, though. The dominant remaining cost there is the cold unread repair: a multi-channel read with no since and a page that never fills. That read needs its own fix (a lookback bound or a better plan) and is out of scope here.

An earlier version re-ran the whole read after a short recent pass. That doubled the work on non-filling reads: errors went from 4.75% to 11.88% at pool 200. This version reads only the missing older rows.

Tests

• recent_pass::tests: floor arithmetic (including the year wrap), the predicate literals, an eligibility table, and an in-memory exactness test that drives the passes at every limit and pins which passes run with which limits.
• e_tag_filter_shape_tests::split_passes_bound_created_at_with_a_literal / recent_pass_skips_lookups: SQL shape (both the e. and m. sides for #p, no bound on unsplit reads or COUNT) and the exclusions, including offset.
• postgres_tests::split_page_reads_match_the_unsplit_read (buzz-db event): rows on both sides of the floor. A full recent page is answered alone; short pages at limits 4, 5 and 6 add exactly the missing older rows; offset reads across the floor are not split; a cursor below the floor works. It reads pg_prepared_statements to show which passes ran. With another session locking a partition only the older pass plans (and a short lock_timeout), a full recent page still answers and a short one returns Err rather than a partial page.
• postgres_tests::channel_window_split_matches_the_unsplit_read (buzz-db thread): the same checks for channel windows, including has_more when the recent pass has no probe row, and an Err when the older pass fails.
• split_read_passes_share_one_snapshot (event, both deadline modes) and channel_window_passes_share_one_snapshot (thread): another session holds a lock that only the older pass needs, so the read stops between its passes. A replacement then commits that deletes a row below the floor and makes its replacement live above it. The page must still be the first pass's snapshot: the replaced row is present and its replacement is absent.
• read_statements_share_one_deadline: per-read, session and inside-a-transaction reads. The second statement runs under less than the full deadline, and finishing the read restores the connection's own timeout.
• channel_window_passes_share_one_deadline (thread): through the production writer entry. The recent pass waits 8 s on a lock and the older pass is blocked for good. The window must be cancelled within the one shared deadline, not a full deadline after its older pass starts.
• channel_scoped_mention_reads_share_one_prepared_statement now expects one recent and one older statement, shared by every channel count.

Mutation-checked. Each of these mutants fails a new Postgres test: dropping the older pass, giving the older pass the full limit, removing the split, splitting offset reads, dropping the window's probe row from the split limit, removing the window's split, swallowing an older-pass error in either path, opening the split read READ COMMITTED (events and windows), opening it with a plain BEGIN, giving every pass the full deadline, and running a window with no read deadline.

What got simpler

Nothing got simpler overall. This adds one small module (store/recent_pass.rs) shared by both read paths. The single-statement deadline helper is now a thin wrapper over ReadTransaction, so the begin, savepoint and tighten logic lives in one place. The alternative, a since in every client plus a new index per read shape, means more code in more places and does nothing for app versions already shipped.

🤖 Generated with Claude Code

[github-actions]

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

• Exact PR diff: 6793cbd25b7f1fa0c3e4750423bbbf4039e2a279...8f62e008850703ee169e9f2206184c3165e8cbc3
• Model: gpt-6.1-sol

💡 Click "edited" above to see earlier reviews for this PR.

---

Review Summary

Overall Risk: NONE

No concrete security, correctness, or reliability regressions found in the authorized PR range.

Findings

No concrete security, correctness, or reliability findings were identified.

Notes

• Review used read-only inspection of the diff, source, callers, migrations, and tests. Builds and tests were not executed.

---

_{Generated by Codex Security Review |
Requested by: @TheSentinel454 |
Workflow run}

[wpfleger96]

🤖 Automated multi-lane review at head 5bf0de03. Verdict: changes requested. The floor arithmetic, ordering, tie handling, has_more probe, literal safety, eligibility exclusions, and older-pass error propagation all check out. Two independent code-review lanes converged on one blocker: the two passes don't read the same snapshot.

1. IMPORTANT (correctness): the recent and older passes read different snapshots, so an atomic replacement between them drops a live row

query_events_on runs each pass through its own fetch_with_read_deadline call (crates/buzz-db/src/store/event.rs:578-582). Each call opens and rolls back its own transaction (event.rs:974-1004), or on a ReadDeadline::Session autocommit connection runs as its own statement. Under READ COMMITTED, each pass therefore sees its own snapshot. get_channel_window_on has the same structure, with two plain fetch_all calls (crates/buzz-db/src/store/thread.rs:669-687). The base branch runs one SELECT, which can't see a half-applied replacement.

Concrete failure: a {kinds: [45010], limit: 100} read (artifacts are not NIP-01 replaceable, so floor_for splits it) finds no current artifact head at or above the floor. Before the older pass runs, an edit commits. It replaces the artifact_heads row for an old revision with a new revision dated now (crates/buzz-db/src/store/artifact.rs:196-199, one transaction). The older pass now drops the old revision through the generic-read artifact_heads predicate (event.rs:694-698), and the new revision falls outside the older pass's range. The page comes back short or empty, and looks exhausted, even though a matching live head existed the whole time. Mixed-kind reads that include replaceable or addressable kinds have the same gap: the old version sits below the floor, and its replacement lands above it between the passes. The "no row is read twice and the page is exact" guarantee in recent_pass.rs:17-19 holds only within a single snapshot.

Corrective action: when the read isn't already inside a transaction with a fixed snapshot, run both passes in one REPEATABLE READ, READ ONLY transaction (with the read deadline applied once to that transaction). Otherwise, don't split. Apply the same fix to the channel-window path, and add a Postgres test that commits a head replacement between the two passes.

2. MINOR: a page that doesn't fill above the floor gets twice the read deadline

Each pass is bounded by its own READ_STATEMENT_TIMEOUT, so a split read can take up to twice the deadline the unsplit read had. Running both passes in a single transaction under one deadline, as in finding 1, fixes this too.

[TheSentinel454]

🤖 Thanks, both findings were right. Fixed in 31c3999.

1. Separate snapshots. A split read now runs both passes in one ReadTransaction, a REPEATABLE READ, READ ONLY transaction, so they read one snapshot. If the caller is already inside a transaction, the passes run in a savepoint instead. Today the only callers that read inside a transaction are routed replica reads, which are already REPEATABLE READ. get_channel_window_on uses the same transaction. Single-statement reads (COUNT, unsplit pages) keep their old path, but now go through the same type.

New Postgres tests: split_read_passes_share_one_snapshot (both deadline modes) and channel_window_passes_share_one_snapshot. In each, another session holds a lock that only the older pass needs, so the read stops between its passes. A replacement then commits in one transaction: it deletes a row below the floor and makes its replacement live above it. The page must still match the first pass's snapshot, with the replaced row present and the replacement absent.

2. Double deadline. Each statement after the first now runs under whatever is left of READ_STATEMENT_TIMEOUT, measured from when the read began. read_statements_share_one_deadline covers per-read sessions, sessions that already carry the deadline, and reads inside a transaction. In each case it checks that the second statement gets less than the full deadline and that finishing the read restores the connection's own timeout.

Mutants in the gate: opening the split read READ COMMITTED (events and windows), opening it with a plain BEGIN, and giving every pass the full deadline. Each one should fail the new tests.

Rereview

[wpfleger96]

🤖 Approved at 8f62e008. Two independent source reviews found no issues. Pages stay exact: the two reads cover ranges that don't overlap, split on the leading sort key, and the older read fetches only the missing rows. The channel window keeps its extra row for has_more. Both reads share one REPEATABLE READ, READ ONLY snapshot and one 20 s read deadline, which fixes both findings from the earlier review. If the older read fails, the read returns an error rather than a partial page. The month floor goes into the SQL as a fixed-format UTC timestamp, with no caller input in it. Offset reads, id/tag lookups, max_limit reads and reads limited to replaceable kinds still run as one query. Channel windows on the primary database now time out after 20 s instead of 2 minutes. That matches every other event read.