镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

fix(relay): route live REQ events by each filter's own channel scope - #8519

Merged
loganj merged 4 commits into
mainfrom
larry/per-filter-live-scope
Nov 6, 2026
Merged

loganj merged 4 commits into
mainfrom
larry/per-filter-live-scope

Conversation

@loganj

@loganj loganj commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

🤖

Summary

The problem. A Buzz client that watches one channel's details (its name, topic, members) gets them once and then never hears about changes. If someone renames the channel, the client keeps showing the old name until it reloads. A related problem: when a client watches a channel's messages and something community-wide in the same request, the channel's new messages stop arriving.

The fix. The relay (the Buzz server) now decides where to send new events separately for each part of a request, instead of once for the whole request.

Background: how a client watches for events

A client sends the relay a subscription: a request made of one or more filters. Each filter describes some events, for example "messages in channel X" or "the details of channel X". The relay answers in two phases:

  1. History. It sends the stored events that match, then a marker that says "history is done".
  2. Live. It keeps the subscription open and pushes each new matching event as it arrives.

For the live phase, the relay must decide which new events a subscription may receive. It sorts new events into topics: one topic per channel, plus one community-wide topic for events that belong to no channel. A subscription only hears the topics it signed up for. This is the security boundary that keeps a private channel's events away from people who cannot read it.

Why it broke

The relay picked the topics once for the whole subscription, with one rule: if every filter names its channels with an #h tag, listen to those channels; otherwise, listen only to the community-wide topic.

That rule fails in two cases:

  • Channel details. A channel's details are a special event (kind 39000) that the relay signs and stores under the channel itself. Clients ask for it with a #d tag, not #h: {kinds:[39000], "#d":[X]}. Because the filter has no #h, the rule chose the community-wide topic, which never carries channel X's events. The client got the current details in history, and then no updates. (NOSTR.md documented this gap as a future enhancement.)
  • Mixed requests. If a request had one filter for channel X's messages and a second filter with no channel, the rule chose community-wide for both. The channel filter silently lost its live messages.

What changes

Each filter now gets its own topics:

Filter Live events it receives
Names channels with #h New events in the channels it names and may read
Channel details (kinds 39000–39003) that name channels with #d Updates for the channels it names and may read, plus community-wide events
Names no channel Community-wide events, the same as before

The relay then sends a new event only through a filter that both matches the event and listens to that event's channel.

Why this is safe

  • No filter can see more than it already could. A filter's channels come only from the channels it names, and each one must pass the same read-permission check the history phase uses. A private channel's outsider gets neither history nor updates (tested).
  • One filter cannot widen another. This is the reason for per-filter topics. The simple fix, "add the #d channels to the whole subscription", would leak. Some channels, such as system channels and future channel types, appear only to a filter that names them. With one shared set of topics, a sibling filter such as "all channel details created by person P" (#P) would start to receive live updates for such a channel, even though that channel never appears in its history. fix(relay): show only legacy channel types to reads that name no channel #8469 closed this leak for the history phase. This PR keeps the live phase consistent with it (tested).
  • Requests that worked before behave the same. A request whose filters all name #h channels got the union of their channels before. Each filter's own #h condition already rejected the other filters' channels, so per-filter topics deliver exactly the same events. A filter with no channel keeps the community-wide topic, as before.
  • Recent joins are not missed. Each server caches who may read which channel for a few seconds, so a server can briefly miss a new member. When a request names channels the cache does not list, the relay checks them against the database. Before, it did this only when every filter used #h. It now checks every channel any filter names, including #d. This matters more now, because a live scope is fixed when the subscription starts: a missed channel would stay missing until the client subscribes again. The check keeps the same token restrictions (tested). More than 2048 #h channels is still refused. More than 2048 named channels in total because of #d is not refused: the #d names then skip this check and live routing, so the request gets history from the cached access set and community-wide live updates, as before this PR. The legacy desktop app sends one unbatched #d query for every channel the user is in (tested).
  • Clean up stays exact. A subscription signs up for each topic once, even when several filters name the same channel. It releases each topic once on close, on replacement with the same ID, or when access to a channel is revoked. Revoking one channel removes only that channel; the subscription keeps its other channels and its community-wide topic. The tests check the counts on each path.

Not in scope

A filter that reaches a channel without naming it still gets that channel's events in history but not live. This is unchanged: before this PR, such filters listened only to the community-wide topic too. Examples are a channel-less filter, which history fills with every channel the reader can see, and the "all system channels" opt-in (#t:["system"] or #P).

Related issue

Follow-up to #8469.

Testing

  • Two new Postgres tests drive the real request handler. (1) A channel-details subscription receives updates after history, from this server and through the handler that receives other servers' events from Redis. (The test calls that handler directly. It does not cover publishing or Redis transport.) A private channel's outsider receives nothing. (2) In a mixed request, the channel filter and the channel-less filter each keep their live events, and a sibling filter by creator (#P) or system label (#t) does not receive a channel it did not name.
  • A Postgres test with a stale access cache: a just-joined channel still reaches both a #d filter and an #h filter beside a channel-less one, in history and live.
  • New unit tests cover the per-filter topic rule and the subscription registry.

@loganj
loganj force-pushed the larry/per-filter-live-scope branch from a57e647 to c25057e Compare October 21, 2026 15:01
@loganj
loganj marked this pull request as ready for review October 21, 2026 15:38
@loganj
loganj requested a review from a team as a code owner October 21, 2026 15:38

@jedwards27 jedwards27 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:bot: Jude’s code review agent

Verdict: APPROVE
Reviewed: cd8a9681793fef83001eb44954734b47c1938550..c25057e975d60fd15bb285483ac4028252862f92 (exact head c25057e975d60fd15bb285483ac4028252862f92)
Risk: high — this changes live relay subscription routing, authorization-sensitive channel scope, Redis/local fanout, and replacement/teardown lifecycle.

Behavior/contracts traced: REQ admission; bounded per-filter explicit-channel extraction; cache-negative database repair; channel/system/private read authorization; per-filter topic retention and matching; mixed named/global sibling isolation; local and Redis fanout; replacement, revocation, timeout, close, reconnect, and reference-count cleanup. Independent systems/integration and product/adversarial passes found the implementation preserves each filter’s own channel scope without widening siblings or bypassing send-time access checks.

Findings: no unresolved blocking or non-blocking code/product defect found.
Author action: none.
Verification owner: reviewer/tooling owns any later manual native/Desktop observation; exact-head PostgreSQL, Relay E2E, Backend Integration, Desktop E2E Integration, unit/lint/security, and platform CI own the automated boundary evidence.

Validation at matching clean HEAD:

  • cargo test -p buzz-relay --lib subscription::tests — 38 passed.
  • cargo test -p buzz-relay --lib handlers::system_channels::tests — 6 passed.
  • Focused REQ lifecycle/query regression invocation — 11 passed.
  • cargo test -p buzz-relay --lib — 1,516 passed; six unrelated media tests failed while seeding their host-bound fixture with Sqlx(PoolTimedOut) in the reviewer environment. This is an infrastructure confidence gap, not a PR-caused failure.
  • Exact-head CI passed PostgreSQL Tests, Relay E2E, Backend Integration, Desktop E2E Integration, Rust lint/unit, Security, and cross-platform gates. PostgreSQL CI explicitly passed named_channels_missing_from_a_stale_cache_still_route_live, group_state_replacements_reach_live_d_subscribers, and named_filters_keep_live_delivery_beside_unnamed_siblings.

Manual/native evidence: not performed; this relay-only source change was validated at the real automated relay/PostgreSQL boundary rather than claimed as manual UI proof.
Residual risk: local Docker/PostgreSQL/Redis reproduction and manual client observation were unavailable. Relevant exact-head integration CI is green; no author action follows from those reviewer-environment gaps. Any new head invalidates this approval.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord’s automated source review via Wes’s account, integrating independent Rocket (systems/lifecycle), Mantis (correctness/security/concurrency), and Groot (product/client flows) reviews.

No actionable findings; no changes requested.

Reviewed head c25057e975d60fd15bb285483ac4028252862f92 against base/merge-base cd8a9681793fef83001eb44954734b47c1938550. Traced per-filter channel/global routing, token restrictions and stale-cache repair, sibling-filter isolation, indexed matching, local/Redis receive access checks, and topic retention through replacement, revocation, close and disconnect. The source preserves each filter’s own scope while retaining the topic union once per subscription. Reviewed the regression-test seams, unit/PostgreSQL lane selection, relevant client metadata reads, and the public description (no attached images). No actionable disclosure found. Minimalism, clarity and source correctness each meet the repository’s 9/10 review target.

Validation limits: source analysis only; no tests, benchmarks, app launches or PR code execution performed by this review. The cross-pod test calls the Redis receive handler directly; it does not establish Redis publishing/transport behavior. Database concurrency, runtime performance, native accessibility and error/retry focus behavior were not exercised. One read-only CI snapshot for this head showed 76 successful checks, 34 skipped, and one cancelled “Run Codex Security Review”; this is not a complete-green or runtime-validation claim.

This is a non-blocking COMMENT review, not approval or merge authorization.

@loganj
loganj force-pushed the larry/per-filter-live-scope branch from c25057e to 9c30155 Compare November 4, 2026 19:45
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

  • Exact PR diff: 744f3946f6fe1ff7a649c132638d5af608900dbd...a3b33476490f9b09077624703c5b331f01de8002
  • Model: gpt-6.1-sol

💡 Click "edited" above to see earlier reviews for this PR.


Review Summary

Overall Risk: NONE

No concrete security, correctness, or reliability findings identified in the authorized PR range.

Findings

No concrete security, correctness, or reliability findings were identified.

Notes

  • Review used read-only source and history inspection. Builds and tests were not executed, as instructed.

Generated by Codex Security Review |
Requested by: @loganj |
Workflow run

@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 4, 2026
@loganj

loganj commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 Fixed the Codex Security Review MEDIUM finding (named-channel limit left the old same-ID holder live) in 7abc88e: the refusal now goes through close::refuse like the #h limit, and in_handler_req_refusal_retires_the_old_holder covers the #d case (red without the fix).

@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 4, 2026

@TheSentinel454 TheSentinel454 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approving at 7abc88e0. One optional, non-blocking note inline.

Comment thread crates/buzz-relay/src/handlers/req.rs Resolved
loganj and others added 4 commits November 6, 2026 13:28
A live REQ took one request-wide routing scope from its filters' aggregate
`#h`. If any filter lacked `#h`, the whole subscription became global, so:

- `{kinds:[39000], "#d":[ch]}` got history and EOSE but never a later
  replacement, because relay-signed group-state events are stored under
  their channel and global subscriptions never see channel events;
- in a mix of named and unnamed filters, the named filter lost live
  delivery too.

Each filter now carries its own server-resolved live scope. A filter
with `#h` routes under the channels it names and may read. A filter
without `#h` keeps the global topic, and a group-state filter also routes
under the readable channels its `#d` names. Fan-out delivers only through
a filter that both matches the event and holds its channel, so a channel
one filter names never reaches a `#P` or `#t` sibling (the confinement
from #8469). The subscription retains each topic in its filters' union
once; CLOSE, same-ID replacement and revocation release it once.

Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
A REQ re-checked stale cache-negatives only for the all-`#h` channel set,
so a group-state `#d` filter, or an `#h` filter beside a channel-less
sibling, skipped the uncached membership check. On a pod whose access
cache lagged a join, such a filter dropped the channel from its history
and from its live scope, and a live scope is fixed at registration, so
updates stayed missing until the client resubscribed.

The REQ now confirms every distinct channel any filter names (`#h`, or
`#d` on group-state kinds), bounded by the explicit-channel limit before
any membership I/O. Token restrictions and per-filter scopes are kept.

Also narrows the test wording for the Redis receive handler: it covers
the handler for other pods' events, not publishing or transport.

Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
The named-channel limit added for `#d` sent a bare CLOSED and returned
before the REQ claimed its ID. A same-ID REQ that named more than 2048
channels by group-state `#d` therefore left the previous live
subscription or artifact sync registered, with its topics retained,
under an ID the client was told is closed.

Refuse it through `close::refuse`, like the `#h` limit beside it: the
old holder is retired under the lifecycle lock, and the connection is
cancelled if CLOSED cannot be queued.

Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
A group-state REQ whose #d names pass the explicit-channel limit was
refused. The legacy desktop app sends one unbatched #d query for every
channel the user is in, so a user in more than 2048 channels lost
channel metadata. Past the limit, #d names now drop out of membership
confirmation and live routing: history is served from the cached access
set and group-state filters route community-global, as before #d routed
live. The #h limit still refuses.

Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@loganj
loganj force-pushed the larry/per-filter-live-scope branch from 7abc88e to a3b3347 Compare November 6, 2026 18:57
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 6, 2026
@TheSentinel454
TheSentinel454 self-requested a review November 6, 2026 19:32
@loganj
loganj merged commit 01d9778 into main Nov 6, 2026
90 checks passed
@loganj
loganj deleted the larry/per-filter-live-scope branch November 6, 2026 19:34
loganj added a commit that referenced this pull request Nov 6, 2026
Rebasing onto main's per-filter live scopes (#8519) lengthened the
claim_admitted_live calls; rustfmt now wraps them.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
bradseiler added a commit that referenced this pull request Nov 6, 2026
* origin/main:
  fix(relay): route live REQ events by each filter's own channel scope (#8519)
  fix(relay): deliver multi-node fan-out concurrently and never drop silently (#9195)
  perf(db): bound since-limited events reads with the month-floor literal (#9197)
  fix(mobile): match Agents sign-in to Home’s reconnect capsule (#9204)
  fix(cli): clearer desktop pairing errors, paired default relay, and auth desktop list (#9196)

Signed-off-by: Brad Seiler <seiler@squareup.com>
TheSentinel454 added a commit that referenced this pull request Nov 6, 2026
…#9214)

## Summary

`buzz-relay` lib tests fail to compile on `main`: `error[E0063]: missing
field received_at in initializer of buzz_pubsub::ChannelEvent` at
`crates/buzz-relay/src/handlers/system_channel_postgres_tests.rs:1549`.

This is a semantic merge conflict. #9195 added
`ChannelEvent::received_at`, and #8519 merged a minute later with a test
helper that builds a `ChannelEvent` without it. Neither PR's CI saw the
other. This sets the field to `tokio::time::Instant::now()`, as every
other initializer does.

It breaks clippy (`Rust Lint`) and the backend-integration test archive
for every open PR, because CI builds the merge with `main`.

### Testing

- `cargo fmt --check` and `cargo clippy -p buzz-relay --all-targets -D
warnings` on a cloud workstation at this head.

## What got simpler

Nothing; a one-field compile fix.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
loganj added a commit that referenced this pull request Nov 6, 2026
Rebasing onto main's per-filter live scopes (#8519) lengthened the
claim_admitted_live calls; rustfmt now wraps them.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
loganj added a commit that referenced this pull request Nov 9, 2026
Rebasing onto main's per-filter live scopes (#8519) lengthened the
claim_admitted_live calls; rustfmt now wraps them.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
loganj added a commit that referenced this pull request Nov 9, 2026
Rebasing onto main's per-filter live scopes (#8519) lengthened the
claim_admitted_live calls; rustfmt now wraps them.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>

This branch was previously deployed

1 inactive deployment
codex-review — a3b33476 Deployed Nov 6, 2026 by loganj via Run Codex Security Review #13572
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-security-review-current The posted Codex security review matches its recorded range.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants