镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

fix(relay): retry live states that can recover - #1633

Merged
loganj merged 9 commits into
mainfrom
fix/live-terminal-retry
Oct 28, 2026
Merged

loganj merged 9 commits into
mainfrom
fix/live-terminal-retry

Conversation

@loganj

@loganj loganj commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

🤖

Summary

  • Some failures of the app's live relay connection stopped it until the app was restarted. Nothing on screen showed this. An agent running in the app stopped hearing mentions, with no sign anything was wrong.
  • These failures now retry on their own, slowly, while the error stays visible:
    • A relay that rejected sign-in, or a sign-in signature that failed (for example, while the signer was locked). Before: final. Now: reconnects after 60 s, doubling to 10 minutes.
    • A channel subscription the relay closed with a reason that needs action (for example blocked: or auth-required:). Before: final. Now: retried on the existing one-minute recovery timer with the same backoff. The "not a channel member" denial flow is unchanged.
    • One event the app could not decode. Before: it failed its whole channel batch (up to 10 channels) for good. Now: only that event is dropped and logged.
    • The dev broker's event stream (browser build) used to stop for good on HTTP 400/401/403/413. It now retries on the same slow timer. Its reconnect backoff also never reset, so after a few drops every reconnect waited 30 s. It now resets once a stream has stayed up for a minute. The first frame does not count, because every new stream sends one at once.
  • While a slow retry runs, the "Live updates limited" notice keeps its reason and its Retry live updates button, so it does not vanish or lose the button on each attempt (Design feedback). Pressing Retry also leaves the notice in place. When the attempt succeeds and the notice goes away with keyboard focus on its button, focus moves on to the composer, as it already did for the session recovery notice.
  • If the recovery callback throws, the one-minute recovery timer used to stop for good. It now logs the error and keeps running.

Details

  • One backoff, liveSlowRetryDelay (src/features/relay/live.ts), serves the auth, route and broker cases. It starts at 60 s, the existing recovery tick, so a refusal is never retried faster than today's timer. It is capped at 10 minutes, so a relay that keeps refusing sees at most 6 attempts an hour per route.
  • Session recovery still treats these states as settled, so it does not retry them every few seconds. Only the slow timer retries them. A signer that does not match the signed-in account stays terminal: retrying cannot fix it.
  • docs/relay-queries.md is updated where it described these states as terminal.

Not in this PR

🤖 Generated with Claude Code

wesbillman

This comment was marked as resolved.

@loganj

This comment has been minimized.

wesbillman

This comment was marked as resolved.

@loganj

This comment has been minimized.

@loganj
loganj force-pushed the fix/live-terminal-retry branch from faef9f0 to a6e2461 Compare October 26, 2026 19:41
loganj and others added 3 commits October 27, 2026 11:30
Several live-connection failures were final until the app restarted, so an
agent in the app could silently stop hearing mentions.

- Drop a single undecodable EVENT instead of failing its whole route.
- Retry a rejected live AUTH or a failed live signature on a slow timer
  (60 s doubling to 10 min). The error stays in state and session recovery
  still treats it as settled, so it cannot shorten the wait. A signer that
  does not match the viewer stays terminal.
- Retry routes CLOSED with a needs-action reason (blocked:, auth-required:,
  unknown text) on the recover tick with the same backoff. The
  "not a channel member" denial flow is unchanged.
- Keep the recover tick running when callbacks.recover throws, and log it.
- Broker SSE client: reset the reconnect backoff once a stream delivers, and
  retry 400/401/403/413 stream rejections slowly instead of stopping.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
…retries

Every accepted broker stream sends its state at once, so resetting the
reconnect backoff on the first frame let a stream that drops right after
opening retry every 500 ms forever. Reset it only after a stream stayed up
for a minute, as the WebSocket path does.

A slow automatic retry can remove the "Retry live updates" button while
it has focus. Track focus for that control as for the session recovery
notice, and hand focus on when it goes away.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
@loganj
loganj force-pushed the fix/live-terminal-retry branch from a6e2461 to 065239d Compare October 27, 2026 15:36
kalvinnchau

This comment was marked as resolved.

loganj and others added 2 commits October 27, 2026 16:57
…deadline

A refused live route gets a recover-tick deadline. Manual Retry or a
scope replacement moved it back to pending with that deadline intact, so
a later unsupported cooldown that stopped the queue was revived by the
recover tick. Route both queue stops through one helper that clears the
deadline and keeps the refusal count.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
Design review: while the live layer quietly retries a refused connection
or route, the "Live updates limited" notice lost its Retry button, or
vanished and came back, on every attempt. Hold the notice, its reason and
its Retry while an attempt runs; the hold ends once the attempt succeeds
or a new failure replaces it. Retry now leaves the notice and its focused
button in place, and focus moves on only when the notice goes away.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
kalvinnchau

This comment was marked as resolved.

Review: the hold counted only a reconnect or a pending route as an
attempt. The minute recovery tick also refreshes the roster, and a head
can refresh too; either made live state busy, so the hold ended, the
focused Retry unmounted and came back unfocused. Hold until
liveRecoveryStatus settles instead: either the failure returns or it has
cleared.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
Comment thread src/bundled/channels/LiveStatus.tsx Resolved
Review: liveRecoveryStatus covers the whole session, so another channel's
pending route or head, a transient error elsewhere or a previews check
kept this channel's old notice and Retry after it was live again, and
focus never moved on. Hold only while this channel still has an issue or
work in flight: connection setup, the roster read, its own head read or a
relevant route.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
kalvinnchau

This comment was marked as resolved.

Review: the hold kept Retry for any displayed issue, so a channel that
recovered to a partial roster or capacity notice kept Retry while
unrelated work elsewhere was unsettled. Hold only for a blocking issue
or this channel's own work in flight.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>
kalvinnchau

This comment was marked as resolved.

Review: issue and blocking repeated the same clauses, so a new actionable
state had to be added to both. issue is now blocking plus the two passive
notices (a capacity-limited route, a partial roster). No behavior change.

Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Logan Johnson <loganj@squareup.com>

@kalvinnchau kalvinnchau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approved. No actionable findings at 647c1fc. The duplicate status predicate is resolved, and Retry focus, passive notices, and channel-local recovery remain covered. Merge remains contingent on required CI passing.

@loganj
loganj merged commit 1003952 into main Oct 28, 2026
37 checks passed
@loganj
loganj deleted the fix/live-terminal-retry branch October 28, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants