镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

buzz-relay has zero HTTP request/body timeouts — unauthenticated clients can pin connections indefinitely by withholding request bodies #4424

Open

Description

@tlongwell-block

Summary

buzz-relay has no HTTP request or body-read timeouts anywhere: nothing in crates/buzz-relay/src/router.rs or main.rs mounts a TimeoutLayer or read deadline, and axum::serve configures none (verified at 42e96a118 and on then-current main 318fbf896; scope: rg over those files). RequestBodyLimitLayer bounds byte count, not elapsed time. Any client can open a connection, send headers, and drip or withhold the body indefinitely, pinning a connection/task per socket.

Pre-auth reachable today, on multiple routes

Found by Wren while reviewing #4401 (whose probe middleware awaits 4 unauthenticated body bytes with no deadline). Survey during that review showed the seam is platform-wide, not new to #4401 — main already has larger unauthenticated deadline-free body reads:

Measured, not theoretical

Sami built the #4401 middleware standalone and opened 200 sockets sending the candidate headers while withholding the body:

variant=A (PR as written)   inflight_report: 200   answered_without_body=0

200 parked tasks, one per socket, indefinitely. A header-only predicate (variant B) removed the parking but was rejected as the wrong trade: the exact-body match is the best property of the #4401 shim, and dropping it to dodge a 4-byte park while accept_policy buffers 1 MiB deadline-free would be false comfort.

Fix shape

A relay-wide timeout layer, not per-route patches:

  1. tower_http::timeout::TimeoutLayer (or equivalent) on the router with a generous default (e.g. 30–60s), with the existing per-route git pack-ops timeouts (PACK_OPS_TIMEOUT 300s in transport.rs) taking precedence on long-lived routes.
  2. Consider a hyper-level header-read timeout so pre-routing slowloris (headers never completing) is also bounded.
  3. Regression test with a never-completing body against an unauthenticated route.

Websocket upgrade routes must be exempted from any body/request deadline.

Credit

Mechanism identified by Wren; platform-wide survey by Eva; load measurement by Sami — all during the #4401 review (buzz-outside-pr-reviews, 2026-08-02). Related: #2723 (startup conformance probe has no deadline), #3651 (reader statements unbounded) — same missing-deadline family, different layers.

Refs: #4401.

Activity

  1. tlongwell-block commented on Aug 3, 2026

    @tlongwell-block
    CollaboratorAuthor

    Minimal core implemented as commit f6e793f9f on PR #4401's branch (pushed, PR head now f6e793f9f). Scope follows Wren's review correction: this is not literally relay-wide — it covers the core unauthenticated request-body surfaces plus the Git probe.

    What landed:

    • router.rs: with_request_deadline() helper applying tower_http TimeoutLayer::with_status_code(408, …) outside RequestBodyLimitLayer (layer order regression-tested) on:
      • api_router — 60s (API_REQUEST_TIMEOUT)
      • media_router — 300s (MEDIA_REQUEST_TIMEOUT); docs note this bounds the future until response headers incl. handler-side body collection, not a streaming response body after headers.
    • transport.rs: PROBE_BODY_TIMEOUT (60s) tokio::time::timeout around the probe's pre-auth 4-byte to_bytes collection; fails closed into the normal auth path on expiry/stream error.
    • WS routes (/, huddle audio): handshake-bounded only; established sessions unaffected (tested, documented per that wording).
    • Deliberately NOT wrapped (documented in the helper doc comment): git router internals beyond the probe (no new outer deadline around authenticated Git/CAS execution — a 360s ceiling could cancel a valid push mid-publication, per review), admin router, git policy router, SPA fallback, health listener, and hyper-level header-read deadlines. Those remain open on this issue.

    Tests (millisecond injectable timeouts, no 60s sleeps): stalled-body → 408 + handler cancellation; dropped-future doesn't invoke handler; WS-session-survives-timeout; layer-order regression (stalled candidate caught by timeout, not parked in body-limit); probe stall fail-closed + dropped-stall. All green; fmt+clippy clean.

    Verification:

    • Full cargo test -p buzz-relay: 844 passed / 1 pre-existing failure (mesh_demo 504, test(relay): mesh_demo echo test flaky under parallel suite — passes standalone, fails full-suite on clean main #2458 — fails identically at 42e96a118 and main 318bf896 in this env, so not caused by wrapping /_mesh/demo/echo in the timed api_router).
    • Live (release relay at f6e793f9f, isolated compose stack): raw-socket POST /events with Content-Length: 100 and body withheld → HTTP/1.1 408 Request Timeout, empty body, at exactly 60.0s. Before this commit that connection parked indefinitely (Sami's variant-A demo).
    • Git paths unaffected live: large-push e2e passed; forced unauthenticated upload-pack probe → 200 and full clone roundtrip.

    Still open on this issue: header-read (slowloris stage-2) deadlines via the axum::serve → hyper_util swap; admin/policy/fallback coverage; a safe whole-request Git budget (needs a cancellation audit of the publication phase first).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions