镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

feat(recall): bundled plugin that quotes helpful history into agent turns - #2429

Draft
tlongwell-block wants to merge 3 commits into
mainfrom
eva/recall-plugin
Draft

tlongwell-block wants to merge 3 commits into
mainfrom
eva/recall-plugin

Conversation

@tlongwell-block

Copy link
Copy Markdown
Contributor

Prepared by Eva (Tyler's agent) with the team: Max (live desktop pass), Wren and Quinn (review), Meli (client).

What

Recall (buzz.recall) is a bundled plugin, off by default. When one of your agents starts a turn, Recall finds the few past messages and memories that would help with the request and quotes them into that turn. If anything goes wrong, the turn goes ahead exactly as it would have without it.

mention ─► turn start ─► find ─────────► choose ─────────► pack ──► appended to the agent's input
                         keywords +       your Decisions     ≤2,400 B, quoted,
                         embeddings,      provider: "does    attributed, marked
                         audience-checked this excerpt help?" "not instructions"
  • find runs the relay's full-text search as the agent (this channel by default, or every channel the agent can read if you opt in), scores the agent's memory on-device (memory is encrypted, so the relay can't search it), and can optionally use vectors from an embedding server you run. Results are merged by rank fusion.
  • choose makes one bounded call to your Decisions provider. Only the excerpts it says help survive. A failure or timeout keeps nothing, never unchecked text.
  • pack writes escaped, attributed one-line excerpts (date, author, buzz:// link), at most 2,400 bytes in total, framed as other people's words rather than instructions.

Why it is safe to leave on

  • Audience: an excerpt shows only where everyone who can read the current chat could already read its source. Memories show only in chats between the owner and the agent, unless the owner opts in.
  • Leased reads: Recall reads only as that agent, only while the turn is being prepared, with a shared 3 s deadline. Access ends on settlement, stop, unload or connection change.
  • Nothing persisted from the relay: the optional vector index lives in RAM, and every hit is revalidated against live reads before it is shown. Recall writes nothing to disk and nothing to the relay.
  • Fresh turns only: steers, wakes, titles and compaction are left alone.

Commits

  1. feat(agents): turn context. Any plugin that declares host.turnContext can register a contributor with ctx.agents.registerTurnContext. The host calls contributors only for ordinary mention starts in Claude, Codex and goose, and appends what they return to the user input (never to developer instructions). Sections are admitted whole, at most 8 KiB each and 16 KiB in total. The install review shows the grant. Documented in docs/plugin-architecture.md.
  2. feat(plugins): wildcard network origin. host.networkOrigins: ["*"] allows any HTTPS server, plus plain HTTP only to parsed loopback IPs (127.0.0.0/8, [::1]). Hostnames such as localhost never qualify for HTTP and are never resolved. The wildcard must be the only entry. URL credentials are refused and redirects stay off. Exact HTTPS origins are unchanged. The install review states it plainly: "Can send what it reads to servers it chooses, including ones on this computer."
  3. feat(recall): the plugin, plus its catalog registration, help text and README.

Semantic recall, without Buzz managing a model

Recall talks to any llama.cpp-compatible /v1/embeddings server: an Embedding server URL setting, an optional bearer key, two optional query/document prefixes, and a Test button. Dimensions are read from the server. A change of model clears the in-RAM index. An empty URL means keyword-only recall. Buzz never downloads, installs or starts a model. Servers that require a model field (OpenAI, vLLM, Ollama) are not supported yet.

Measured with EmbeddingGemma 2 Q8 on llama-server b11669 (M-series Mac), through Recall's shipped client code:

Recall@10, keyword-only (today) 58%
Recall@10, vector / hybrid 69% / 67%
Hybrid over today +10 pts, 95% CI [+7, +13]
Gemma prefixes, on their own +10 pts (so they stay as optional fields)
Query embed latency 11 ms p50, 14 ms p99 (turn budget 3,000 ms)
Indexing 90 msgs/s, 485 MB server RAM, 1.3 s cold start to first answer

The shipped client matches the independent eval harness exactly, across all 16,528 per-question checks. In a semantic-only control, a message sharing zero keywords with the question still came back first among 48 distractors, and it disappeared once the URL was cleared.

A batch the server answers with any HTTP error (llama-server reports an oversize input as 500, not 400) is skipped, and later batches still index. A broken or absent response stops the snapshot.

Testing

At head f5f211c7, in a clean tree:

  • tsc --noEmit, biome check --error-on-warnings, cargo fmt --check, cargo clippy --workspace --all-targets -D warnings: clean.
  • cargo test --workspace --locked: 1,199 passed, 0 failed. One test was skipped: window_state::tests::onboarding_snapshot_is_kept_until_next_launch_restore SIGSEGVs the native lib binary under MockRuntime on local macOS. It does the same on main, comes from Updated onboarding #2183, and this PR doesn't touch it.
  • Vitest, full suite: 16,678 passed, 6 failed, 21 skipped. All 6 failures were roughly 5 s timeouts in navigation.test.tsx, codex-sh.test.ts and ProfileAgentIdentity.test.tsx, while the run shared the machine with the cargo suite. None of those files is touched here, and all three pass in isolation (54/54).
  • Recall + turn-context + access-review focus run: 248/248. src/bundled/recall/mutate.sh: 67/67 mutants killed at this head; unmutated and restored suites both pass.

Live (agent-run), native desktop build against a fresh local TLS relay with a throwaway identity, at the pre-squash head whose tree this PR reproduces (only comments, docs and the mutation-log path differ):

  • Save and Test over native IPC succeed against real Gemma (768 dims, 45 ms).
  • The URL persists across a relaunch.
  • A dimension change at the same URL is rejected.
  • A localhost HTTP URL is rejected before any request is sent.
  • Stopping the server mid-session makes Test report the failure.
  • Clearing the URL reports "Lexical recall only. RAM index cleared."
  • Credential metadata is identical before and after.

Not yet done: rendered/pointer acceptance of the Settings card. The isolated test window stayed inactive, and Settings stayed at opacity 0. There is also no human test yet and no actual agent-answer pass. That is why this PR is a draft.

Not in this PR

  • Batch bisection for small-context embedding models. That will be a follow-up.
  • Model-selector servers (OpenAI / vLLM / Ollama).

Try it

  1. Enable Recall in Settings → Plugins → Manage plugins, and set up Settings → Decisions API.
  2. Optional, for semantic recall: llama-server --model embeddinggemma-2-Q8_0.gguf --embedding --pooling mean --host 127.0.0.1 --port 8080 --ctx-size 4096 --batch-size 4096 --ubatch-size 4096. Then, in Settings → Plugins → Recall, set the URL to http://127.0.0.1:8080, the query prefix to task: search result | query: and the document prefix to title: none | text: , and press Test. Working looks like "768 dims · N ms".
  3. Mention an agent with a question that an earlier message in the channel answers, using different words. The agent's input should end with a quoted, attributed excerpt of that message.

tlongwell-block and others added 3 commits November 4, 2026 10:02
A plugin that declares `host.turnContext` can register a contributor with
`ctx.agents.registerTurnContext`. The host calls contributors only for
ordinary mention starts in Claude, Codex and goose (never steers, wakes,
titles or compaction) and appends what they return to the user input.

Each contributor gets trusted turn facts and leased reads (messages and
audience facts as the agent, the owner's memory listing). The lease ends on
settlement, stop, unload, connection change or a shared three-second
deadline. Failures yield no section; sections are admitted whole, up to
8 KiB each and 16 KiB total. The install review shows the grant.

Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@squareup.com>
Signed-off-by: Tyler Longwell <tlongwell@squareup.com>
`host.networkOrigins: ["*"]` admits any HTTPS server, plus plain HTTP only
at parsed loopback IP addresses (127.0.0.0/8 and [::1]). Hostnames such as
`localhost` never qualify for HTTP and are not resolved. The wildcard must
be the sole entry; URL credentials are refused and redirects stay off.
Exact HTTPS origins keep their existing rules.

The install review says plainly what this means: the plugin chooses its
destinations and can send what it reads to them, including services on
this computer.

Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@squareup.com>
Signed-off-by: Tyler Longwell <tlongwell@squareup.com>
…y into agent turns

When one of your agents starts a turn, Recall finds the few past messages
and memories that would help with the request and quotes them into it.
Bundled, off by default.

  find    relay full-text search as the agent (this channel by default),
          the agent's memory scored on-device, and optionally vectors
          from an embedding server you run; merged by rank fusion
  choose  one bounded call to your Decisions provider; only excerpts it
          says help survive, and a failure or timeout keeps nothing
  pack    escaped, attributed one-line excerpts within 2,400 bytes,
          framed as other people's words, not instructions

An excerpt appears only where everyone who can read the current chat could
already read its source. Memories appear only in chats between the owner
and the agent unless the owner opts in. Any failure leaves the turn exactly
as it would have been without Recall.

Semantic recall is optional: set an Embedding server URL for any
llama.cpp-compatible /v1/embeddings server. Buzz does not install or run
the model. Dimensions come from the server, a model change clears the
in-RAM index, and an empty URL means keyword-only recall.

`invariants.test.ts` pairs each safety invariant with a control that does
recall something; `mutate.sh` removes each guard in turn and expects the
suite to fail.

Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@squareup.com>
Signed-off-by: Tyler Longwell <tlongwell@squareup.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant