[Bug]: Muse refresh keeps offering a model removed from a complete catalog #34982

Before submitting

• I searched existing issues and did not find a duplicate.
• I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

Use a fresh clone at the commit in Version or commit with the repository's locked dependencies installed. Save the script below as repro.sh (anywhere outside the checkout) and run it from the checkout root, passing the checkout root as its argument:

tmp="$(mktemp -d)"
# save the script below as "$tmp/repro.sh"
mkdir -p "$tmp/scratch"
TMPDIR="$tmp/scratch" sh "$tmp/repro.sh" "$PWD"

The script needs no credentials and starts no model turn. It creates a per-run fixture with private HOME, TMPDIR and T3CODE_HOME, a version-only fake muse executable, and SDK host responses injected through the existing createHost argument of checkMuseProviderStatus; application sources are unchanged, and the fixture is removed on exit. It then:

1. Builds a real managed Muse provider (makeManagedServerProvider with checkMuseProviderStatus) and the real ProviderRegistry.layer, with a complete model/list response (source: "providerCatalog") listing muse-current-fixture and muse-retired-fixture, both ordinary non-custom rows. It calls ProviderRegistry.refreshInstance("muse") and checks that both appear.
2. Changes the fixture to a complete, nonempty providerCatalog response listing only muse-current-fixture and calls the same registry refresh. It checks that the real status probe returned installed/ready with only the current row, and that the registry result still contains both rows.
3. JSON-round-trips the returned array in the { providers } shape the server refresh handler returns, passes it through the real web deriveProviderInstanceEntries, resolveSelectableProviderInstanceEntry, isProviderInstancePickerReady and shared resolveSelectableModel, and checks that muse-retired-fixture resolves as selectable. It also reads the isolated muse.json status cache and checks that it contains the retired slug.
4. Runs controls: the initial list is correct, the current row still resolves, a failed Muse catalog read (probe status error) keeps the previous list, and mergeProviderSnapshot with the same successful-probe shape relabeled as Cursor drops the missing row.

The script exits zero only when the stale Muse row survives and every control passes. It closes all fixture SDK hosts before exiting.

#!/bin/sh
set -eu
ROOT=$(cd "${1:-.}" && pwd)
: "${TMPDIR:?Set TMPDIR to an isolated scratch directory}"
RUN=$(mktemp -d "$TMPDIR/muse-registry.XXXXXX")
trap 'rm -rf "$RUN"' EXIT HUP INT TERM
mkdir -p "$RUN/userhome" "$RUN/tmp" "$RUN/t3"
export HOME="$RUN/userhome" TMPDIR="$RUN/tmp" T3CODE_HOME="$RUN/t3" CHECKOUT_ROOT="$ROOT" REPRO_RUN="$RUN"
cat > "$RUN/muse" <<'CLI'
#!/bin/sh
if [ "$1" = '--version' ]; then
  echo 'muse 1.0.3-R2198.1'
else
  exit 9
fi
CLI
chmod +x "$RUN/muse"
ln -s "$ROOT/apps/server/node_modules" "$RUN/node_modules"
cat > "$RUN/probe.mjs" <<'JS'
import assert from 'node:assert/strict';
import { pathToFileURL } from 'node:url';
import { readFileSync } from 'node:fs';
const root = process.env.CHECKOUT_ROOT;
const run = process.env.REPRO_RUN;
const pkg = async (name) => import(name);
const local = async (name) => import(pathToFileURL(`${root}/${name}`).href);
const Effect = await pkg('effect/Effect');
const Layer = await pkg('effect/Layer');
const Stream = await pkg('effect/Stream');
const Schema = await pkg('effect/Schema');
const PubSub = await pkg('effect/PubSub');
const NodeServices = await pkg('@effect/platform-node/NodeServices');
const Contracts = await pkg('@t3tools/contracts');
const { makeManagedServerProvider } = await pkg('@t3tools/provider-core/server/managedProvider');
const { ProviderHost } = await pkg('@t3tools/provider-core/server/ProviderHost');
const Latest = await pkg('@t3tools/provider-core/server/ProviderLatestVersions');
const { makeManualOnlyProviderMaintenanceCapabilities } = await pkg('@t3tools/provider-core/server/maintenanceResolver');
const { resolveSelectableModel } = await pkg('@t3tools/shared/model');
const { MuseSettings } = await local('packages/provider-muse/src/settings.ts');
const { checkMuseProviderStatus, makePendingMuseProvider } = await local('packages/provider-muse/src/server/status.ts');
const Registry = await local('apps/server/src/provider/ProviderRegistry.ts');
const Instances = await local('apps/server/src/provider/ProviderInstanceRegistry.ts');
const Mutator = await local('apps/server/src/provider/ProviderInstanceRegistryMutator.ts');
const Manifest = await local('apps/server/src/provider/ModelManifest.ts');
const Config = await local('apps/server/src/config.ts');
const Client = await local('apps/web/src/providerInstances.ts');
const settings = Schema.decodeSync(MuseSettings)({enabled:true,binaryPath:`${run}/muse`});
let listed = ['muse-current-fixture','muse-retired-fixture'];
let failed = false;
let requests = 0;
let closes = 0;
const snapshots = [];
const createHost = async () => ({
  initializeResult:{grantedCapabilities:[]},
  connection:{
    request: async (method, input) => {
      assert.equal(method,'model/list'); assert.deepEqual(input,{}); requests++;
      if (failed) throw new Error('fixture catalog failure');
      return {providerId:'meta',profileId:null,source:'providerCatalog',models:listed.map((modelId,index)=>({modelId,displayLabel:modelId,providerId:'meta',profileId:null,isDefault:index===0,isActive:false,contextLimit:null,outputLimit:null,cost:null,description:null,releaseDate:null,variants:'unknown'}))};
    },
    command:async()=>({}),mintCommandId:()=> 'fixture',onNotification:()=>{},onServerRequest:()=>{},onProtocolError:()=>{},closed:new Promise(()=>{}),
  },
  exited:new Promise(()=>{}),close:async()=>{closes++;},
});
const stamp = (draft) => ({...draft,instanceId:'muse',driver:'muse'});
const slugs = (provider) => provider.models.map(x=>x.slug);
const program = Effect.gen(function*(){
  const services = yield* Effect.context();
  const host = {
    paths:{cwd:run,baseDir:`${run}/t3`,stateDir:`${run}/t3/state`,providerStatusCacheDir:`${run}/t3/caches`,attachmentsDir:`${run}/t3/attachments`},
    settings:{get:Effect.succeed(Contracts.DEFAULT_SERVER_SETTINGS),withSnapshot:use=>use(Contracts.DEFAULT_SERVER_SETTINGS),changes:Stream.empty,subscribe:Effect.succeed(Stream.empty)},
    shouldRunBackgroundWork:()=>Effect.succeed(false),resolveAttachmentPath:()=>null,credentials:()=>Effect.die('unused'),
  };
  const check = checkMuseProviderStatus(settings,process.env,run,createHost).pipe(
    Effect.map(stamp), Effect.tap(snapshot=>Effect.sync(()=>snapshots.push(snapshot))),Effect.provideContext(services),
  );
  const managed = yield* makeManagedServerProvider({
    resolveMaintenance:()=>Effect.succeed(makeManualOnlyProviderMaintenanceCapabilities({provider:'muse',packageName:null})),
    getSettings:Effect.succeed(settings),streamSettings:Stream.empty,haveSettingsChanged:()=>false,
    initialSnapshot:()=>makePendingMuseProvider(settings).pipe(Effect.map(stamp)),checkProvider:check,refreshInterval:0,refreshOnInterval:false,
  }).pipe(Effect.provideService(ProviderHost,host));
  const instance = {
    instanceId:'muse',driverKind:'muse',continuationIdentity:{driverKind:'muse',continuationKey:'muse:instance:muse'},displayName:undefined,enabled:true,snapshot:managed,
    orchestrationAdapter:{},textGeneration:{},
  };
  const changes = yield* PubSub.unbounded();
  yield* Effect.addFinalizer(()=>PubSub.shutdown(changes));
  const instances = {getInstance:id=>Effect.succeed(id==='muse'?instance:undefined),listInstances:Effect.succeed([instance]),listUnavailable:Effect.succeed([]),streamChanges:Stream.empty,subscribeChanges:PubSub.subscribe(changes)};
  const registryLayer = Registry.layer.pipe(
    Layer.provide(Layer.succeed(Instances.ProviderInstanceRegistry,instances)),
    Layer.provide(Layer.succeed(Mutator.ProviderInstanceRegistryMutator,{reconcile:()=>Effect.void,reloadSecrets:()=>Effect.void})),
    Layer.provide(Manifest.layerTest),Layer.provide(Latest.layer),
    Layer.provideMerge(Config.layerTest(run,`${run}/t3`)),Layer.provide(NodeServices.layer),
  );
  yield* Effect.gen(function*(){
    const registry = yield* Registry.ProviderRegistry;
    const first = (yield* registry.refreshInstance('muse'))[0];
    assert.deepEqual(slugs(first),listed); assert.equal(first.status,'ready');
    console.log('CONTROL initial registry refresh:',JSON.stringify({status:first.status,auth:first.auth,models:slugs(first)}));
    listed = ['muse-current-fixture'];
    const providers = yield* registry.refreshInstance('muse');
    const nextProbe = snapshots.at(-1);
    assert.deepEqual(slugs(nextProbe),listed);assert.equal(nextProbe.status,'ready');assert.equal(nextProbe.installed,true);
    console.log('REAL Muse status probe after catalog shrink:',JSON.stringify({status:nextProbe.status,auth:nextProbe.auth,models:slugs(nextProbe)}));
    // ws.ts refreshProviders returns this exact shape when no optional decoration is requested.
    const response = JSON.parse(JSON.stringify({providers}));
    const stale = response.providers[0];
    assert.deepEqual(slugs(stale),['muse-current-fixture','muse-retired-fixture']);
    console.log('REGISTRY refresh result serialized as refreshProviders body:',JSON.stringify({status:stale.status,availability:stale.availability,models:slugs(stale)}));
    const entry = Client.resolveSelectableProviderInstanceEntry(Client.deriveProviderInstanceEntries(response.providers),'muse');
    assert.ok(entry);assert.equal(Client.isProviderInstancePickerReady(entry),true);
    const selected = resolveSelectableModel(entry.driverKind,'muse-retired-fixture',entry.models);
    assert.equal(selected,'muse-retired-fixture');assert.ok(entry.models.some(x=>x.slug===selected));
    console.log('CLIENT picker ready:',Client.isProviderInstancePickerReady(entry),'selected:',selected);
    assert.equal(resolveSelectableModel(entry.driverKind,'muse-current-fixture',entry.models),'muse-current-fixture');
    console.log('CONTROL client can choose current model: muse-current-fixture');
    const config = yield* Config.ServerConfig;
    const cache = JSON.parse(readFileSync(`${config.providerStatusCacheDir}/muse.json`,'utf8'));
    console.log('PERSISTED cache contains retired slug:',JSON.stringify(cache).includes('muse-retired-fixture'));
    assert.ok(JSON.stringify(cache).includes('muse-retired-fixture'));
    const cursor = Registry.mergeProviderSnapshot({...first,instanceId:'cursor',driver:'cursor'},{...nextProbe,instanceId:'cursor',driver:'cursor'});
    assert.deepEqual(slugs(cursor),listed);
    console.log('CONTROL Cursor merge with same successful probe shape:',JSON.stringify(slugs(cursor)));
    failed=true;
    const errorProviders=yield* registry.refreshInstance('muse');
    assert.equal(snapshots.at(-1).status,'error');assert.deepEqual(slugs(snapshots.at(-1)),[]);
    assert.deepEqual(slugs(errorProviders[0]),slugs(first));
    console.log('CONTROL failed Muse discovery:',JSON.stringify({probe:snapshots.at(-1).status,retained:slugs(errorProviders[0])}));
    console.log('REPRODUCED: completed Muse discovery drops a model, registry and selectable client inventory retain it.');
  }).pipe(Effect.provide(registryLayer));
});
await Effect.runPromise(program.pipe(Effect.scoped,Effect.provide(NodeServices.layer)));
assert.ok(requests>=3);assert.equal(closes,requests);
console.log('CLEANUP: all',closes,'SDK fixture hosts closed; scoped registry and managed-provider fibers stopped.');
JS
cd "$ROOT/apps/server"
node "$RUN/probe.mjs"

The production caller of this registry operation is the server refresh handler (

Expected behavior

When Muse returns a successful, complete providerCatalog with fewer visible rows than before, the refreshed Muse model choices should match that snapshot, so a row Muse no longer lists is no longer offered or selectable. T3's Muse documentation (

This expectation covers only a successful, nonempty providerCatalog response. It does not treat an empty or bundled catalog, a failed catalog read, a ready status or a cached model list as proof of account access, and keeping the previous list when the catalog read fails remains correct. The same documentation notes that Muse can list models an account cannot use, so this report makes no claim about whether a retained row is usable.

Actual behavior

Summary. When Muse's model/list returns a complete, nonempty providerCatalog that no longer contains a model it listed before, the real Muse status probe reports only the current models, but ProviderRegistry refresh still returns the removed model, keeps the provider ready, and writes the removed model to the persisted provider status cache. The web client's selection helpers then accept the removed model from that inventory. Merged #28018 (fixing #28008) made a completed catalog refresh drop missing models for Cursor, and its description says the other drivers that fall through to default retention (claudeAgent, fx, muse) were deliberately left unchanged. This report does not say #28018 missed Muse; it asks for the Muse follow-up on the basis of Muse's own snapshot contract and T3's Muse refresh documentation. Evidence comes from controlled SDK responses driven through the real Muse status check, managed provider, registry refresh and cache writer, plus the real client selection helpers; no live WebSocket exchange, rendered picker, real account or catalog change, or paid model turn was exercised.

Observed at both tested builds with identical output: after the catalog shrinks, the real Muse status probe returns installed/ready with only muse-current-fixture, but the registry refresh returns both muse-current-fixture and muse-retired-fixture with status ready, and the isolated muse.json status cache contains muse-retired-fixture. The real client helpers mark the Muse entry picker-ready and resolve muse-retired-fixture as a selectable model. The controls behave as expected: the current row resolves, a failed Muse catalog read keeps the previous two rows, and the same successful-probe shape labeled as Cursor drops the missing row.

Source path at the baseline: the Muse model/list decoder (

Read from source only, not exercised: the client carries returned models into provider instance entries (

fx has the same shape from source inspection only (an authenticated, successfully decoded fx models --json listing reports ready, and fx also falls through to default retention); it was not reproduced and is not part of this report.

Hypothesis, not verified: the default retention branch treats a completed Muse catalog shrink the same way as incomplete discovery.

Evidence

• Expected source: T3's Muse documentation, docs/user/providers-muse.md lines 35-44, and the pinned @muse-code/sdk 1.4.6 dist/src/msp.d.ts (ModelCatalogEntry and ModelListResult doc comments) defining model/list as a call-time snapshot of visible, selectable rows.
• Failure source: observed output of repro.sh above, through checkMuseProviderStatus (packages/provider-muse/src/server/status.ts lines 125-153) → managed provider → ProviderRegistry.refreshInstance (apps/server/src/provider/ProviderRegistry.ts lines 616-631, 490-548) → shouldRetainMissingProviderModels default true (lines 179-211) and mergeProviderModels (lines 235-265) → persisted muse.json and the client selection helpers.
• Evidence provenance: observed
• Local verification: reproduced
• Reproduction completeness: complete

The reproduction controls only the external muse --version output and the SDK host's model/list responses; the Muse status check, managed provider, registry, cache writer and client helpers are unchanged production code. Instance discovery, the model manifest and latest-version lookups are test layers. It does not start an HTTP or WebSocket server, render the picker, use a real Muse account, or run a model turn.

The script output is under Logs or stack traces; it is identical at both tested builds.

Restoration check

With the script above, after the complete providerCatalog shrinks to muse-current-fixture, the registry refresh result and the persisted muse.json cache contain only muse-current-fixture, and resolveSelectableModel no longer resolves muse-retired-fixture from the returned inventory (the script's stale-row assertions then fail by design). The controls must keep their current results: the initial list holds both rows, the current row resolves, and a failed Muse catalog read (probe error) still keeps the previous list. An empty successful catalog (probe warning) should also keep its current retention behavior.

References

• Cursor model list never drops models after an account change; refreshes only ever add to it #28008 (Cursor model list never drops models after an account change; refreshes only ever add to it #28008), closed, and merged fix(cursor): a completed catalog refresh drops models it no longer lists #28018 (fix(cursor): a completed catalog refresh drops models it no longer lists #28018): a completed Cursor catalog refresh now drops models it no longer lists, by giving Cursor the rule Grok already had (keep previous models only while the probe is not installed and ready). fix(cursor): a completed catalog refresh drops models it no longer lists #28018 states: "Other drivers that fall through to the default (claudeAgent, fx, muse) are deliberately left unchanged." Cursor model list never drops models after an account change; refreshes only ever add to it #28008 noted from source reading that Muse also falls through to default retention but did not reproduce it.
• Related, not duplicates: [Bug]: Muse provider warns "returned no models" and asks for login on a working install #34260 (closed) concerns an empty bundled catalog producing a misleading warning, a different input; [Bug]: Muse Ember 1.0 doesn't work even though it works using standard terminal while having Muse Code sub #34409 (open) reports a Muse Ember turn failure without a catalog-removal transition, and this reproduction does not establish its cause; merged fix(server): refresh stale model catalogs before refusing delegation #33426 refreshes stale catalogs before refusing delegation, which adds newly available models rather than removing missing ones.

Impact

Minor bug or occasional failure

Assessed P2 (supported). After a complete, nonempty Muse catalog drops a model, the removed built-in stays in the refreshed inventory, remains selectable through the client helpers and is persisted to the status cache, which can mislead model selection; the reproduction uses controlled SDK responses, and no real account change, rendered picker, failed model turn or data loss was observed.

Version or commit

commit d9805b2; repeated on main at 4be96e0 with identical output; current main 2de3370 leaves ProviderRegistry.ts unchanged (source comparison, not rerun)

Environment

macOS arm64, Node v26.12.1, repository-pinned dependencies including @muse-code/sdk 1.4.6. Controlled SDK model/list responses and a version-only fake Muse CLI driven through the real Muse status check, registry refresh, cache writer and client selection helpers; no real Muse account, model turn or rendered client.

Logs or stack traces

CONTROL initial registry refresh: {"status":"ready","auth":{"status":"unknown"},"models":["muse-current-fixture","muse-retired-fixture"]}
REAL Muse status probe after catalog shrink: {"status":"ready","auth":{"status":"unknown"},"models":["muse-current-fixture"]}
REGISTRY refresh result serialized as refreshProviders body: {"status":"ready","models":["muse-current-fixture","muse-retired-fixture"]}
CLIENT picker ready: true selected: muse-retired-fixture
CONTROL client can choose current model: muse-current-fixture
PERSISTED cache contains retired slug: true
CONTROL Cursor merge with same successful probe shape: ["muse-current-fixture"]
CONTROL failed Muse discovery: {"probe":"error","retained":["muse-current-fixture","muse-retired-fixture"]}
REPRODUCED: completed Muse discovery drops a model, registry and selectable client inventory retain it.
CLEANUP: all 4 SDK fixture hosts closed; scoped registry and managed-provider fibers stopped.

Screenshots, recordings, or supporting files

No response

Workaround

Choosing a model Muse still lists works; the reproduction shows the current row stays selectable, though a model turn with it was not run. Restarting is not a workaround because the stale row is persisted to the status cache and reloaded.