镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Scope all caching to the KG client, and fix stale and incorrect cache entries - #167

Merged
apdavison merged 2 commits into
HumanBrainProject:masterfrom
apdavison:simplify-caching
Oct 20, 2026
Merged

apdavison merged 2 commits into
HumanBrainProject:masterfrom
apdavison:simplify-caching

Conversation

@apdavison

@apdavison apdavison commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Currently, fairgraph caches in four places: two caches in each KGClient (JSON-LD documents, and stored query definitions), and two module-level caches, fairgraph.caching.object_cache (resolved objects) and save_cache (the ids found by existence queries). The review in #80 found several problems with them:

  • The module-level caches are shared by every client in the process. In a web service that creates a client per request with the user's token, an object retrieved for one user can be returned to another, with no permission check, when a proxy to it is resolved.
  • Wrong content.
    • The document cache ignores the release status, and object_cache ignores it when resolving proxies, so "released" and "in progress" content are mixed up.
    • Minimal documents (where the user lacks full access) are returned where full access is required.
    • Cached documents are returned by reference.
  • Stale entries.
    • A rejected update is cached as if it had been saved.
    • Deleted instances stay in save_cache.
    • Releasing, unreleasing or moving an instance doesn't clear the document cache.
    • store_query() leaves the old definition cached.

With this PR, all caching is done by the client:

  • The module-level caches are removed. KGProxy.resolve() retrieves instances through the client's document cache, and returns a new object each time.
  • The ids from existence queries are remembered by the client. These cover the delay before a new instance shows up in KG queries. On reuse, the instance is fetched by id to fill in properties not given locally. That fetch also works straight after creation, which I checked against the pre-production KG.
  • The document cache keeps the in-progress and released documents of each instance separately, and returns copies.
    • A lookup with release_status="any" for an instance that hasn't been released is answered from the cache, but a direct lookup of the released document always asks the KG, so releases made by others are seen.
    • Instances that also have an openMINDS URI, such as controlled terms, are cached once, under their KG URI, so that changing them clears the cache whichever URI was used.
    • The cache is cleared for an instance when it is changed, released, unreleased, moved or deleted through the same client.

It also fixes KGQuery.resolve(), which doesn't currently set the release status of the objects it returns. Resolving their own links therefore reverts to "released", which matters more now that resolution respects the release status strictly.

Changes in behaviour, listed in the release notes:

  • A link to an instance that hasn't been released, including one created in the same session, must now be resolved with release_status="in progress" or "any". Previously, object_cache often returned it anyway. With follow_links, such links are left as proxies, with a warning.
  • Resolving a proxy without a client is no longer possible.
  • fairgraph.caching.object_cache and save_cache no longer exist, and KGClient.cache has a new layout.

Limiting how long cache entries live, or how many there are, is left to #168. Caches now last as long as the client.

Fixes #80

… entries

The module-level caches fairgraph.caching.object_cache and save_cache are removed. They were shared by every client in the process, so in a web service objects retrieved with one user's token could be returned to another user, and they ignored the release status, kept unsaved changes after a rejected update, and kept the ids of deleted instances. The ids of instances found by, or created after, an existence query are now remembered by the client, and KGProxy.resolve() retrieves instances through the client's document cache, returning a new object each time (#80).

The client's document cache now keeps the "in progress" and "released" documents of each instance separately, applies the require_full_data check after the cache lookup, and returns copies. A lookup with release_status="any" for an instance that has not been released is answered from the cache, but a direct lookup of the released document always asks the KG. The cache is now also cleared when an instance is released, unreleased or moved to another space, and deleting an instance removes it from the existence cache. store_query() now invalidates the cached query definition.

Instances that also have an openMINDS URI, such as controlled terms, are cached once, under their KG URI, whichever URI they were retrieved with, so that they are found in the cache by either URI, and cleared from it when they are changed.

KGQuery.resolve() now sets the release status of the objects it returns, so that resolving their own links uses the same release status, where it previously reverted to "released".
@apdavison
apdavison merged commit 1fab5c5 into HumanBrainProject:master Oct 20, 2026
9 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in fairgraph development Oct 20, 2026
@apdavison
apdavison deleted the simplify-caching branch October 20, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

Development

Successfully merging this pull request may close these issues.

Review caching mechanisms

1 participant