Merge pull request #101 from docker/feat/host-mount-capability

feat(spec,tck): add host-shared directory capability

Author: cdupuis

Taskfile.yaml

@@ -15,7 +15,7 @@ vars:
   # so they only build once those kits have been pushed to the namespace
   # their `registry` arg names. Build one on its own:
   # `task kit:build KIT=team BUILD_ARGS='--build-arg registry=docker.io/me'`.
-  KITS: hello gh tool shell alpine builder motd wordpress claude claude-mixin claude-acp codex codex-mixin codex-acp cursor cursor-mixin devin devin-mixin docker-agent docker-agent-mixin gemini gemini-mixin opencode opencode-mixin skills dev-tools optional-cache git-signing github-ssh review-skill
+  KITS: hello gh tool shell alpine builder motd wordpress claude claude-mixin claude-acp codex codex-mixin codex-acp cursor cursor-mixin devin devin-mixin docker-agent docker-agent-mixin gemini gemini-mixin opencode opencode-mixin skills dev-tools optional-cache shared-cache git-signing github-ssh review-skill
   # Empty means "the kit's own version answers": build and push refresh an
   # upstream-tracked descriptor from the vendor's latest release and tag
   # with it, so an image tag can never name a version the kit did not

docs/spec/SPEC-v3.md

@@ -567,7 +567,8 @@ other.
 
 **Instance-shaped types appear once per thing requested**, deduplicated on
 their own key: `credential@1` on (service, phase), with each listed phase
-participating independently, `volume@1` on path,
+participating independently, `volume@1` and `host-mount@1` on a shared
+cleaned storage path key,
 `agent-skills@1` on path,
 `agent-skill@1` on effective name, `port@1` on (container, transport),
 `ssh-agent@1` on each phase it names.
@@ -684,6 +685,7 @@ behavior** for a runtime supporting the type:
 | `com.docker.sandbox/credential@1` | [credential@1](capabilities/com.docker.sandbox/credential@1.md) | per (service, phase) |
 | `com.docker.sandbox/ssh-agent@1` | [ssh-agent@1](capabilities/com.docker.sandbox/ssh-agent@1.md) | per phase (one entry may name both) |
 | `com.docker.sandbox/volume@1` | [volume@1](capabilities/com.docker.sandbox/volume@1.md) | per path |
+| `com.docker.sandbox/host-mount@1` | [host-mount@1](capabilities/com.docker.sandbox/host-mount@1.md) | per path; shared storage key with volume@1 |
 | `com.docker.sandbox/port@1` | [port@1](capabilities/com.docker.sandbox/port@1.md) | per (container, transport) |
 | `com.docker.sandbox/usb-device@1` | [usb-device@1](capabilities/com.docker.sandbox/usb-device@1.md) | instance |
 | `com.docker.sandbox/resources@1` | [resources@1](capabilities/com.docker.sandbox/resources@1.md) | singleton |

docs/spec/capabilities/com.docker.sandbox/host-mount@1.md

@@ -0,0 +1,81 @@
+# `com.docker.sandbox/host-mount@1`
+
+One host directory shared with the host and across sandboxes composing
+the same Kit. The runtime chooses its location and mount mechanism;
+Linux bind mounts and VM filesystem sharing can satisfy the same grant.
+Use [volume@1](volume@1.md) for sandbox storage without this sharing
+contract.
+
+- **Shape**: instance — one entry per path; duplicates rejected.
+- **Permission surface**: yes — the path, separately marked host-shared.
+
+## Config
+
+```yaml
+- type: com.docker.sandbox/host-mount@1
+  config:
+    path: /home/agent/.cache/pip   # REQUIRED, absolute and canonical
+    mode: "0755"                 # optional initial directory permissions
+```
+
+| Field | Type | Rules |
+|---|---|---|
+| `path` | string | REQUIRED. Absolute, canonical in-container path; no `.`, `..`, doubled separators, trailing slash, NUL, or root `/`. |
+| `mode` | string | optional. Octal (`755`, `0755`, `1777`), applied when the runtime creates the directory. |
+
+There is no host-path field. The Kit requests where storage appears
+inside the sandbox, not which host files it can access.
+
+## Runtime behavior
+
+A conforming runtime:
+
+- **MUST NOT** let the Kit choose the host directory's location. <!-- tck: host-mount@1/runtime-owned-location -->
+  The runtime owns it; a user MAY explicitly choose a replacement.
+- **MUST** key the directory on the declaring Kit's identity and the <!-- tck: host-mount@1/isolated-by-kit -->
+  cleaned `path`, so sandboxes composing the same Kit share storage,
+  while another Kit declaring that path does not inherit it.
+- **MUST** keep distinct destinations of the same Kit in separate <!-- tck: host-mount@1/isolated-by-path -->
+  directories. A Kit's identity alone is not the storage key.
+- **MUST** use a Kit identity another Kit cannot claim. A published <!-- tck: host-mount@1/identity-not-self-declared -->
+  repository can supply that identity; descriptor display metadata and
+  `source` attribution cannot. Identity for unpublished Kits is
+  runtime-owned.
+- **MUST** reuse the directory across tag or digest updates within one <!-- tck: host-mount@1/survives-kit-update -->
+  Kit identity. Updating a Kit does not allocate fresh storage.
+- **MUST** mount the directory at `path` before lifecycle hooks run. <!-- tck: host-mount@1/mounted-before-hooks -->
+- **MUST** allow concurrent sandboxes of the same Kit to read and write <!-- tck: host-mount@1/shared-concurrently -->
+  the directory. Kits coordinate access; the grant promises no locking
+  or transactional behavior.
+- **MUST** keep the directory's contents independently of any sandbox, <!-- tck: host-mount@1/survives-sandbox-removal -->
+  including after its last sandbox is removed.
+- **MUST** let the user find and remove the directory and access its <!-- tck: host-mount@1/listed-and-removable -->
+  contents from the host. Sandbox writes reach this directory.
+- **SHOULD** make the mount root writable by the agent user (uid 1000). <!-- tck: host-mount@1/agent-writable-root -->
+- **SHOULD** apply `mode` when creating the directory; reopening it <!-- tck: host-mount@1/initial-mode-applied -->
+  preserves existing permissions and contents.
+
+Filesystem semantics MAY be weaker than those of `volume@1`: ownership
+may be mapped from the host and overlayfs upper layers or xattrs may be
+unavailable. Kits requiring those semantics use `volume@1`.
+
+A runtime without host-directory sharing does not advertise this type.
+An unclaimed required entry **MUST** fail closed; an optional one <!-- tck: host-mount@1/unadvertised-is-unmet -->
+**MUST** be skipped and recorded without a host-sharing grant.
+
+## Composition
+
+Paths union across the set. `host-mount@1` and `volume@1` share the
+cleaned in-container path as their storage identity key.
+
+Two Kits declaring the same path, whether both use `host-mount@1` or <!-- tck: host-mount@1/no-silent-merge -->
+one uses `volume@1`, conflict. A runtime **MUST NOT** silently merge
+them, even when their configs are identical: shared storage has one
+declaring Kit identity.
+
+## Gate
+
+The permission surface **MUST** list host-shared paths separately from <!-- tck: host-mount@1/separate-permission-surface -->
+`volume@1` storage paths. Data written here reaches the host and other
+sandboxes of the same Kit. A new path widens the grant; moving a path
+from `volume@1` to `host-mount@1` also widens it. `mode` changes do not.

docs/spec/capabilities/com.docker.sandbox/volume@1.md

@@ -44,11 +44,23 @@ A conforming runtime:
   [lifecycle](lifecycle@1.md) startup hook (a fresh mount may come up
   root-owned).
 
+Whether removing a sandbox and later creating one with the same name
+counts as a recreate is runtime-owned. Removal may delete its storage
+or retain it for a later sandbox.
+
+A runtime reattaching storage across sandbox removal **MUST** check <!-- tck: volume@1/reattach-checks-kit-identity -->
+the Kit identity before attaching it, so a different Kit cannot inherit
+the data. Descriptor display metadata and `source` attribution are not
+sufficient identity evidence.
+
 ## Composition
 
 Paths union across the set. Two Kits declaring the same path is a
 composition conflict — a runtime MUST NOT silently merge them. <!-- tck: volume@1/no-silent-merge -->
 
+The cleaned path also conflicts with a
+[host-mount@1](host-mount@1.md) declaration at that destination.
+
 ## Gate
 
 The path is permission surface. A new path widens; `size`/`mode` changes do

docs/spec/conformance.md

@@ -72,6 +72,9 @@ An adapter **MUST NOT** require interactive input.
 | `rm` | `<id>` | — | Discard the sandbox |
 | `wait-idle` | `<id>` | — | Disconnect the final client session and wait beyond the normal auto-stop grace period |
 | `status` | `<id>` | `running` or `stopped` | Observe sandbox state without starting it or attaching a session |
+| `host-mounts` | `<kit-ref>` | JSON array of `{id, path, hostPath}` | List retained host directories for the resolved Kit identity |
+| `host-mount-read` | `<mount-id> <relative-path>` | the file's bytes | Observe sandbox writes from the host |
+| `host-mount-rm` | `<mount-id>` | — | Remove a runtime-owned directory by its opaque listing handle |
 
 `create --env name=value` supplies a container environment override.
 Adapters **MUST** apply it after image defaults and Kit argument exports,
@@ -127,6 +130,42 @@ not a stopped state. The suite reads status before probing the background
 process, so an exec that implicitly starts a stopped sandbox cannot hide
 auto-stop, and reads it again after explicit stop.
 
+### Host-shared directory observation
+
+The `host-mount-*` verbs are required only for adapters claiming
+`com.docker.sandbox/host-mount@1`. They use the runtime's ordinary user
+interfaces for discovering, reading, and removing host directories.
+Adapters **MUST NOT** manufacture storage outside the runtime's
+provisioning path to satisfy these checks.
+
+`host-mounts` resolves the Kit reference to the same identity used by
+`create`, and returns its retained directories even with no live sandbox.
+Each record **MUST** contain a nonempty opaque `id`, its canonical
+in-container `path`, and the host location `hostPath` a user can find.
+There is one record per path; an empty listing is `[]`, not `null`.
+Listing is an observation and **MUST NOT** allocate storage.
+
+The `host-mount-version-v1` and `host-mount-version-v2` fixtures exercise
+updates within one published Kit identity. Adapters claiming
+`host-mount@1` **MUST** publish or import them as distinct versions of one
+suite-owned repository, resolving their supplied fixture references to
+those versions for `create` and `host-mounts`. Distinct tags or digests
+identify the versions; neither version may be substituted with the
+other. The suite writes through the first version, removes its sandbox,
+and reads through the second, then verifies writes from the second are
+visible when reopening the first. Testing two unrelated local Kit
+identities cannot establish this repository identity guarantee.
+
+`host-mount-read` **MUST** read from the host directory independently of
+sandbox exec. The suite passes only a relative fixture filename; an
+absent file returns a nonzero status. `host-mount-rm` **MUST** remove the
+listed directory and its contents so a later create starts empty.
+
+The suite uses fresh in-container paths for each check. It removes only
+directories at those paths for the fixture Kits, after removing their
+sandboxes; adapters **MUST NOT** interpret cleanup as a request to remove
+other Kit directories or user content.
+
 ### Git identity binding
 
 An adapter claiming `com.docker.sandbox/git-identity@1` **MUST** accept

examples/shared-cache/README.md

@@ -0,0 +1,26 @@
+# Shared pip cache
+
+A declaration-only mixin using `com.docker.sandbox/host-mount@1` to
+share pip downloads across sandboxes composing this Kit. The runtime
+chooses the host directory and how it appears inside the sandbox.
+No image content or host path is supplied by the Kit.
+
+The optional group couples the mount with pip's configuration file.
+A runtime without host sharing skips both requests; pip then uses its
+ordinary cache. The workload must already provide pip.
+
+The cache is keyed by this Kit's runtime-resolved identity and the
+in-container path. It survives removal of the last sandbox and remains
+accessible to the host user, who can find and remove it through the
+runtime's storage interface. Concurrent sandboxes share read/write
+access; pip is responsible for coordinating cache writes.
+
+This grants host sharing as a separate permission from sandbox-private
+storage. For a cache without that sharing contract, see
+[optional-cache](../optional-cache/README.md).
+
+Build against the current frontend source:
+
+```sh
+task kit:dev KIT=shared-cache
+```

examples/shared-cache/shared-cache.yaml

@@ -0,0 +1,24 @@
+# syntax=docker/sandbox-kit:3
+# yaml-language-server: $schema=../../schema/kit.schema.json
+schemaVersion: "3"
+kind: mixin
+displayName: Shared pip cache
+description: Reuse pip downloads across sandboxes of this Kit and the host
+capabilities:
+  - group:
+      name: Host-shared pip cache
+      description: Share cache files across sandboxes and retain them after removal
+      optional: true
+      capabilities:
+        - type: com.docker.sandbox/host-mount@1
+          config:
+            path: /home/agent/.cache/pip
+            mode: "0755"
+        - type: com.docker.sandbox/lifecycle@1
+          config:
+            files:
+              - path: /home/agent/.config/pip/pip.conf
+                overwrite: false
+                content: |
+                  [global]
+                  cache-dir = /home/agent/.cache/pip

schema/capabilities/com.docker.sandbox/host-mount@1.schema.json

@@ -0,0 +1,34 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "title": "com.docker.sandbox/host-mount@1",
+  "description": "Config schema for a runtime-owned host directory shared with the host and across sandboxes of one Kit. Instance-shaped, keyed by the cleaned in-container path, sharing its storage identity key with volume@1. No Kit-supplied host path.",
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "path"
+  ],
+  "properties": {
+    "path": {
+      "type": "string",
+      "anyOf": [
+        {
+          "pattern": "^(/([^./\\x00][^/\\x00]*|\\.[^./\\x00][^/\\x00]*|\\.\\.[^/\\x00]+))+$"
+        },
+        {
+          "$ref": "../../definitions/kit-arg.schema.json#/definitions/bearing"
+        }
+      ]
+    },
+    "mode": {
+      "type": "string",
+      "anyOf": [
+        {
+          "pattern": "^[0-7]{3,4}$"
+        },
+        {
+          "$ref": "../../definitions/kit-arg.schema.json#/definitions/bearing"
+        }
+      ]
+    }
+  }
+}

schema/kit.schema.json

@@ -391,6 +391,28 @@
             }
           }
         },
+        {
+          "if": {
+            "properties": {
+              "type": {
+                "const": "com.docker.sandbox/host-mount@1"
+              }
+            },
+            "required": [
+              "type"
+            ]
+          },
+          "then": {
+            "properties": {
+              "config": {
+                "$ref": "capabilities/com.docker.sandbox/host-mount@1.schema.json"
+              }
+            },
+            "required": [
+              "config"
+            ]
+          }
+        },
         {
           "if": {
             "properties": {

skills/create-kit-v3/SKILL.md

@@ -131,6 +131,7 @@ often got wrong:
 | `ssh-agent@1` | git over SSH, SSH commit signing | Set `unrestricted: false` to bound it. `sign: [git]` is all commit signing needs; `authenticate: [git@github.com]` is a login to one server. An entry with `unrestricted: true` (the default) signs anything with every key the user's agent holds — declare that only when a client cannot bind sessions (OpenSSH can), and prefer `credential@1` with HTTPS when a token will do. `optional: true` unless the kit cannot work without it: many users have no agent running. A hook that uses it declares `SSH_AUTH_SOCK` in `env:`, and only hooks of the granted phase get it. `authenticate` grants a signature, not a connection: the server must also be reachable under the network policy. `phase` accepts a single phase or `[install, runtime]` with shared rules. Commit signing needs git config too (`gpg.format=ssh`, `gpg.ssh.defaultKeyCommand="ssh-add -L"`). |
 | `lifecycle@1` | install/startup hooks, staged files | Hook environments are **deny-by-default**. Declare every variable in `env:`, including ones only a child process reads — `curl`, `pip` and `npm` need `HTTP_PROXY`/`HTTPS_PROXY`, and `docker` needs `DOCKER_HOST`. |
 | `volume@1` | persistent paths | Always set `size`. An unsized kit volume is formatted at 512 MiB, which is a cache or a package store running out of room mid-run rather than anything visible at create. |
+| `host-mount@1` | host-shared caches, datasets, or artifacts | Declare only the absolute, canonical in-container `path` and optional octal `mode`. The runtime owns the host location. Data is shared across sandboxes of the declaring Kit, survives sandbox removal, and is visible to the host user. Use an optional group to couple cache setup with the mount; see `examples/shared-cache`. |
 | `agent-context@1` | instructions the agent reads | A workload can supply the legacy workspace-sibling `filename`. An agent workload or mixin supplies `filename` plus an absolute `directory` at its discovery location; this overrides the legacy fallback. Different explicit destinations conflict. Tool mixins contribute bodies alone. Use `contentFile:` for a static body, but inline `content:` when the body interpolates an arg — a staged body is never arg-expanded. |
 | `long-running@1` | workloads or service mixins that outlive client sessions | Config-less; a request from any Kit applies to the whole sandbox. A background hook or published port does not prevent session auto-stop. Required by default; use `optional: true` only if auto-stop is tolerable. This does not request restart after failure. |
 | `git-identity@1` | commits attributed to the runtime-provided user | Config-less; requests only user.name and user.email, not a configuration-source mount. Requires permission; optional when missing identity is tolerable. Signing/authentication are separate grants. |
@@ -139,6 +140,14 @@ often got wrong:
 | `agent-skills@1` | where an agent discovers skills | Declare on the agent workload or agent mixin. The runtime links or otherwise exposes every selected `agent-skill@1` bundle here, and includes host-shared skills when available and enabled. Missing host skills never block startup; `mode` bounds host-store access only. |
 | `port@1`, `resources@1`, `privileged@1` | inbound ports, limits, elevation | Do not declare on speculation; `privileged@1` is the largest widening available. |
 
+Host sharing is a separate grant from `volume@1`, including when the
+in-container path stays the same. Do not supply a host path or assume a
+Linux bind mount: runtimes may use VM filesystem sharing. Host-shared
+directories may have weaker filesystem semantics than private volumes.
+Concurrent writers coordinate access themselves. The initial `mode`
+does not reset existing directory permissions on later creates.
+Two host mounts at one path, or a host mount and a volume there, conflict.
+
 ## Bundled skills
 
 Use `agent-skill@1` to package a skill as a mixin; see