# Participant disclosure, cli.sgit.ai > This site is written by the team that builds the sgit CLI, about its own tool. What that means for how to read it. *Source: · generated from the same content as the page, so the two cannot drift.* --- # Participant disclosure This site is published by the sgit project and written by the team that builds the sgit CLI, people and the agents working with them, about their own tool. Read it as a participant's account. What that changes, and what is done about it: - **The command reference is generated, not written.** It is the CLI's account of its own parser, so it cannot be flattering; it can only be out of date, and it names the release it came from. - **The research articles are published as written**, including the ones that report the tool's own bugs (a pull that discarded uncommitted edits, branches that did not work, paths silently rewritten) and the reviews that corrected the team's designs. - **The limits are on their own page**: [Limitations](../docs/limitations.md), moved here from sgit.ai with the rest. - **sgit is beta.** It has no compliance certification of any kind. Corrections are welcome, and are published above the mistake: [the corrections file](../admin/corrections.md). --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/about/participant.html)* # Corrections, cli.sgit.ai > Where the brief was wrong or unclear, what was found, and what was done instead. The brief itself is never edited. *Source: · generated from the same content as the page, so the two cannot drift.* --- # Corrections to the brief The brief is [published as given](../briefs/index.md). Where it was wrong, unclear or overtaken, the finding goes here, dated, newest first, and the brief stays as it was. ## 2026-10-20, while building v0.1.0 1. **The names in the transcript are transcription errors.** "cli.sketo.ai" is `cli.sgit.ai` (the repository's README says so); "Escrip. AI" and "sk.ai" are `sgit.ai`; "the Skit" is sgit. Read throughout as such. 2. **"The API website for the CLI" is read as the CLI's documentation, with the HTTP API in it.** The brief goes on to describe a documentation site, so the site is that; the HTTP API pages moved here too, because the CLI is a client of that API and nothing else documents it as a whole. 3. **"Move" cannot be finished from this repository.** The pages are copied here and marked as moved, but sgit.ai still serves them and does not point here. Until sgit.ai's team adds links and redirects, both copies exist, and the one here is maintained from now on. The request is on [What moved](../admin/moved.md). 4. **"The most technical and the most detailed" needed a line drawn.** Drawn at: pages about the command-line tool, its keys, its protocol and its operation moved; pages about vaults as a platform (vault apps, `_page.json`, the browser bridge, publishing) stayed, since they are about the vault rather than the CLI. Twenty-seven pages moved. 5. **"The articles in the CLI repository" were not articles yet.** Two are article drafts for the newsroom; the rest are research notes, designs, reviews, measurements and fix write-ups for Dinis. Twenty-three of the October documents, those that take one topic as far as it goes, are published as articles, with a *kind* that says which of these each is. Replies to other teams and per-commit review responses were left in the CLI repository. 6. **The guidance "at sgit.ai for how to set up these websites" is two pages.** [Every sgit repository](https://sgit.ai/docs/guidance/repositories.html) and the vault guidance. The sibling sites (pki.sgit.ai, sgit.ai itself) supplied the pipeline. Where this site does not yet meet the guidance (the review folder, roles and a board, a release script that checks the live site), it is said on [the engineering page](../admin/index.md) and in [the reality file](../admin/reality.md). 7. **One difference from the sibling sites, chosen on purpose.** pki.sgit.ai's pages are hand-written HTML with the chrome injected; here every page is generated from markdown, because most of this site's content arrives as markdown (sgit.ai's twins, the CLI repository's documents, the CLI's own reference) and converting it by hand would be the drift the guidance warns about. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/admin/corrections.html)* # Engineering, cli.sgit.ai > How cli.sgit.ai is built, checked and released: one content folder, one generator, one gate, one version, deployed from dev by GitHub Pages; and how it measures against the estate's guidance for every sgit repository. *Source: · generated from the same content as the page, so the two cannot drift.* --- # Engineering: how this site is built and released One folder of content, one generator, one gate, one version number. Deployed to https://cli.sgit.ai by GitHub Pages on every push to `dev`, the same pipeline as [sgit.ai](https://github.com/SGit-AI/SGit-AI__Website) and [pki.sgit.ai](https://github.com/SGit-AI/SGit-AI__Website__PKI). ## The shape | Path | What it is | |---|---| | `admin/content/.md` | the source of the page at `.html`: flat front matter, then markdown | | `admin/content/articles/.md` | one article; adding the file publishes it ([the contract](https://github.com/SGit-AI/SGit-AI__Website__Cli/blob/dev/admin/content/articles/README.md)) | | `admin/content/reference/sgit-cli-reference.json` | the output of `sgit help all --format json`; the command reference is generated from it | | `REALITY.md`, `BRIEF-CORRECTIONS.md` | at the repository root, published as [Reality](../admin/reality.md) and [Corrections](../admin/corrections.md) | | `admin/build/build.py` | the generator: pages, markdown twins, nav and footer, `llms.txt`, `llms-full.txt`, sitemap, `assets/version.js` | | `admin/build/md.py` | the markdown renderer, from pki.sgit.ai and newsroom.sgit.ai; escapes everything, so content cannot carry HTML | | `admin/build/validate.js` | the gate | | `admin/build/import_sgit_ai.py` | moves pages from sgit.ai, from their markdown twins ([what moved](../admin/moved.md)) | | `admin/build/import_cli_research.py` | publishes documents from the CLI repository as articles | | `admin/build/version.txt` | the site version, the one place it is written | | `.github/workflows/deploy-pages.yml` | validate, tag, deploy | Every generated file is listed in `admin/build/generated.json`, and the build deletes a listed file it no longer produces, so removing a content file removes its page and its twin. ## Releasing 1. Bump `admin/build/version.txt` (vX.Y.Z, once per release) and add a row to `admin/content/admin/versions.md` with one sentence on what changed and why. 2. `python3 admin/build/build.py` 3. `node admin/build/validate.js` 4. `git commit -am "site vX.Y.Z: ..." && git push origin dev`. The `site vX.Y.Z:` prefix is load-bearing: the tag job finds the release commit by it and refuses a release whose newest prefix disagrees with `version.txt`. 5. Check that https://cli.sgit.ai/assets/version.js serves the new version. A clean push is not a release. ## The gate `node admin/build/validate.js` fails the build on any of these, in CI on every push and pull request, before anything is tagged or deployed: | Check | Why | |---|---| | The build is current (`build.py --check`) | a page edited by hand, or content committed without building, ships something the source does not say | | The version agrees: `version.txt`, `assets/version.js`, `llms.txt`, one row in the release history | the badge on every page reads `version.js`; a release without its row has no record | | Every internal link and asset resolves | an orphaned or misspelled link is a page nobody can reach | | Every page has a canonical link on the CNAME host, and a markdown twin, and is in `llms.txt` | pages agents cannot read, or that claim to live elsewhere | | No page is an orphan: every page is linked from another page | a page nothing links to is unpublished in practice | | The leak tripwire: vault-key shapes, private-key markers, cloud and provider key prefixes, across every file | this site discusses keys constantly; it must never contain one | | Advisory, not failing: em-dashes in prose on this site's own pages | the estate's writing rule; pages moved or published verbatim keep their authors' punctuation | ## Against the estate's guidance [Every sgit repository](https://sgit.ai/docs/guidance/repositories.html) lists what each repository carries. Where this one stands at v0.1.0: | The guidance asks for | Here | |---|---| | The brief, published in full | [yes](../briefs/index.md), with the initial prompt | | A reality file and a corrections file | [yes](../admin/reality.md), [yes](../admin/corrections.md) | | One version, up on every push, shown on every page | yes: `version.txt`, the badge, the [release history](../admin/versions.md) | | A gate that fails the build | yes, above | | Machine-readable twins and `llms.txt` | yes, generated from the same source | | Licences and a participant disclosure | yes: Apache-2.0 for code, CC BY 4.0 for content, [the disclosure](../about/participant.md) | | Nav and footer injected at build | yes, from one definition in `build.py` | | A credentials tier that is never committed | `.local/` is gitignored; nothing in this repository needs a secret to build | | The review folder (`review/`) | **not yet**, recorded in [the reality file](../admin/reality.md) | | Roles and a board, as files | **not yet**, recorded in the reality file | | A release script that verifies the live site | **not yet**: step 5 above is by hand | --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/admin/index.html)* # What moved from sgit.ai, cli.sgit.ai > The pages that moved from sgit.ai to cli.sgit.ai on 20 October 2026, each at the same path, and why: the CLI now has use cases of its own, and sgit.ai is the umbrella. *Source: · generated from the same content as the page, so the two cannot drift.* --- # What moved from sgit.ai On 20 October 2026 the CLI's most technical pages moved here from sgit.ai, as [the brief](../briefs/index.md) asks: the CLI now has a set of use cases of its own, and sgit.ai is the umbrella for the project and its commercial side. Each page kept its path, so `sgit.ai/docs/quickstart.html` is now `cli.sgit.ai/docs/quickstart.html`, and a redirect on sgit.ai is one rule. The pages were taken from sgit.ai's markdown twins, which are generated from the same source as its HTML, at sgit.ai site v0.7.87, by `admin/build/import_sgit_ai.py`. The text is unchanged. Links to a page that moved now stay on this site; links to anything else go back to sgit.ai. | Was on sgit.ai | Is now | Section here | |---|---|---| | | [Append lanes](../api/append-lanes.md) | HTTP API | | | [Authentication](../api/authentication.md) | HTTP API | | | [Errors and limits](../api/errors.md) | HTTP API | | | [The HTTP API](../api/index.md) | HTTP API | | | [The transfer API, one encrypted handover, one link](../api/transfers.md) | HTTP API | | | [Vault objects](../api/vault-objects.md) | HTTP API | | | [Agents sharing one vault](../docs/agents-sharing-one-vault.md) | Guides | | | [Working with AI agents](../docs/agents.md) | Guides | | | [Append-lane messaging between agents that hold no one else's vault key](../docs/append-lane-messaging.md) | Guides | | | [Vault credentials: what each one can do, and what its prefix declares](../docs/credentials.md) | Security | | | [History integrity: the format gate, verifying signatures, and rewinds](../docs/history-integrity.md) | Guides | | | [Using sgit from Claude on a Team or Enterprise plan: the network allowlist](../docs/how-to/claude-team-egress.md) | Guides | | | [Installation](../docs/installation.md) | Get started | | | [Vault key management: where each key lives, and how it travels](../docs/key-management.md) | Security | | | [When NOT to use sgit](../docs/limitations.md) | Get started | | | [Partial clones: a folder scope and a history depth](../docs/partial-clones.md) | Guides | | | [Keys, signatures and encrypting to someone else](../docs/pki.md) | Security | | | [Quickstart](../docs/quickstart.md) | Get started | | | [Self-hosting a vault server for agents](../docs/self-host-for-agents.md) | Guides | | | [Send a vault key to the registry](../docs/send-a-vault-key.md) | Guides | | | [sgit for git users](../docs/sgit-for-git-users.md) | Get started | | | [The two-branch model](../docs/two-branch-model.md) | Get started | | | [Update sgit-ai to 0.18.0](../docs/update-to-0-18-0.md) | Guides | | | [Update sgit-ai to 0.21.0](../docs/update-to-0-21-0.md) | Guides | | | [Sending messages between vaults](../docs/vault-messaging.md) | Guides | | | [What is sgit](../docs/what-is-sgit.md) | Get started | | | [Security model](../security/index.md) | Security | ## What stayed on sgit.ai, and why - **Vaults as a platform**: vault apps, `_page.json`, the browser bridge, sub-vaults, static hosting, publishing a vault, and the guidance for working on one. They are about the vault, not the command line. - **The project**: why it exists, use cases, case studies, the published vaults, comparisons, the newsroom, partnerships. - **The docs hub and the agent-contact protocol**, which serve the whole network. ## What sgit.ai still has to do This site cannot change sgit.ai. Until sgit.ai's team acts, both copies exist, and **the copy here is the one maintained from now on**. The request to them, as recorded in [the corrections file](../admin/corrections.md): 1. Point each moved page at its new address: a link at the top, then a redirect. 2. Point sgit.ai's docs hub and `llms.txt` at cli.sgit.ai for the CLI. 3. Add cli.sgit.ai to the network directory (`admin/content/sites/cli.md`, the Cartographer's file). --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/admin/moved.html)* # Reality, cli.sgit.ai > What exists on cli.sgit.ai today, what does not, and what is only proposed. If it is not here, it does not exist, whatever a brief says. *Source: · generated from the same content as the page, so the two cannot drift.* --- # Reality: what exists, at v0.1.2 (20 October 2026) The rule, from the SG/Send repository's Librarian: a feature that is not in this file does not exist, and nobody may say a thing works because a brief describes it. Kept current by whoever ships. ## Exists | | | |---|---| | **The site** | Static HTML generated by `admin/build/build.py` from `admin/content/`, deployed to GitHub Pages from `dev` by `.github/workflows/deploy-pages.yml`; CNAME `cli.sgit.ai` | | **27 pages moved from sgit.ai** | At their sgit.ai paths, text unchanged, from sgit.ai site v0.7.87; each states where it came from ([the list](../admin/moved.md)) | | **The command reference** | 33 pages, one per top-level command, covering all 158 commands of sgit-ai v0.23.0, generated from `sgit help all --format json`; the JSON is published beside it | | **23 research articles** | From `SGit-AI__CLI` at 5597526, `team/humans/dinis_cruz/claude-code-web/`, 2 to 16 October 2026, bodies as written | | **The brief** | Published in full, with the initial prompt | | **Machine-readable twins** | A `.md` twin for every page, `llms.txt`, `llms-full.txt`, `sitemap.xml`, all generated | | **The gate** | `admin/build/validate.js`: build freshness, version agreement, links, canonical host, twins, orphans, leak tripwire; runs in CI before tag and deploy | | **The live site** | https://cli.sgit.ai served v0.1.1 on 20 October 2026, checked against `assets/version.js` after the deploy; the github.io address redirects to it | | **Versioning** | `admin/build/version.txt`, badge on every page via `assets/version.js`, release history, tags `vX.Y.Z` made by CI | ## Does not exist yet | | | |---|---| | **Redirects on sgit.ai** | The moved pages still exist on sgit.ai, and nothing there points here yet. Requested in [the corrections file](../admin/corrections.md) | | **The review folder** | `review/` (intent from the brief, code derived from the tree, commits read upwards) is required by the estate's guidance from the first commit and is not built | | **Roles and a board** | No `team/` roles or board files yet | | **A release script** | Releasing is the four commands on [the engineering page](../admin/index.md); nothing polls the live site for the new version | | **Automatic reference updates** | The reference JSON is regenerated by hand per CLI release; nothing in the CLI's pipeline sends it here | | **An article pipeline from the CLI repository** | `import_cli_research.py` is run by hand, from a list | | **Search** | None. `llms.txt` and the docs hub are the indexes | ## Proposed, not built - The CLI's release pipeline writing `sgit help all --format json` into this repository, so the reference moves with every release. - Per-version reference pages (`/reference/v0.23.0/`), so a reader on an older CLI reads the surface they have. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/admin/reality.html)* # Release history, cli.sgit.ai > Every release of cli.sgit.ai, newest first, with one sentence on what changed and why. *Source: · generated from the same content as the page, so the two cannot drift.* --- # Release history The site's version is in `admin/build/version.txt` and goes up on every push to `dev`; the badge at the top of every page reads it. The command reference carries its own version, the sgit-ai release it was generated from. | Version | Date | Command reference | What changed, and why | |---|---|---|---| | v0.1.2 | 2026-10-20 | sgit-ai v0.23.0 | The reality file catches up: cli.sgit.ai was checked live after v0.1.1 deployed (DNS, Pages and the `dev` environment all work), so it moves from "does not exist yet" to "exists". | | v0.1.1 | 2026-10-20 | sgit-ai v0.23.0 | The generator and the gate reach the repository: the Python `.gitignore` template's `build/` rule had kept `admin/build/` out of the v0.1.0 commit, so CI could not run `validate.js` and nothing deployed. `.gitignore` now re-includes `admin/build/`. | | v0.1.0 | 2026-10-20 | sgit-ai v0.23.0 | The first version, built from [the brief](../briefs/index.md) the same day: 27 pages moved from sgit.ai (site v0.7.87) so the CLI's technical documentation has one home, the command reference generated from `sgit help all --format json` (158 commands), and 23 research documents from the CLI repository (`SGit-AI__CLI` at 5597526) published as articles, with the generator, the gate and the deploy pipeline the sibling sites run. | --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/admin/versions.html)* # Append lanes, cli.sgit.ai > The six append endpoints) the write-only vault-to-vault message transport. Four separated capabilities, the blind write response, server-assigned sortable filenames, idempotent mark-processed, and the limits. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # Append lanes A write-only channel attached to a vault, gated by a token the writer holds and nothing else. This is the transport behind [vault-to-vault messaging](../docs/vault-messaging.md) and behind [`sg.append`](https://sgit.ai/docs/vault/sg-bridge.html) in the browser bridge, and it is deliberately generic: the same primitive carries messages, logs, signals, control messages and state flows. **Naming.** This API was called **inbox** before v0.32.7. Every `/api/vault/inbox/*` URL is gone, and `purge` takes `folder:"pending"` or `"processed"`, the old `"inbox"` value returns **400**. If you are reading older material, translate. **Checked in use, 26 September 2026.** Two agent teams ran two-way, signed messaging over these lanes on 25 and 26 September, and wrote down every place where this page differed from what the server did. [Their write-up](../docs/append-lane-messaging.md) is published on this site. The table below, and the sections marked *checked 26 September*, are corrected from it. ## The six endpoints All **POST**, all under `/api/vault/append/`. | Path | Gate | Does | |---|---|---| | `configure/{vault_id}` | `x-sgraph-vault-write-key`**and** the SG/Send access token (401 without it) | Registers `append_anchors`, hashes of accepted senders, and `enum_key_hash`. **It replaces the anchor list**: to add a sender, send every existing anchor plus the new one, or the existing lanes silently stop accepting writes | | `write/{vault_id}` | `append_token`**in the body** | Appends a payload. **Account-less**: no access token required | | `list/{vault_id}` | `x-sgraph-vault-enum-key` | Paginated listing, optionally with inline content. Each entry's `inbox` is the lane's **raw token** (see below) | | `fetch/{vault_id}` | enum key | Body `{"inbox": , "file_ids": [...]}`: the lane must be named. **Pending files only**: once a file is marked processed it cannot be fetched again, so keep the ciphertext if you may need to re-verify it | | `mark-processed/{vault_id}` | enum key | Body `{"inbox": , "file_ids": [...]}`. Moves pending → processed. Idempotent | | `purge/{vault_id}` | `x-sgraph-vault-write-key` and the SG/Send access token (401 without it; checked 29 September) | Body `{"inbox": , "folder": "pending" | "processed"}`. Deletes | ## Where the routes are, and what a failure looks like checked 26 September - **The routes are on `https://dev.send.sgraph.ai` only.** `send.sgraph.ai` has no append routes today. - **A bad key or an unknown token returns 404 with an HTML page**, not a 403 with JSON. A missing field returns a JSON 400. When a call 404s, check the key before the path. - **Getting the write key.** It is derived from the vault key: `Vault__Crypto().derive_keys_from_vault_key(vault_key)['write_key']`. The parser strips the `sgit_private_vault_` prefix first; deriving from the raw string gives a different key, and every call 404s. ## The payload, exactly checked 26 September `sgit pki encrypt` writes the `.enc` file as base64 text of a JSON envelope. The `payload` sent to `write` is base64 of that file's bytes, so the envelope is **encoded twice**: ``` payload = base64(open("msg.eml.enc", "rb").read()) # sending envelope = json.loads(b64decode(b64decode(content))) # draining ``` A sender that encodes only once is rejected by a strict drainer. The recommendation to the sgit team is a single encoding, since the `.enc` text is already valid base64; until that changes, this is the format. ## Why the gates are split this way Four capabilities, and the split is the whole design: | Capability | Holder | Can | Cannot | |---|---|---|---| | `append_token` | the sender | write | list, fetch, read anything | | `enum_key` | the vault owner | list, fetch, mark-processed | write, purge | | `write_key` | the vault owner | configure, purge | none | | private key | the vault owner | **decrypt** | *never sent to the server* | A sender can put something into your vault and learn nothing at all, not the contents, not the volume, not whether anyone else writes there. The server stores `SHA-256` of the first three capabilities and compares hashes; the fourth it never sees. ## Four contracts worth relying on - **The write response is blind.** `/write` returns exactly `{"ok": true}`, no file ID, no count, no metadata. This is a security property, not an oversight: a returned ID would leak lane state to a party who is only allowed to write. - **Filenames are server-assigned** as `{epoch_ms:013d}_{24-hex}.enc`. Being chronologically sortable is what makes cursor pagination stable, pass the last file ID you saw as `after_file_id`. - **Metadata-only listing is free.** `include_content: false` reads **zero** payloads. Poll with it, then fetch on demand; polling with content is the common way to hit the 3 MB ceiling for no reason. - **mark-processed is idempotent.** An already-moved file comes back in `missing` rather than as an error, so retrying a batch after a timeout is safe. ## From a vault app: one verb crosses vaults, five do not This is the constraint to understand before designing anything that spans two vaults, and it was undocumented until the SG/API team's review of 6 September 2026 (quoted here CC BY 4.0). The bridge builds **one** client bound to the **currently open vault**, with the enum key derived from that vault's read key. So: | Verb | Which vault it acts on | Bridge permission | |---|---|---| | `write` | **Any**: it takes an explicit `vault_id` and posts with no headers at all | `append.write` | | `list` | Always the open vault. There is no way to address a remote vault with these from an app | `append.list` | | `fetch` | `append.read`, **not**`append.fetch` | | `markProcessed` | `append.markProcessed` | | `purge` | `append.purge` | | `configure` | `append.configure` | All six are plain booleans, **default-deny**, not path-scoped. The asymmetry is deliberate rather than a gap: listing a remote lane needs the recipient's enum key, and shipping that inside a published app *"would give every visitor read access to the whole lane."* **Two things that are *not* gates on `append`, despite appearances.** There is **no read-only check** anywhere in the append handler, `sg.app.writable` is irrelevant to it, and a read-key session can write to a lane given the grant. And the CSP is not an append rule: the frame ships `connect-src blob: data:`, so a *direct* `fetch` to these endpoints is blocked unless the app declares `permissions.network: true`, which reopens all egress and is the worse choice. Use the bridge. See [the build brief](https://sgit.ai/docs/briefs/vault-telemetry-append-lanes.html), which had both of these wrong until the review corrected it. ## The `inbox` field is the lane id, and today it is the token A `list` response labels each entry with an `inbox` value. That value is the storage folder name, and the folder is named by the **raw append token**, byte for byte, while `config.append_anchors` stores `sha256(token)`. So the design hashes the token in config and then writes the plaintext token into the object key. The disclosure is bounded: `list` is gated by the enum key, so it reaches only the vault owner, who already holds every credential in play; and the token is a **write-only lane address**, closer to an email address than a password. The residual risk is storage rather than API, the raw token becomes an object key, so it lands in access logs, inventory reports and backups, *"surfaces with a very different audience from the enum-key holder."* The team intends to fold the folder name to `sha256(token)`, which is a breaking change needing a migration. **Documented here now so that nobody has to discover it from a debrief.** Lanes live at `bare/append/{token}/pending/` and `…/processed/`, **outside the version-controlled commit tree**: appends never touch a branch and never conflict with a push. ## Several senders on one vault Register several `append_anchors` and each sender writes into their own lane, distinguished by their token. A listing can be scoped to one lane, so one correspondent flooding you does not bury another, and revoking one sender is removing one anchor, with no effect on the rest. ## Limits and codes | Limit | Value | On breach | |---|---|---| | Payload per write | 5 MB | 413 | | Pending files per token | 1000 | 507 | | File IDs per batch | 100 | 400 | | Inline content when listing | 3 MB cumulative | 413 | | Page size | 50 default, 200 max | clamped silently | The `append_token` pattern is `^[0-9a-f]{16,128}$`, hex only. A prefixed token returns **400**, not 403; see [errors](../api/errors.md) for why that distinction saves time. ## See also - [Vault messaging](../docs/vault-messaging.md), this API composed with PKI into a worked example - [Append-lane messaging between agents](../docs/append-lane-messaging.md), the write-up of two-way use on 25 and 26 September: per-session keys, a pinned key registry, ephemeral inboxes, the threat model and the recommendations - [`sg.append`](https://sgit.ai/docs/vault/sg-bridge.html), the same six operations from a vault app - [Errors and limits](../api/errors.md) --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/api/append-lanes.html)* # Authentication, cli.sgit.ai > The six authentication headers, what each one gates, and the hash-comparison model: the server stores SHA-256 of every capability key and never holds a raw key or a private key. Includes why vault reads are open by default. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # Authentication Six headers exist. Which apply depends on the endpoint and on how the server is deployed. None of them is a password, and none of them is stored: the server keeps `SHA-256` hashes and compares. ## The headers | Header | What it gates | |---|---| | `x-sgraph-access-token` | The SGraph account or deployment. In a **single-key self-hosted deployment** this gates *every* route, including reads | | `x-sgraph-vault-write-key` | Vault write capability: writes, deletes, destroy, and append `configure` / `purge` | | `x-sgraph-vault-enum-key` | Append lane enumeration: `list`, `fetch`, `mark-processed` | | `x-vault-read-key` | Read key, for public-vault routing | | `x-vault-public` | Public-vault routing flag | | `x-sgraph-transfer-delete-auth` | [Transfer deletion](../api/transfers.md), SG/Send transfers, not vaults. Carries the *secret*, never the hash, and only works if a `delete_auth_hash` was set at create time | One capability is **not** a header: `append_token` travels in the request body of `/append/write`. That is deliberate, see [append lanes](../api/append-lanes.md). ## The hash-comparison model The server never stores a capability key. It stores `SHA-256(key)` and, on each request, hashes what you presented and compares. Two consequences worth designing around: - **A server compromise yields hashes and ciphertext**, not keys and not plaintext. An attacker with the whole database still cannot read a vault. - **There is no recovery path.** The server cannot email you your key, because it has never had it. Same property as the vault key itself, for the same reason. ## Why reads are open by default `GET /api/vault/read/{vault_id}/{file_id}` requires no authentication on the shared host, which surprises people. It follows from the encryption model rather than being a gap in it: the bytes are AES-256-GCM ciphertext under a key the server has never seen, and the file IDs are HMAC-derived opaque identifiers that reveal no filename. Serving them to an anonymous caller discloses nothing a reader could use. It is also the property that makes a [published read key](https://sgit.ai/demos/vaults/index.html) work at all, and what lets vault content sit behind an ordinary CDN. If that trade is wrong for your deployment, run in single-key mode, where the access token gates reads too. What an unauthenticated reader *can* learn is covered honestly on the [security page](../security/index.md#server-sees): object sizes, timing, and approximate activity volume. ## See also - [Errors](../api/errors.md), what a 403 means as against a 400 - [Security model](../security/index.md), the crypto these capabilities sit on --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/api/authentication.html)* # Errors and limits, cli.sgit.ai > 400, 403, 413 and 507: what each means, why a malformed token returns 400 rather than 403, the input patterns enforced before storage is reached, and every documented ceiling. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # Errors and limits Four status codes carry almost all the failures, and telling them apart saves real debugging time, particularly 400 versus 403, which fail for opposite reasons. ## Status codes | Code | Means | What to do | |---|---|---| | **400** | Invalid input, the value never reached a gate. Malformed token, malformed file ID, batch too large, unknown `folder` value | Fix the *shape* of what you sent. Your credential may be perfectly valid | | **403** | Gate failure, well-formed input, wrong capability | Fix the *credential*. The request shape was fine | | **413** | Too large, payload over 5 MB, or an inline-content listing over the 3 MB ceiling | Split the payload, or list without inline content and fetch on demand | | **507** | Lane full, 1000 pending files for that token | Fetch and `mark-processed`, then `purge` the processed folder | **The 400 that looks like a 403.** Path and token components are validated by typed primitives *before* they reach storage, which is how path traversal is blocked at the type level. A prefixed token such as `tok_abc…` (or a CLI fingerprint pasted straight in, `sha256:a461…`) fails the hex-only pattern and returns **400**. It reads like a rejected credential; it is a rejected string. Use raw hex. ## Input patterns | Input | Pattern | |---|---| | `append_token` | `^[0-9a-f]{16,128}$`: hex only, no prefix | | `file_id` (append) | `^\d{13}_[0-9a-f]{24}\.enc$`) server-assigned, so you should be echoing one back | | `vault_id` | lowercase alphanumeric, 8–24 characters, no hyphens | ## Limits | Limit | Value | On breach | |---|---|---| | Payload per append write | 5 MB | 413 | | Pending files per append token | 1000 | 507 | | File IDs per batch (fetch / mark / purge) | 100 | 400 | | Operations per `/vault/batch` request | 100 | 400 | | Inline content when listing | 3 MB cumulative | 413 | | Append list page size | 50 default, 200 max | clamped silently | | `read-base64` response | ~3.75 MB | use [presigned reads](../api/vault-objects.md) above this | The page-size clamp is silent: ask for 500 and you get 200 with no warning. Page with `after_file_id` rather than assuming your requested size was honoured. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/api/errors.html)* # The HTTP API, cli.sgit.ai > The protocol surface behind sgit: base URL, the capability model, and the reference for vault objects, append lanes, authentication headers and error codes. Built from a code-verified audit at v0.33.54, with unresolved endpoints labelled rather than guessed at. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # The HTTP API Everything sgit does, it does over this API. The CLI is a client; so is the browser bridge; so is anything you write. This section is the protocol surface, endpoints, headers, gates, limits and the error codes you will actually hit. **Two families share this host**: [vaults](../api/vault-objects.md), which are versioned trees you clone and push, and [transfers](../api/transfers.md), which are single encrypted handovers. **Where this came from, and how far to trust it.** This section is built from a code-verified audit supplied by the SG/API team, read out of the route tables and constants at **v0.33.54** rather than recalled. It is *their* verification, not ours: this site has not independently exercised every endpoint. Where their audit and the shipped client disagreed we checked and said so; where something is registered but unconfirmed it is labelled **PROPOSED** and you should not build on it yet. Same rule as the rest of this site, if it is not verified, it says so. ## Base URL `https://send.sgraph.ai` for the hosted service, or your own host if you [run it yourself](https://sgit.ai/deploy/index.html). All request and response bodies are JSON unless stated otherwise. ## The one idea worth reading first The server is a **capability-checked ciphertext store**, and every access-control decision it makes is a hash comparison. It stores `SHA-256` of each capability key and checks `H(presented) == stored`. It never holds a raw key, and it never holds a private key at all. That is what makes the rest of the design possible: a vault ID is an address rather than an account, reads are public by default because the bytes are useless without a key, and a stranger can write to your vault without ever having a credential on the platform. ## The reference | Page | What is in it | |---|---| | [Authentication](../api/authentication.md) | The six headers, what each gates, and the hash-comparison model | | [Vault objects](../api/vault-objects.md) | The pointer store (read, write, batch, destroy) plus the caching contract and storage layout | | [Append lanes](../api/append-lanes.md) | The six append endpoints: the vault-to-vault message transport | | [Transfers](../api/transfers.md) | The other family on this host: one encrypted payload, uploaded once, handed over as a single link, the SGMETA envelope, the three calls, and when to reach for this instead of a vault | | [Errors and limits](../api/errors.md) | 400, 403, 413, 507, what each actually means, and the ceilings that produce them | ## Four capabilities Most of the API surface is one of four capability tiers. They are separated so that holding one tells you nothing about the others: | Capability | Presented as | Grants | |---|---|---| | `append_token` | a field in the request body | Write to one append lane. Nothing else, not even listing it | | `enum_key` | `x-sgraph-vault-enum-key` | List, fetch and mark-processed on append lanes | | `write_key` | `x-sgraph-vault-write-key` | Write and delete objects; configure and purge lanes | | private key | *never presented* | Decryption, client-side only | The fourth row is the point of the design rather than a footnote: the server cannot decrypt anything it stores, so the strongest capability in the system is the one it never sees. ## Where to start - Sending an encrypted message between two vaults → [Vault messaging](../docs/vault-messaging.md), which composes [append lanes](../api/append-lanes.md) with [PKI](../docs/pki.md). - Reading or writing files → [Vault objects](../api/vault-objects.md). - Building a vault app in the browser → the [SG bridge](https://sgit.ai/docs/vault/sg-bridge.html) wraps this API so you never write HTTP. - Using the CLI instead → [Quickstart](../docs/quickstart.md). ## Open questions we will not guess at Three items came through the audit unresolved. They are listed rather than documented, because a confident page about an endpoint that does not exist is worse than no page: | Item | Status | |---|---| | `GET /api/vault/zip/{vault_id}` | **UNRESOLVED**: registered in the route table, marked PROPOSED in the team’s reality document. Not documented either way until that is reconciled | | `/join/*` | **UNRESOLVED**: the reality document lists three live endpoints, but the routes are not registered in the deployed app. Not documented | | Client-side lane addressing | **PROPOSED**: `append_token = H(public key)` is the intended model; no shipped command emits it. See the [addressing note](../docs/vault-messaging.md#addressing) for what to do meanwhile | --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/api/index.html)* # The transfer API, one encrypted handover, one link, cli.sgit.ai > The other API family on this host: a single encrypted payload uploaded once and shared as one link. The two secrets that must not be confused, the SGMETA envelope that keeps the filename off the server, the three calls, revocation that is opt-in at create time only, and when a transfer beats a vault. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # The transfer API, one encrypted handover, one link The other half of the API on this host. A **vault** is a versioned encrypted tree you clone, pull and push; a **transfer** is a single encrypted payload uploaded once and handed over as one link. Same server, same zero-knowledge property, different job, and until now this site documented a transfer *header* without documenting a single transfer *endpoint*. **Provenance.** This page is derived from an integration guide written by the team that owns the SG/Send API, who executed **every request, response, status code and byte layout in it against the production user API on 9 September 2026**. That run is theirs, not ours. We have not re-executed it here, and the date is the one to check it against. Their guide is released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). ## Transfer or vault? Pick this first; it decides everything after it. | | Transfer | [Vault](../api/vault-objects.md) | |---|---|---| | **Shape** | One payload, uploaded once | A versioned tree of files | | **History** | None. There is no second version | Commits, branches, merges | | **The reader gets** | The whole payload, or nothing | Any single file, without fetching the rest | | **Lifecycle** | Expires, caps downloads, can be revoked | Lives until you delete it | | **The credential** | A decryption key in the URL fragment | A [read key](https://sgit.ai/docs/vault/reading-a-vault-file.html), derived one-way | | **Reach for it when** | An agent finishes a job and hands the result over **once**: a bundle, a report, a dated export | The thing is read repeatedly, browsed, updated, or rendered as an app | The SG/API team's own guidance, for a workflow that ships one bundle per voice memo, is that the transfer API is *“the simpler and better fit”*. That is the right instinct in general: **a vault is the wrong answer for a handover that happens once.** ## Two secrets that must never be confused This is the part that goes wrong first, and it goes wrong silently, a leaked access key does not throw. | | What it authorises | Where it goes | |---|---|---| | **The access key** `x-sgraph-access-token` | *You*, to upload | A request **header**. Never a URL, never a share link, never a log | | **The decryption key** | *Anyone*, to read | The URL **fragment**: everything after `#`, which no browser sends to the server | They are unrelated values. The fragment is what makes a share link both shareable and private: the key reaches the recipient's client and never the origin serving the page. The API also accepts `?access_token=` as a query fallback and it works, **do not use it**; query strings land in proxy logs, browser history and workflow execution records. What the server can see: ciphertext size, a hashed sender IP, a creation timestamp, and the `content_type_hint` you supply. What it cannot see: the filename, the contents, or the key. ## The payload format, two layers, in this order The filename must not reach the server, so it is wrapped *inside* the plaintext before encryption. Getting the order wrong produces a corrupt download with no name. **Inner, the SGMETA envelope:** ``` "SGMETA\0" 7 bytes 53 47 4D 45 54 41 00 meta_len 4 bytes uint32, big-endian metadata JSON meta_len {"filename":"…"} UTF-8 file bytes rest your payload ``` **Outer, AES-256-GCM over that whole buffer:** ``` IV 12 bytes random, fresh per upload, prepended ciphertext + tag rest GCM appends its own 16-byte tag ``` - **Key:** 32 random bytes, shared as **base64url, unpadded**: 43 characters. - **IV:** 12 random bytes, never reused with the same key. - **Overhead:** the uploaded payload is exactly **28 bytes larger** than the plaintext (12 + 16). Same primitive as everything else here: AES-256-GCM, done client-side, with the server holding ciphertext under a key it never had. [The security page](../security/index.md) covers why that is the whole product rather than a feature of it. ## The three calls | Call | Endpoint | Notes | |---|---|---| | **1 · create** | `POST /api/transfers/create` | Returns `transfer_id`. Body carries `file_size_bytes` (the *plaintext* length, display metadata only), `content_type_hint` (stored in the clear, keep it generic), `max_downloads`, `auto_delete`, `expires_at` | | **2 · upload** | `POST /api/transfers/upload/{id}` | **Raw bytes**: not base64, not multipart. `Content-Type: application/octet-stream` | | **3 · complete** | `POST /api/transfers/complete/{id}` | Nothing is downloadable until this call lands | Only those three need the access key. `info` and `download` are **public by design**: the decryption key is the gate, which is exactly why a share link works for someone with no account. **Two traps in the create body.** `expires_at` is in **milliseconds** since epoch, not seconds; `0` means never. And `max_downloads: 0` means *unlimited*, not zero, once a finite allowance is exhausted, downloads return **410**. ### Build the share link yourself `complete` returns a `download_url` of `/d/{id}`. **Ignore it**: verified as a 404; it is not a live route. The link to hand out is: ``` https://send.sgraph.ai/en-gb/download/#{transfer_id}/{key_base64url} ``` Omit `/{key}` for a link-only form when the key travels through a second channel. The fragment parser splits on the **first** `/` and truncates at a `|`, so `|` is reserved, base64url never produces one, so following the format above keeps you clear. ## What a receiving agent has to do This is the payoff, and it is two calls with no account and no browser: `GET /api/transfers/download/{id}` needs no auth, and the key from the fragment does the rest, split the IV off the first 12 bytes, AES-GCM decrypt, check the `SGMETA\0` magic, read the big-endian length, and the filename and payload fall out. For a JSON-only transport such as an MCP tool, `GET /api/transfers/download-base64/{id}` returns the same bytes wrapped in JSON. **Downloads are counted.** An agent retrying a failed run burns the allowance. For agent-facing links prefer `max_downloads: 0` with a short `expires_at`, time-bounded rather than attempt-bounded, because a retry is not a reader. ## Expiry, caps and revocation Revocation is opt-in **at create time and only then**: set `delete_auth_hash` to `sha256(secret)` in hex and store the secret beside the transfer id. Without it, deletion is disabled for that transfer forever. To revoke, `DELETE /api/transfers/delete/{id}` carrying the access key and `x-sgraph-transfer-delete-auth: `, the one header [the authentication page](../api/authentication.md) already listed before this page existed to explain it. **410 and 404 are both normal end states, not errors to retry:** 410 is gone-as-expected (expired or exhausted), 404 is never-existed-or-deleted. ## Size | Path | Ceiling | When | |---|---|---| | **Direct upload** | keep the encrypted payload under **~4 MB** | The normal case | | Multipart / presigned | Larger | Needs the S3 backend, check `GET /api/presigned/capabilities` first | The ~4 MB figure is a working limit rather than a published one: the Lambda URL caps a request body at 6 MB and the browser client switches to presigned at 5 MB. Text bundles compress far below it; **audio does not**, and is usually the most sensitive part of a bundle as well as the largest. ## Verified behaviour The SG/API team's run, 9 September 2026, reproduced here as they reported it: | Case | Result | |---|---| | Header `x-sgraph-access-token` | **200** | | Header `x-sgraph-send-access-token` | **401**: that name does not work | | No token, or a wrong one | **401** | | `download` / `info` with no token | **200**: public by design | | Round trip: wrap → encrypt → create → upload → complete → download → decrypt → unwrap | **Byte-identical** payload recovered, filename intact | | Download #3 against `max_downloads: 2` | **410** | | Download of an expired transfer | **410** | | `DELETE` with wrong / correct delete-auth | **403** / **200**, then **404** | | `GET /d/{transfer_id}` | **404**: not a route | | Key length | 32 bytes → 43 base64url characters | | Encrypted payload overhead | plaintext + 28 bytes | ## The rules worth holding to 1. **Never log or return the decryption key** outside the share link. It is not an identifier, anyone holding it reads the payload. In a workflow engine that means keeping it out of node output that gets persisted. 2. **Never put the access key in a URL.** Header only. 3. **Never reuse an IV.** Twelve fresh random bytes per upload. 4. **Keep the filename inside the envelope.** `content_type_hint` is in the clear; the name is not, and that is the reason the envelope exists. 5. **Always set an expiry.** A link pasted into a chat lives as long as the chat does; the transfer should not. 6. **Bundle for a reader who is an agent**: a `manifest.json` saying what each file is, and an `llms.txt`, cost nothing and are the difference between a payload that can be used and one that has to be opened by a person first. Adapted with attribution from the SG/API team's integration guide of 9 September 2026 (CC BY 4.0); the workflow-specific half of that document (credential wiring, node shapes, the bundle layout for one product) stays with the product it was written for. [API reference](../api/index.md) · [Vault objects](../api/vault-objects.md) · [Append lanes](../api/append-lanes.md) · [Working on a vault](https://sgit.ai/docs/guidance/index.html) --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/api/transfers.html)* # Vault objects, cli.sgit.ai > The pointer store: read, write, batch, destroy and presigned endpoints, the caching contract that separates immutable content-addressed objects from mutable refs, and the storage layout behind every vault. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # Vault objects The pointer store: the ciphertext objects a vault is made of, addressed by opaque identifiers the server cannot interpret. This is what `sgit push` and `sgit pull` are doing underneath. ## Endpoints | Method | Path | Auth | Notes | |---|---|---|---| | PUT | `/api/vault/write/{vault_id}/{file_id}` | access token + write key | Body is raw ciphertext | | GET | `/api/vault/read/{vault_id}/{file_id}` | none | Raw bytes. [Reads are public by default](../api/authentication.md#reads) on the shared host | | GET | `/api/vault/read-base64/{vault_id}/{file_id}` | none | Base64 in JSON, MCP-safe. Around 3.75 MB ceiling | | DELETE | `/api/vault/delete/{vault_id}/{file_id}` | access token + write key | | | POST | `/api/vault/batch/{vault_id}` | per operation | Mixed reads and writes, **max 100** per request | | POST | `/api/vault/list/{vault_id}` | none | | | GET | `/api/vault/health/{vault_id}` | none | | | GET | `/api/vault/public-info/{vault_id}` | none | | | DELETE | `/api/vault/destroy/{vault_id}` | write key | Body `{vault_id, purge?}`. Without `purge` it writes a tombstone that blocks reuse of the ID | Large blobs go through presigned URLs rather than the request body: `POST /api/vault/presigned/initiate|complete|cancel/{vault_id}` and `GET /api/vault/presigned/read-url/{vault_id}/{file_id}`. `GET /api/vault/zip/{vault_id}` is registered in the route table but marked PROPOSED upstream. It is deliberately [not documented](../api/index.md#unresolved) until that is reconciled. ## Destroy, and why the tombstone matters `destroy` without `purge:true` leaves a tombstone that prevents the vault ID being registered again. That is a security property, not bookkeeping: vault IDs are short and a released ID is a squattable namespace. If somebody still holds a read key pointing at that ID, a new vault claiming it would be answering reads meant for the old one. `purge:true` skips the tombstone and frees the ID, use it only when you are sure no credential survives. ## The caching contract Anyone writing a client needs this, and getting it wrong produces a bug that looks like corruption: | Object | Response header | |---|---| | File ID containing `-imm-`, immutable, content-addressed | `Cache-Control: public, max-age=31536000, immutable` | | Refs and indexes, mutable | `Cache-Control: no-store` | Immutable objects are content-addressed, so their bytes can never change and a year-long cache is safe, which is what lets a vault sit behind a CDN. Mutable refs must never be cached: a stale ref renders a previous commit’s tree from ciphertext that is itself perfectly valid, so nothing errors and the reader simply sees the wrong version of the vault. **Measured 21 September 2026: the immutable header is not currently being sent.** A GET of an `obj-cas-imm-` object on `/api/vault/read/` returned **no `Cache-Control` header at all**, and the CDN reported a miss. The table above is the contract, not a description of what is served on that path today. Build as though the header may be absent: cache on the id, which is a hash of the ciphertext and therefore cannot be stale, rather than on a header you were sent. That is what the client-side caches in this estate already do, and [why caching ciphertext is the easy case](https://sgit.ai/demos/fractal-graphs/performance.html#caching). ## Storage layout ``` bare/data/obj-cas-imm-{hex} blobs, trees, commits — content-addressed, immutable bare/keys/key-rnd-imm-{hex} PKI public keys — immutable bare/refs/ branch refs — mutable bare/indexes/ branch index — mutable bare/append/ append lanes — managed, outside the commit DAG ``` The prefixes are load-bearing. `obj-cas-imm-` is SHA-256 over the *ciphertext*, which gives deduplication without the server knowing any plaintext, and makes the caching rule above derivable from the ID alone. Append lanes sit outside the commit DAG on purpose, a message arriving does not rewrite your history. This site is itself served from a vault, so these are the objects behind the page you are reading. The [live-vault case study](https://sgit.ai/case-studies/live-vault-docs.html) shows the fetch sequence. ## See also - [Append lanes](../api/append-lanes.md), the write-only channel that does not touch the commit tree - [Errors and limits](../api/errors.md) --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/api/vault-objects.html)* # Branches check, test performance, test quality — 8 Oct 2026, cli.sgit.ai > An end-to-end check found that branches did not work. What was fixed behind it, the tests that now cover it, and a review of the test suite's speed and quality. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Branches check, test performance, test quality — 8 Oct 2026 *CLI team. Done as part of the 0.21.0 release.* ## 1. Branches: broken, now fixed and covered An end-to-end check (create, list, switch in and out, work on a branch, push to it, pull from it, merge either way) found that **branches did not work**: push, pull and fetch looked up the named branch called `current` by name, so work committed on `feature` was pushed to main and the feature ref never reached the server. Fixed, together with everything the check turned up behind it (first push of a new branch, signing keys of new clone branches, switch dropping unpushed commits and tripping a false rewind, switch not fetching, duplicate names, `--from` not checking out, no way to merge a branch) and one bug that was not about branches at all: **every merge commit a pull created was unsigned**, because the pull state carried the key id in a type that turns `-` into `_`. | | Where | Tests | |---|---|---| | Unit, two real clones on the in-memory API | `tests/unit/review/test_Branches__End_To_End.py` | 22 | | Integration, real SG/Send server | `tests/integration/test_Branches_Tags_Integrity__Integration.py` | 5 (branches, tags, format 2, signatures-required, rewind + lease) | | Live dev API, through the CLI | manual run, two clones | create, push, switch, two people on one branch, diverged merge into main | Why the existing tests missed it: the step tests for pull, push and fetch run against fake branch managers whose `get_branch_by_name` returns the main branch whatever is asked; they could not see work going to the wrong branch. The end-to-end tests run the real objects. ## 2. Performance: 70 s → 45 s, all 4,042 unit tests | Change | Effect | |---|---| | Test vaults use a fixed key (`TEST_VAULT_KEY`) instead of a random one | Deriving a vault's keys is two PBKDF2 runs, ~190 ms by design. With one key, the process-level KDF cache makes every test vault after the first per worker nearly free. Each test still has its own directories and its own in-memory server | | Test HTTP servers poll every 20 ms (static server 100 ms) | `shutdown()` waited up to 500 ms per test; 28 tests | | Unit tests no longer reach the internet | 20 tests waited on the live dev server | Checked and left: xdist scheduling (`load`/`loadscope`/`loadfile` within 2 s of each other), collection (4 s), the vault-move test (22 real PBKDF2 runs is what a move does), two timing tests whose subject is the timer itself (retry back-off, idle-socket age). ## 3. Quality **Integration tests run real things.** All 12 files start the real SG/Send User Lambda on a local port (in-memory storage) and nothing in them is mocked. They were all failing locally: `mcp` 2.x breaks `fastapi-mcp` 0.4.0, which the server mounts at startup; CI pins `mcp<2`, the local setup did not. Fixed in CLAUDE.md. 68 integration tests pass in 18 s. **Unit tests are hermetic now.** `tests/unit/conftest.py` sends the default server to a closed local port and fails, by name, any unit test that opens a connection to a non-loopback host. Twenty were doing so: CLI commands on test vaults without a remote fell back to the live dev server, one prompt test only passed because the live server answered, and `push`'s local dirty check built a network client with no server. Production fixes for the last two. **Where the mocking is.** It is concentrated, and it is the part of the suite that hid the branch bug: | | Count | |---|---| | unit test files | 335 (4,042 tests) | | files that monkeypatch sgit_ai internals | 20 (258 patches) | | files with hand-written Fake classes | 4 (37 classes) | | tests named after a source line number | 82 | Almost all of it is in files named `*__Coverage*`, written to reach lines rather than to check behaviour. Recommendation, in order: 1. For new work, prefer the pattern the review tests use: real clones on the in-memory API (`Vault__Test_Env`), built once per class, restored per test (5 ms). 2. Replace the fake-based step tests (`test_Step__Pull/Push/Fetch__Coverage.py`) with workflow tests on real clones; delete the line-number tests whose behaviour another test already covers. 3. Treat a monkeypatch of an sgit_ai method in a unit test as a review flag: it usually means the code needs a seam (a parameter, a field) instead. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/branches-and-test-suite-review.html)* # Clone performance — the tree walk re-fetched shared sub-trees once per parent, cli.sgit.ai > A clone stuck walking trees: the walk re-fetched shared sub-trees once per parent, 10,098 requests for 2,236 unique trees. The instrumented run and the fix. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Clone performance — the tree walk re-fetched shared sub-trees once per parent **Date:** 2026-10-02 **Branch:** `claude/charming-babbage-5hzcxq` (from `dev` @ `0c9be4b`, v0.16.3) **Vault probed:** `y8pwtjlw` (the DC collaboration vault, 172 commits at the time of the run) **Reported with:** sgit-ai v0.14.27, stuck at `▸ Walking trees: obj-cas-imm-4c9e63979601` ## 1. What the clone was doing An instrumented run of the real clone steps (derive keys → index → branch meta → walk commits → walk trees) against the live vault, with the BFS tree walk mirrored exactly and every batch request timed: | phase | time | requests | notes | |------------------|--------|----------|-------------------------------------------------------------| | walk commits | 32 s | ~100 | linear chain: one commit per round trip | | walk trees | 377 s | 205 | 10,098 tree ids requested for **2,236 unique** trees | | (blobs, not run) | ~15 s | ~52 | 2,563 small blobs, fetched 8 chunks in parallel | The per-level picture of the tree walk is the whole story: | level | queue | unique | duplication | requests | download | |-------|-------|--------|-------------|----------|----------| | 1 | 170 | 170 | 1.0× | 4 | 7 s | | 2 | 2263 | 269 | **8.4×** | 46 | 86 s | | 3 | 1938 | 508 | 3.8× | 39 | 73 s | | 4 | 3561 | 811 | 4.4× | 72 | 132 s | | 5 | 2135 | 460 | 4.6× | 43 | 75 s | | 6 | 31 | 18 | 1.7× | 1 | 1 s | Two defects compound: 1. **`Vault__Graph_Walk.walk_trees` queued a shared sub-tree once per parent.** `next_q` was only guarded by `sub not in visited`, and `visited` is updated at *processing* time, so within a level every root tree that pointed at the same unchanged folder enqueued its own copy. A 170-commit history whose root trees share most of their folders made the next level ~8× larger than it needed to be, and all of those duplicates went to `on_batch_missing` and so to the server. Nothing was written twice (the per-tree `visited` guard still held), but 4.5× the bytes and requests went over the wire. 2. **`Vault__API.batch_read` fetched its 50-id chunks one after another.** Each Lambda round trip costs ~1.7 s for 50 objects, so a 3,561-id level became 72 serial requests (132 s) during which the only progress line is the last *decrypted* tree id — which is why it looked hung on `obj-cas-imm-4c9e63979601`. The blob download step already fanned out 8 chunks in parallel; the tree walk did not. The commit walk is a third, smaller cost: a linear chain needs one round trip per commit because the parent id is inside the ciphertext. ~0.19 s per request × 172. ## 2. The fix (this branch) - `Vault__Graph_Walk.walk_trees` keeps a `seen` set (queued ∪ visited) so every tree id is enqueued — and requested — exactly once. `on_batch_missing` still gets one call per BFS level. Pull's tree walk uses the same class and benefits identically. - `Vault__API.batch_read` fans its chunks out over a bounded pool (`BATCH_READ_WORKERS = 8`, matching the blob fan-out); each chunk keeps its own 502 → per-file → presigned-S3 fallback, and the first chunk failure still propagates as before. Single-chunk calls stay inline. `Vault__API__Static` already did this; `Vault__API__In_Memory` is unaffected. - `Step__Clone__Walk_Trees` / `__Head_Only` emit `Walking trees: fetching N tree(s)` before each level's download so a large level is visible instead of looking stuck. Measured on the same vault after the change, full `sgit clone`: | phase | before (probe) | after | |----------|--------------------|--------------| | commits | 32 s | 32 s | | trees | 377 s | **24 s** | | blobs | ~15 s | 14 s | | checkout | – | 0.4 s | | **wall** | **~7–8 min** | **73 s** | `sgit check fsck` on the result reports 17 missing objects — all blobs, all also absent on the server (`not_found` on a direct read), referenced by trees in commits `7b72ec534ca6`, `3cd27e9b3222`, `ff4cc8df75e8`, `81c7fd7c6cb4`, `68797954d2ef` (several are `*.conflict` files and `baseline.json`). That is a push from some client that never uploaded those blobs; it is pre-existing and unrelated to the walker (the old walker's probe store lacks them too). Worth noting: the blob step prints `2586 blobs` and does not say that 17 came back empty — see §4. Tests: 3 new walker tests (shared sub-tree requested once across 100 roots, duplicate roots, one call per level) and 4 new `batch_read` tests (chunking + all returned, single chunk inline, chunk failure propagates, 502 fallback confined to the failing chunk). `pytest tests/unit/ -n auto` → **3841 passed**. ## 3. Would the newer releases have helped? No. PyPI's latest is **0.16.0** (published 2026-08-20 from the last `dev → main` merge); `dev` is at v0.16.3 but nothing after 0.16.0 has reached PyPI because publishing runs on `main` only (`ci-pipeline__main.yml` sets `should_publish_pypi: true`; the dev pipeline had publishing removed on 2026-08-14). Between 0.14.27 and today the walk code changed once — the verify-before-write hook on 2026-08-20 — which does not touch the algorithm. `Vault__Graph_Walk` itself was last edited 2026-05-11. So 0.16.0 clones this vault exactly as slowly as 0.14.27; the fix needs a release that includes this branch. The one `dev`-only change that *is* relevant to the follow-up question is `sgit vault move` rewriting object ids (2026-08-20/21), which is also not on PyPI. ## 4. Follow-ups (not done here) - **Commit walk round trips.** 172 serial requests at ~0.19 s. `curl` timing shows ~0.11 s of each ~0.43 s request is the TLS handshake (a reused connection answers in 0.16 s). `urllib.request.urlopen` opens a fresh connection per call; a per-thread persistent `http.client.HTTPSConnection` in `Vault__API._request` would roughly halve the commit walk and shave the tree/blob phases too. Needs care with proxies (`HTTPS_PROXY` is honoured by urllib today) and the retry path. - **Silent missing blobs on clone.** `_download_blobs_by_id` drops `None` payloads without a word; the clone reports success and only `fsck` reveals the gap. A one-line `warning` with the count (and the first few file names, which the tree entries give) would have told the user about the 17 absent blobs at clone time. - **`sgit check fsck` has the same per-commit re-walk**: it reported `44256 trees` for a vault with 2,264 unique trees because it restarts the tree walk from each commit with a fresh `visited_trees`. Local disk only, so it is seconds not minutes, but it scales with commits × tree size. - **Full-clone shortcut**: a non-sparse clone downloads every reachable object anyway, so `list_files('bare/data/')` + one parallel bulk fetch + a local walk would collapse all three phases into one fan-out. It changes semantics (unreachable objects would be fetched too), so it is a design decision, not a patch. ## 5. Rekey while keeping history — where things stand The brief's todo: *"a rekey loses the entire commit history"*. That is true of `sgit vault rekey` (the `wipe` → `init` → `commit` flow: it deletes `.sg_vault/` and starts a fresh history from the working tree). It is **not** true of `sgit vault move` on `dev`: - `Step__Move__Build_Temp_Vault._rewrite_object_graph` classifies every object by reachability from the refs, then re-encrypts bottom-up — blobs, then trees with their `blob_id`/`tree_id` remapped and names re-encrypted deterministically, then commits with `tree_id` and `parents` remapped — storing each at its recomputed content address. Unreachable objects are carried too. Refs and the branch index are re-pointed. - Verified locally on a throwaway vault (`init` + 3 commits): after the rewrite the new store under the new read key resolves the clone ref to a 4-commit chain (`commit 3 → commit 2 → commit 1 → init`) with the right file names at each step, 11 objects in, 11 out, and **zero** shared ids between old and new store. Only the local build step was run (no access token in this session, so push/verify/delete-source were not exercised against the server). So "rekey keeping history" already exists as `sgit vault move --new-key `; the hashes change (as expected) but the chain, messages and trees all survive. What it does *not* do, and what the user may actually want: - **Keep the same vault id.** `move` derives a new vault id from the new key, so it is a fork-and-delete, not an in-place rotation. Sharing links and `.sg_vault/local` configs on every clone go stale. An in-place variant would need the server to accept a full object-set replacement under the same id (write new objects, swap refs/index, delete old), and the read-key → ref-id derivation (`derive_ref_file_id(read_key, vault_id)`) means the named ref id changes with the key even if the vault id does not. - **Ship it.** It is on `dev` only; PyPI 0.16.0 still has the old in-place re-encrypt that keeps ids (and therefore produces vaults that the new strict verify refuses). - **`sgit vault rekey` should either delegate to `move` or say plainly that it discards history**, since today the two commands overlap in name and differ exactly on the point the brief cares about. A full-chain check of `move` against the server (push → verify → clone the new key → `history log` shows all commits) is the next thing to run, with a token. --- ## 6. Follow-up (same day): keep-alive connections — done, with a measurement caveat Dinis asked for the connection-reuse follow-up without a new dependency. Implemented as `sgit_ai/network/api/Vault__HTTP_Pool.py` (stdlib `http.client`) with `Vault__API._send` on top; `_request` / `_request_bytes` keep their signatures and retry loop. **Rules, as agreed, each with a test against a real local HTTP/1.1 keep-alive server (`tests/unit/network/api/test_Vault__HTTP_Pool.py`, 21 tests):** | rule | test | |---|---| | sequential calls share one socket; headers still sent per request; no `Connection: close` | `test_sequential_requests_share_one_connection`, `test_headers_still_sent_per_request` | | server `Connection: close` honoured | `test_server_connection_close_is_honoured` | | stale keep-alive → resend once, **reads only** (GET + batch read) | `test_stale_keep_alive_is_retried_for_idempotent_reads`, `test_batch_read_is_treated_as_idempotent` | | stale keep-alive on a write → `URLError`, never replayed, pool recovers | `test_stale_keep_alive_never_resends_a_write` | | failure on a *fresh* connection is not retried; half-read body poisons nothing | `test_fresh_connection_failure_is_not_retried` | | same error text as before, token masked, connection reusable after a 4xx | `test_http_error_shape_and_body_preserved` | | 502/503 retry loop unchanged, on the same connection | `test_transient_502_retries_then_succeeds_on_same_connection`, `…_exhausted…` | | redirects not followed (token never replayed to another host) | `test_redirects_are_not_followed` | | 8 threads × 5 calls → bounded pool, no errors | `test_parallel_callers_share_a_bounded_pool` | | key = scheme/host/port/verify/proxy; `HTTPS_PROXY` → CONNECT tunnel with Basic auth; `NO_PROXY`; plain-http proxy → absolute URI | `Test_Vault__HTTP_Pool__Keys` | | `SGIT_HTTP_NO_KEEPALIVE=1` → one connection per request (A/B switch) | `test_env_switch_disables_keep_alive` | The pool is shared with a lock rather than thread-local on purpose: `batch_read` and the blob download create a *new* `ThreadPoolExecutor` per call, so thread-local connections would be discarded after every BFS level. Other details: `User-Agent: sgit-ai` (the layer-import test forbids `network` importing `_version`, so no version in it); the presigned-S3 fallback and the one-off `urlopen` calls in doctor/static transport are untouched; `Vault__API.close()` drops the pool. `pytest tests/unit/ -n auto` → **3861 passed** (one failure on the way: the layer-import rule, fixed by dropping the version from the User-Agent). **Measurement — read this before quoting numbers.** Every TLS connection from this cloud session, including "direct" ones with `NO_PROXY=*`, is terminated by the session's egress gateway (`openssl s_client` shows issuer `Egress Gateway SDS Issuing CA`, not Amazon). So the numbers below describe *that* path, not a laptop talking to CloudFront: | path | fresh connection per request (old) | reused (new) | |---|---|---| | via CONNECT proxy, 40 serial reads, mean | 388 / 340 ms | 292 / 283 ms | | "direct" (still gateway-terminated), p50 | 217 / 185 ms | 327 / 329 ms | The reused-connection path through the gateway is **bimodal**: ~145 ms or ~330 ms per request, and `curl` shows exactly the same pattern on HTTP/1.1 and HTTP/2 keep-alive, so it is the gateway's upstream handling, not Python and not SG/Send (I ruled out client delayed-ACK with `TCP_QUICKACK` and the two-segment request by sending headers+body in one record — no change). Through the explicit proxy, where a fresh connection costs more, keep-alive wins ~25 %. The full clone from here: `commits 27.4s trees 36.3s blobs 15.4s` for 194 commits / 2,735 blobs (the vault grew by 22 commits during the session), wall 86 s. On a laptop with no interception the expected saving is the handshake itself, which Dinis can measure in one line (second number is the reused connection): ``` curl -sS -o /dev/null -w 'first=%{time_total}\n' https://dev.send.sgraph.ai/api/vault/read/y8pwtjlw/bare/refs/x \ --next -o /dev/null -w 'reused=%{time_total}\n' https://dev.send.sgraph.ai/api/vault/read/y8pwtjlw/bare/refs/x ``` and for the real thing, the same clone twice: `sgit clone …` and `SGIT_HTTP_NO_KEEPALIVE=1 sgit clone …`, comparing the `⏱ commits … trees … blobs …` line. **Full-clone A/B from this session, same vault state (194 commits / 2,735 blobs), alternating:** | run | keep-alive | commits (serial, 1 req each) | trees | blobs | wall | |-----|-----------|------------------------------|-------|-------|------| | 1 | on | 27.3 s | 34.5 s | 14.9 s | 79 s | | 2 | off | 50.3 s | 27.1 s | 15.6 s | 96 s | | 3 | on | 26.8 s | 24.7 s | 13.7 s | 67 s | | 4 | off | 47.2 s | 26.4 s | 15.2 s | 91 s | The serial commit walk — the phase that is one round trip per request — is where reuse shows: 27 s vs 47–50 s. The parallel phases are within noise of each other (the gateway's bimodal reused-connection latency cancels most of the handshake saving there). --- ## 7. Final review of the release PR (SGit-AI/SGit-AI__CLI#6) — findings and fixes Ran the code-review pass at high effort over the three commits unique to this session (the rest of `dev` was reviewed by the architecture sessions in August). Ten findings; what was done with each: | # | finding | verdict | action | |---|---------|---------|--------| | 1 | an idle connection the server has already closed could be handed to a **write** (push: ref read → long local encryption → batch POST), failing with `URLError` where urlopen's fresh socket succeeded | **real, the important one** | pool now checks liveness before reuse (`select` readable == FIN, or TLS bytes pending → drop) and never reuses a connection idle > 30 s (a NAT that forgot the mapping sends no FIN). Push gets a fresh socket after a long local phase, exactly as before | | 2 | the stale-read resend re-acquired from the same idle bucket, so two stale sockets failed an idempotent read | real | the resend uses `acquire(fresh=True)`, bypassing the bucket | | 3 | lazy pool creation raced when the first call was a parallel `batch_read` (8 threads → up to 8 pools, 7 orphaned) | real | double-checked lock around creation; test drives 8 chunks as the first call and asserts one pool holds every socket | | 4 | plain `http://` base URL through an authenticated proxy never sent `Proxy-Authorization` | real, edge | `proxy_headers(key)` adds it per request on that path; the CONNECT path carries it on the tunnel as before; dead double-construction removed | | 5 | a 3xx is now a hard failure with no explanation | real (UX) | `_api_error` adds a hint naming the `Location` and saying to set `--base-url` to the final https URL; wording avoids the 404/405/501 markers the auto transport looks for | | 6 | `visited` included trees whose load failed (pre-existing) | real, pre-existing | with `seen` preventing re-enqueue the guard is unnecessary; `visited` is now "actually loaded", so `n_trees` is honest | | 7 | dead code in `open_via_proxy` | cleanup | folded into #4 | | 8 | three ad-hoc `ThreadPoolExecutor` fan-outs (`batch_read`, static transport, clone blobs) | follow-up | not in this PR — a shared bounded fan-out helper is a refactor, not a fix | | 9 | the four `batch_read` tests used `unittest.mock.patch` (CLAUDE.md: no mocks) | real | rewritten against the local keep-alive server, which now has a real batch endpoint (base64 of the file id; scripted 502 for one chunk) | | 10 | `object`-typed pool fields | convention | `timeout_seconds` / `max_idle_seconds` are `Safe_UInt`; the socket bookkeeping (`idle`, `warm`, `lock`) stays `object`, the same pattern as `Clone__Workspace`'s managers | Also added while here: every pooled socket has a 120 s per-operation timeout (urlopen had none; a kept-alive socket is the one place a silent peer can hang a command forever). One pre-existing behaviour the 502 test made visible, left alone: a 502 from the batch endpoint (Lambda response-size limit) is first retried whole three times with 2 + 4 + 8 s of sleeping before `batch_read` falls back to per-file reads. For a *size*-caused 502 the retries can never succeed; skipping them would save 14 s on that path. Suite after the fixes: `pytest tests/unit/ -n auto` → **3871 passed**. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/clone-refetched-shared-subtrees.html)* # Compatibility matrix: sgit CLI × vault web UI × SG/Send API, across the integrity roll-out, cli.sgit.ai > The CLI, the vault web UI and the SG/Send API across the integrity roll-out, step by step: what each version reads, writes and loses, under the rule that no client may break another still writing. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Compatibility matrix: sgit CLI × vault web UI × SG/Send API, across the integrity roll-out **Date:** 7 Oct 2026 · **Rule we hold:** a vault written by any client version stays readable and writable by every other version in use. A newer client may *lose* nothing that an older client does not already lose today (the web UI's single-entry index overwrite is the baseline loss), and must never *break* an older client that is still writing. The API does not change in any step below (Q1 answered: it is id-agnostic). ## 1. What ships now (on the branch, no cross-version effect) | Change | Old CLI (≤ 0.18.0) on the same vault | Today's web UI on the same vault | API | |---|---|---|---| | `fsck` dedupe (270 s → 12 s) | local only | n/a | none | | whole-vault guard names `sgit dev dump` | n/a | n/a | none | | **canonical (JCS) signing input** | reads the commit as before; nothing verifies, so the changed bytes are invisible | same; it never reads `signature` | none | | **`author_key_id` on new signed commits** | the field has been in `commit_v1` since March 2026 (`Safe_Str__Author_Key_Id`, `key-rnd-imm-…` matches); older CLIs parse it; any client without the field drops it on read | ignores it | none | | interop vectors (fixtures) | n/a | gives them something to test against | none | | website debrief, notes, briefs | n/a | n/a | none | **Verdict: merge and release as 0.18.x or 0.19.0 with no coordination.** One property to state in the release note: `vault move` re-encrypts and re-ids every commit, so signatures made before a move do not verify after it (they never did; it only becomes visible once verification exists). Move re-signs its own sentinel. ## 2. Next CLI batch (0.19): safe with today's web UI, degrades gracefully with old CLIs | Change | Effect on an old CLI writing the same vault | Effect with today's web UI | Loss vs today | |---|---|---|---| | **Gate fields in the index** (`format`, `min_client`, `features`), read and preserved; written only by an owner command | an old CLI's push re-uploads its local index from a Type_Safe object that **dropped the unknown fields**, so the gate is erased by the next old-CLI push. **Fails open** (vault reads as format 1 again), never closed | the web UI's single-entry overwrite erases it the same way | none: a gate that is erased leaves the vault exactly as today | | **`pull` / `status` re-read the remote index and merge entries** | old CLI unaffected (still reads its local copy) | after a web overwrite the new CLI merges its local entries back in and re-uploads, so the mapping is **restored** rather than lost | none; a gain when a new CLI pulls | | **Index upload with `write-if-match` + merge-retry** | old CLIs keep blind writes; last-writer-wins remains between them, as today | web overwrites still win when they come last, as today | none | | **Signature verification, warn mode** | read-only; no writes | read-only | none, but see noise below | | **Hint fix** on the pattern-mismatch error ("run `sgit update`") | applies only to clients that have it | n/a | none | | **`migrate apply` refuses on signed history** without `--force`, re-signs with it | old CLIs can still run the old migration; it rewrites history exactly as it does today | n/a | none | **Noise, not breakage:** on the collaboration vault today, 247 of 674 commits verify, 315 have no key mapping and 112 are unsigned (web UI and keyless clones). Warn mode must therefore be quiet by default: a summary line in `sgit check fsck` and `history log --verify`, never a warning per pull. Otherwise agents will learn to ignore it before it means anything. ## 3. Gated steps (need the other teams first; shipping them early would break the rule) | Change | What breaks without the other side | Who must ship first | |---|---|---| | **Ref monotonicity** (0.20) | today's web UI pushes the named ref unconditionally and can drop another writer's commits; a monotonic CLI reports each one as a rollback. True, but a stream of alarms | web shells on `pushIfMatch()` with a reconcile step | | **Signatures `required`** | every web commit fails policy (unsigned); 427 of 674 historic commits on the example vault can never pass | per-device web keys registered in the index; a checkpoint rule for history | | **Format 2 (32-hex ids)** | **the one hard break:** any CLI ≤ 0.18 fails on a 32-hex id. Correction (8 Oct, reproduced): not a validation error; verify-before-write refuses the object, so a fresh clone reports `integrity check refused vault data … corrupt or substituted content` and a pull reports `missing file … try sgit check fsck`, both pointing at the wrong fix (`sgit update` is the fix). Today's web UI reads them fine but writes 12-hex ids for new objects (mixed, allowed) | web `computeObjectId` format-aware; old web releases retired; and, on our side, no pre-gate CLI still writing to vaults that are raised | ## 4. The matrix API unchanged throughout. Cells read: what happens to a vault that both clients write. | | **Web UI today** (single-entry index overwrite, random-IV trees, unconditional push, 12-hex ids) | **Web UI with the index fix** (read-merge-CAS, honours the gate) | **Web UI id-aware** (+ format-aware ids, deterministic trees, `pushIfMatch`) | |---|---|---|---| | **CLI ≤ 0.18.0** | today. Index mapping lost on each web push; tree ids differ per writer; lost updates possible on web push | mapping kept; otherwise as today | as today for this client; it **crashes on a format-2 vault** | | **CLI 0.18.x / 0.19** (canonical signing, author_key_id, gate read+preserve, pull merges index, CAS upload, warn) | works. Gate erased by web pushes → fails open; new CLI restores mapping on its next pull; warn-mode counts are noisy | works. Gate holds; mappings hold; verification covers all new CLI commits | works. Verification covers web commits too once they sign | | **CLI 0.20** (+ ref monotonicity) | works, but web lost-updates surface as rollback warnings (correct, noisy) | same noise until `pushIfMatch` | clean | | **CLI format-2-aware** (+ 32-hex ids on raised vaults) | raised vaults: web reads fine, writes 12-hex ids (mixed, fine), **erases the gate** → an old CLI may then be let in and crash | raised vaults hold their gate; old CLIs refused by name only if they are ≥ 0.19, older ones crash | clean | ## 5. The two scenarios behind the question **"A vault pushed by the new CLI must work in the old web UI and the old CLI."** It does, for everything in §1 and §2. Nothing in those batches writes a byte an old reader cannot parse: `author_key_id` is an existing field, the gate fields are dropped by old readers, the index merge produces the same schema, canonical signatures are opaque base64 to everyone. The only future write an old reader cannot parse is a 32-hex object id, and that is §3, gated. **"It is fine if the web UI overwrites and we lose a little mapping."** Agreed, with the precise statement of the loss: the web overwrite loses *other branches' index entries*, which is what it loses today, and from 0.19 the next CLI pull restores them. Two things it must not be allowed to lose silently: a *raised* gate (so raising a vault to format 2 waits for the web index fix) and a *required* signature policy (same). Both fail open rather than closed, so the failure mode is "back to today", never "vault unusable". ## 6. Order, with the dependency each step actually has 1. **Now:** merge the branch; release. No dependency. 2. **0.19:** gate (read/preserve/owner-write), pull index refresh and merge, CAS index upload, warn-mode verification in `fsck` and `history log --verify`, hint fix, migrate guard. No dependency: everything fails open against today's web UI and old CLIs. 3. **Web index fix ships** (their §3.1). From then on gates and mappings hold. 4. **0.20:** ref monotonicity, after the web shells use `pushIfMatch`. 5. **Format 2:** after the web `computeObjectId` change and once the fleet of CLIs writing to a vault is ≥ 0.19 (the gate can refuse by name only from 0.19 up). Raise vaults one at a time. 6. **`required` signatures:** after per-device web keys. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/compatibility-matrix-cli-web-ui-api.html)* # Possible future features, after 0.21.0, cli.sgit.ai > The git-parity and security ideas that are not in 0.21.0, what each would take and what it could break, starting from the constraint that a new commit field breaks older clients' signature checks. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Possible future features, after 0.21.0 *CLI team, 8 Oct 2026. The ideas from "Fix merge and undo, then claim the browser" (the git and sgit feature comparison) and "Git security features → sgit" that are NOT in 0.21.0, with what each would take and what it could break. 0.21.0 already ships: reflog, signed tags, short ids and revision shorthand, `history undo`, log filters and `--stat`, `revert --as-commit`, `commit --amend`, `push --force-with-lease`, the path-guard hardening.* ## The constraint every commit-level idea must respect **A new field in commit objects breaks signature verification on clients from 0.19.0 on.** They parse the commit, drop the field they do not know, rebuild the canonical bytes without it, and the signature no longer matches: the commit shows as `bad`, and under `signatures-required` the pull is refused. (Verified with sgit-ai 0.20.0.) So any idea that adds a commit field needs the format gate: the vault owner raises `--min-client` first. Two fields are already in every client's schema and are safe to fill: `attestations` (a list) and `author_signature`. The branch index is different: unknown index fields are dropped by older clients when they rewrite the index and restored by the next current client (how tags work). New object types in `bare/data` are invisible to older clients. ## Ranked | # | Idea | Value | Cost | Side effects / constraints | |---|---|---|---|---| | 1 | **Union merge driver for declared paths** (`.sgattributes`: `runs/*.md merge=union`, `*.jsonl merge=union`) | High: concurrent appends by agents to one log stop conflicting | Medium: decrypt base/ours/theirs, line-merge, write a new blob inside the merge | Limited to opted-in paths. Clients without it still produce today's conflict (not a wrong merge). The attributes file is itself in the vault, so it is versioned and encrypted | | 2 | **Line-level three-way merge for all text files** (diff3) | High | Medium | Changes merge results for every text file; conflict markers inside files instead of `.conflict` copies. Ship after 1 has proven out, behind a setting at first | | 3 | **Shared operation log**: the reflog as encrypted, append-only objects that sync | High and distinctive: team-wide undo and audit, no git equivalent | Medium–large: a new object type and its sync | New objects only; old clients ignore them. Needs a retention rule | | 4 | **Author identity**: a display name per signing key | Medium: `log --author "Alice"` | Small if the name lives in the branch index entry (index field, safe); large if in the commit (format gate) | Put it in the index | | 5 | **Agent provenance in `attestations`**: prompt/tool/run records, encrypted | Medium–high for agent teams | Medium | Safe: the field exists in every client's schema | | 6 | **Conflicts as data**: commit a conflict as an object a human or another agent resolves later | High for agents that cannot stop mid-task | Large | Needs a conflict object type and every command to understand it | | 7 | **`history grep` / pickaxe / blame** | Medium | Medium: decrypts every changed blob in the walk; cache needed | Read-only. Must say where `--depth`/`--path` truncates the walk | | 8 | **Branch merge / delete / rename, cherry-pick** | Medium | Low–medium | Merge any branch reuses the three-way merge; delete needs the index tombstone pattern tags use | | 9 | **Stable change ids** | Medium | Medium | A commit field: format gate (see the constraint above) | | 10 | **Resumable push** | Medium | Medium | Verify today's behaviour first (large files already chunk) | | 11 | **Format 2 (128-bit ids) by default for new vaults** | Security default | One line in `init` | Clients ≤ 0.18 cannot read such a vault and call it corrupt. Do it once 0.19+ is the norm | | 12 | **Per-folder keys** | Distinctive: scoped clone becomes an access boundary | Large: key hierarchy, rekey, scope semantics | Crypto design work; an architect review first | | 13 | **Bisect, local hooks, aliases** | Low | Low each | Hooks only local, never read from vault content | | 14 | **`history restore`** as the name for today's file-level `history revert`, so `revert` can mean git's inverse commit | Clarity for git users | Low | Renaming breaks agents' scripts: add `restore`, keep `revert` working with a one-line note, decide later | ## Out of scope by design - **Server-side enforcement, per-writer authorisation, push certificates**: need the server to know writers (per-writer write keys or server-verified signed writes). An SG/Send API design question. - **Server-side search, forge-style review, CI status**: need a host that reads content. - **`format-patch` / `am`**: plaintext patches by email defeat the encryption. - **Partial staging**: refused by design (whole-file commits). - **Prune / gc**: only a key-holding client can compute reachability, and deleting objects other clones still reference conflicts with rewind detection. Needs a design of its own. ## Suggested order 1 (union driver) → 4 (names in the index) → 3 (shared oplog, building on the reflog) → 5 (provenance) → 2 (full line merge). Each is useful alone, none needs the format gate, and 1–3 are the ones the comparison note identifies as both valuable to agent teams and ahead of git. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/future-features-after-0-21.html)* # Review: "Git security features → sgit: a one-way mapping, cli.sgit.ai > A review of the one-way map from git's security features to sgit, checked against the code: three corrections, one missing gap, and the four easy gaps closed for 0.21.0. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Review: "Git security features → sgit: a one-way mapping" *CLI team, 8 Oct 2026. Review of the research note of the same date, checked against the code, plus what was built from it (on the branch, for 0.21.0).* ## Verdict A good, honest map. The framing is right: one direction only, "N/A by model" kept separate from real gaps, and the ranking puts the structural limit (no server-side enforcement) first. Three statements are wrong or overtaken, one important gap is missing, and the four easy gaps are now closed. ## Corrections | § | The note says | Actually | |---|---|---| | 2 | `author_key_id` is "reserved / unused" | **Live since 0.19.0.** Every CLI commit carries it; it names the signing key and marks the canonical (JCS) signing form, and `check verify` uses it. The schema comment still said "reserved", which is what misled the note; the comment is fixed. `author_signature` and `attestations` are the genuinely unused fields | | 6 | symlink tree entries "unconfirmed; one to verify" | **Closed, N/A by model.** A tree entry is a blob or a tree, nothing else; there is no symlink type and no code that creates one, so a vault cannot plant a symlink to write through | | 6 | case/Unicode variants of `.git` are a "possible gap" | **Confirmed, and fixed** (see below). Your test was exactly right | ## A gap the note misses **No per-writer authorisation.** In git hosting, who may push to which branch is decided per user by the server. In sgit the write key is one per vault and every writer holds it, so authorisation is all-or-nothing among key holders. Signatures give attribution (who wrote this commit), not permission (who was allowed to). A key holder can write anything the server accepts: refs, indexes, objects. The client-side defences (rewind refusal, `signatures-required`, tag signature checks) detect much of the damage afterwards; nothing prevents it. This belongs next to gap 1 in the ranking, and it is the real reason "server-side enforcement" matters: the server does not know which writer is which. ## What was built from the note (branch, all with tests; verified on the live dev API) | Gap (note's rank) | Now | |---|---| | 3. No tags | **`sgit vault tag create/list/show/delete`.** A tag is an immutable object (name, commit, message, tagger key, timestamp) signed over its canonical form, encrypted, content-addressed. Names live only in the encrypted branch index. Verification checks the signature and that the index entry's name equals the signed name, so an entry cannot re-point "v1.0" at another tag's object. Only pushed commits can be tagged; tags move only with `--force`, and pull reports moves. Deletes are tombstones a stale clone cannot undo. Tag names work in `history show/reset`. A 0.20.0 client drops tags from the index on its first push; the next current pull restores them (seen live) | | 2. No reflog | **`sgit history reflog`**: every local ref move, newest first, local only, capped at 1,000; `history reset ` brings a head back | | 2. No force-with-lease | **`sgit push --force-with-lease []`**: forces only if the remote is where this clone last saw it before the push (or at the commit given), and writes with compare-and-swap. It deliberately ignores the refresh that push itself does first; git's version has a known footgun there (a fetch silently satisfies the lease) | | 4. `.git` spellings | **Path guard refuses** `.GIT`, `.Git`, `.git.`, `.git `, `GIT~1`, `SG_VAU~1`, `.git::$INDEX_ALLOCATION`, `.g‌it` and the same for `.sg_vault*`, on every platform (the vault may be checked out anywhere). `.github`, `.gitignore`, `src/git`, `notes.git` stay allowed | Found on the way: `sgit push --force` from a clone that was behind the remote crashed ("object … is not in the local store") on 0.20.0. Fixed. ## Still open, and why - **Server-side enforcement and per-writer authorisation** (1 + the missing gap): need the server to know writers (per-writer write keys or signed writes the server verifies). An SG/Send API design question, not a CLI change. - **Push certificates / nonces** (5): low value until the server verifies anything; rewind detection covers replay for clients that were there before. - **Notes, attestations, a second signer** (6): the reserved fields can carry them; worth doing when there is a review workflow to serve. - **Format 2 by default** (7): agree in principle. The blocker is clients ≤ 0.18, which cannot read a vault with 128-bit ids and say "corrupt", not "update". Make it the default for new vaults once 0.19+ is the norm; it is a one-line change in `init`. ## Suggested wording changes for the note - §2 row 5: "author_key_id is live (0.19.0+); author_signature and attestations are reserved". - §6: symlink row → N/A by model; case-variant row → Matched (0.21.0). - §3/§5: reflog, force-with-lease, tags → Matched (0.21.0); signed tags Matched. - Ranking: add "no per-writer authorisation" beside gap 1. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/git-to-sgit-security-mapping.html)* # History integrity: what the three fixes would cost, and how to ship them without a `vault move`, cli.sgit.ai > What signature verification, ref monotonicity and 128-bit ids would cost, measured on a real vault, and how to ship all three without a vault move. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # History integrity: what the three fixes would cost, and how to ship them without a `vault move` **Date:** 7 Oct 2026 · **Vault measured:** the DC / CRM collaboration vault (`y8pwtjlw`), fresh full clone this session · **Ask:** "do a set of performance tests to see the implications [of signature verification, ref monotonicity, 128-bit ids], and can we add those fixes in a forward-compatible way, supporting existing vaults without a move?" ## 0. The vault today | | | |---|---| | commits reachable from the named head | 674 (626 yesterday) | | unique trees / tree entries | 8,589 / 107,445 (59,944 blob refs, 47,501 tree refs) | | objects in the store | 19,780 · 146 MB | | branches in the index / public keys on disk | 30 / 29 | | full clone from this sandbox (0.18.0) | 93 s | | `sgit status`, up to date | 2.5 s (network-bound) | ## 1. Cost of each fix, measured All local work, this sandbox's CPU. Network is unchanged by any of the three. | Fix | What it adds per command | Measured | Verdict | |---|---|---|---| | **Signature verification** on pull / status / fsck | one ECDSA P-256 verify per *new* commit (pull) or per commit (fsck) | verify 66 µs, sign 23 µs; the whole 674-commit history verifies in **39 ms**; decrypting and parsing the whole history is 155 ms (230 µs/commit) | free | | **Ref monotonicity** (the new remote head must descend from the last known one) | an ancestry walk from the new head to the last known head, i.e. over the N new commits | status already does this walk since 0.18.0 (0 object reads when up to date, N when N behind); worst case the whole history: **155 ms** | free | | **128-bit object ids** (32 hex instead of 12) | nothing in compute (sha256 is already computed; the truncation is what goes away); more bytes in trees, commits and listings | +2.18 MB on 34.5 MB of tree+commit objects (**+6.3 %** of those, **+1.5 %** of the 146 MB store); the store listing grows 396 KB; expected clone cost ≈ +1.5 % of bytes, under 2 s | negligible | And the exposure the third fix removes, on this vault: accidental collision among 19,780 ids at 48 bits ≈ 7×10⁻⁷ (not the problem); a *second-preimage* by someone holding the read key is 2⁴⁸ sha256 ≈ 8 hours at 10¹⁰ hashes/s on a GPU (the problem). ### What signature verification would actually cover today | commits | | |---|---| | signed | 562 of 674 | | signed and verifiable (the branch index still maps the branch to its public key) | **247** (all verify; 0 bad) | | signed, but the index no longer lists the branch | 315 | | unsigned | 112: 10 with no `branch_id` (web UI), 102 from CLI clone branches that had no local signing key | Two consequences for the design: the commit's reserved `author_key_id` field has to be populated from now on (the key is in `bare/keys/`, but the only branch→key mapping is the index, and clone branches leave it), and a "signatures required" policy can only apply from a checkpoint commit forward, since 427 of the 674 commits here cannot be verified whatever we do. ### A side finding: `sgit check fsck` was quadratic Full verification is the thing that makes "always detect corruption" practical, and it took **270 s** on this vault: the walk kept a per-commit set of visited trees, so 8,589 unique trees were checked 282,202 times and every blob was re-hashed once per commit. One global set for trees and one for blobs: **12.2 s**, identical findings (42 missing objects; they are the pre-0.18.0 push bug and are recoverable only from the clones that created them, via `sgit check upload-objects`). Fixed on the branch; 32 fsck tests pass, full suite green. ## 2. Compatibility: what exists, what does not **Today there is no vault-level "minimum client version".** - Objects carry a `schema` tag (`commit_v1`, `tree_v1`, `branch_index_v1`). Nothing reads it except `vault move`. - The migration ledger (`sgit migrate`) is **local** (`.sg_vault/local/migrations.json`); the vault itself does not say which migrations it has had. - Old clients drop unknown fields silently (`Schema__Branch_Index.from_json` with a `format` or `min_client` key just loses them), so a flag written today is invisible to 0.18.0 and older. They fail only when they meet data they cannot parse: a 32-hex id raises `ValueError … exceeds maximum length of 24` and the CLI prints `incompatible vault data … hint: re-clone the vault with the current CLI`, which is the wrong remedy for that case. So: forward-incompatible vaults are *possible* today only by accident, and the failure mode for an old client is a confusing error, not a "please upgrade". ## 3. Proposal: a format gate, then each fix on top of it ### 3.1 The gate (no format change; ships first) Add to `Schema__Branch_Index` (encrypted, written by init/push, loaded first by every command): ``` format : Safe_UInt = 1 # absent == 1: every existing vault min_client : Safe_Str__Semver = None # refuse below this, naming it features : list[Safe_Str__Feature] # e.g. 'ids-128', 'signatures-required' ``` - A client at or above the release that ships the gate refuses a vault whose `min_client` is newer than itself: `this vault needs sgit-ai ≥ 0.20.0 (you have 0.19.1): sgit update`. - Existing vaults carry no field, read as format 1, and behave exactly as now. **No move.** - Clients older than the gate cannot be taught anything retroactively; the one thing we can do for them now is fix the hint on the pattern error to say `sgit update` first. - Raising `min_client` on a vault is a deliberate owner action (`sgit vault format --min-client 0.20.0`), recorded in the index, so a team is never surprised by a vault that stops working for their older agents. ### 3.2 Each fix against the gate | Fix | Format change? | Needs `vault move`? | Old clients on an upgraded vault | Notes | |---|---|---|---|---| | Verify signatures (pull, status, fsck) and populate `author_key_id` | none; both fields already exist in `commit_v1` (`author_key_id` reserved) | no | unaffected: they ignore signatures as they do now | policy per vault in `features`: `signatures: warn` → `required` from a checkpoint commit; the web UI must sign before `required` | | Ref monotonicity | none | no | unaffected | client rule; `push --force` records the intent so the next pull can tell a rewind from an attack; a signed ref log is a later, format-2 addition | | 128-bit ids | **format 2**: new objects get 32-hex ids, old objects keep theirs, references mix | **no**: mixed-id vault; `vault move` (which already rewrites every id) is the optional "rewrite everything" path | **break at the first long id** → this is what `min_client` is for | prerequisites outside the CLI: the server must accept the longer `file_id` (unverified from here), the web UI must read and write both lengths (interop requirement) | ### 3.3 Order 1. **0.19**: the gate, the hint fix, `author_key_id` on every new commit, fsck dedupe. No vault changes behaviour; every client keeps working on every vault. 2. **0.20**: signature verification (`warn` default, `required` opt-in per vault), ref monotonicity. Still format 1. 3. **format 2**: 128-bit ids for new vaults and opted-in vaults, once the server and web UI accept both lengths. Existing vaults stay format 1 until their owner raises them; raising never requires a move. The measurements say none of this is a performance decision: the costs are milliseconds and a few percent of bytes on a vault that was impractical to clone a week ago and now clones in 93 s. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/history-integrity-cost-and-compatibility.html)* # Research, cli.sgit.ai > Articles from the sgit CLI team, newest first: measurements, designs, reviews, decisions and fixes, each taking one topic as far as the evidence goes, with its source in the CLI repository. *Source: · generated from the same content as the page, so the two cannot drift.* --- # Research The CLI team's articles, newest first. Each takes **one topic, one implementation or one idea** as far as the evidence goes: a measurement on a real vault, a design and the review that corrected it, a bug traced to its cause. They are, in the words of [the brief](../briefs/index.md), "our new generation of briefs", and this is where they are captured first. The first 23 were written in the CLI repository between 2 and 16 October 2026 (`team/humans/dinis_cruz/claude-code-web/`) and are published here as written, each with its source path and commit. The *kind* on each card says what it is worth: a **measurement** is numbers from a run; a **design**, **plan** or **analysis** is a proposal, not a shipped feature, unless it says so; a **fix** or **finding** names the code. - [sgit is a standard, not a server: everything in a vault is one hash away](../articles/sgit-is-a-standard-one-hash-away.md): A vault is a set of files laid out by a convention, every immutable object named by the hash of its own ciphertext, so the server is the least interesting part and anything in the path can cache blindly. - [Response to "Scion and sgit" (newsroom.sgit.ai, article No. 141, v1.0.0)](../articles/response-to-scion-and-sgit.md): The CLI team's response to the newsroom's "Scion and sgit" article: what it gets right, the three factual errors, and the measurements from a 35k-object vault the article should have carried. - [One vault, ten agents, 35,000 objects: what sgit does when the repository is big and the clients are many](../articles/one-vault-ten-agents-measured.md): What sgit does when the repository is big and the clients are many: the comms vault, 35,000 encrypted objects and ten agents, cloned eight ways and measured from a cloud sandbox. - [Response to "sgit issues found on 14 October 2026", and the comms-vault performance report](../articles/sgit-issues-2026-10-14.md): The issues reported on 14 October, each traced to its cause in the code, with what was fixed in the same push and what was left open. - [Security CI: a dedicated pipeline, and an AI review that can break the build](../articles/security-ci-and-ai-review.md): A dedicated security pipeline on every pull request, and a keyless AI security review that can break the build on anything new: why, how it is wired, and what it costs per reviewed change. - [AC-L-001 / TM-R36 — characters outside `[a-zA-Z0-9/\_.- ]` in paths: what really happens](../articles/non-ascii-paths-refuse-dont-rewrite.md): Characters outside a narrow set in a path are silently rewritten, so one vault gets two roots and a false push conflict. What really happens, reproduced, and a fix that raises an alarm instead of rewriting. - [`sgit serve`: container first. Options map and spike results](../articles/sgit-serve-container-first.md): The decision to run the SG/Send container image rather than re-implement the server: the runtimes supported (including Apple's container and Podman), the spike results, and phase 1 as built. - [`sgit serve`: running a vault locally — research and plan](../articles/sgit-serve-running-a-vault-locally.md): How to run a vault locally: the vault server's API inventory, the browser modes, when TLS is and is not needed, and how much plumbing sgit would need. - [Case study — a tool-assisted security review of sgit-ai 0.21.0 (round 1)](../articles/security-review-case-study.md): What each tool found in the round-1 security review of sgit-ai 0.21.0, with counts and cost: the skill vetting, the derived graph, 270 function records, 68 STRIDE threats and the insider overlay. - [Threat-modeling tools for an agent-driven review — Matt Adams' stack, the classics, and the house method](../articles/threat-modeling-tools.md): Threat-modeling tools for an agent-driven review: Matt Adams' stack (STRIDE GPT, the insider-threat model), the classics, and the house method, with the agent itself named as an actor. - [Agent skills for security review — what exists, what it would do here](../articles/security-review-skills.md): Agent skills for security review, each checked at its source, ranked by whether it brings a method, a tool with deterministic output, or vets the other skills, against this code base. - [Plan — the next security review of sgit-ai, with skills, and the review graphs it produces](../articles/security-review-plan.md): The plan for a tool-assisted, data-producing security review of one slice of sgit-ai: the code the zero-knowledge guarantee and the trust boundaries live in. - [Sealed files — a second encryption layer on top of a vault](../articles/sealed-files-layer.md): A second encryption layer on top of either vault mode: files sealed to named recipients with ECDH P-256, HKDF and AES-256-GCM, with the corrections from the sgit.ai agent's review. - [Native PKI vault mode — architecture map](../articles/native-pki-vault-mode.md): An architecture map for a second vault mode built on key pairs from the start, revised twice after review: sequence-number pinning, frozen hosts, concurrent writers. - [Read and write keys as PKI — design analysis](../articles/read-write-keys-as-pki.md): Are the read key and the write key really a public and a private key? Half right: the write key should become a signing key, and the read key cannot become a public key. What that would take. - [Branches check, test performance, test quality — 8 Oct 2026](../articles/branches-and-test-suite-review.md): An end-to-end check found that branches did not work. What was fixed behind it, the tests that now cover it, and a review of the test suite's speed and quality. - [Possible future features, after 0.21.0](../articles/future-features-after-0-21.md): The git-parity and security ideas that are not in 0.21.0, what each would take and what it could break, starting from the constraint that a new commit field breaks older clients' signature checks. - [Review: "Git security features → sgit: a one-way mapping](../articles/git-to-sgit-security-mapping.md): A review of the one-way map from git's security features to sgit, checked against the code: three corrections, one missing gap, and the four easy gaps closed for 0.21.0. - [Compatibility matrix: sgit CLI × vault web UI × SG/Send API, across the integrity roll-out](../articles/compatibility-matrix-cli-web-ui-api.md): The CLI, the vault web UI and the SG/Send API across the integrity roll-out, step by step: what each version reads, writes and loses, under the rule that no client may break another still writing. - [History integrity: what the three fixes would cost, and how to ship them without a `vault move`](../articles/history-integrity-cost-and-compatibility.md): What signature verification, ref monotonicity and 128-bit ids would cost, measured on a real vault, and how to ship all three without a vault move. - [Partial clones — a folder scope and a history depth (6 Oct 2026)](../articles/partial-clones-scope-and-depth.md): Cloning just the folders an agent reads and writes, read and write, plus a shallow mode, without changing the full clone: the Merkle-spine idea and the measurements. - [Fix — `sgit pull` silently discarded uncommitted edits to tracked files](../articles/pull-discarded-uncommitted-edits.md): sgit pull silently discarded uncommitted edits to tracked files. The root cause, and the fix: git's rules for the working tree, applied before any write. - [Clone performance — the tree walk re-fetched shared sub-trees once per parent](../articles/clone-refetched-shared-subtrees.md): A clone stuck walking trees: the walk re-fetched shared sub-trees once per parent, 10,098 requests for 2,236 unique trees. The instrumented run and the fix. ## Publishing an article Adding one file publishes it: `admin/content/articles/.md`, with a title, a date and a summary in its front matter. The contract is in `admin/content/articles/README.md`. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/index.html)* # Native PKI vault mode — architecture map, cli.sgit.ai > An architecture map for a second vault mode built on key pairs from the start, revised twice after review: sequence-number pinning, frozen hosts, concurrent writers. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Native PKI vault mode — architecture map **Date:** 2026-10-09 · **Status:** design for review, no code · **Follows:** [`design-analysis__read-write-keys-as-pki.md`](../articles/read-write-keys-as-pki.md) (same folder), which covered adding a writer signature to today's symmetric vaults. This document maps a second **vault mode** built on key pairs from the start. Companion: [`design__sealed-files-layer.md`](../articles/sealed-files-layer.md) (encryption on top of either mode). > **Revised 2026-10-10 after the sgit.ai agent's review (dev at 0a0707d).** **Sequence-number > pinning is a requirement, not an option.** Every signed ref and the signed index carry a > monotonic `seq`. Each clone pins the highest `seq` it has accepted per ref and for the > index, and refuses (fails closed) anything lower, or equal with different content. Without > it, a host replaying an old signed ref or index (TM-R29) survives the signatures. A fresh > clone still has nothing to compare against: it is told the `seq` it got, and a witness or > transparency log stays optional (RFC question 5). > **Revised again 2026-10-10 after the release-gate review (dev at eed8084).** `seq` pinning > closes replay for clones that have seen a state. Four things it leaves open are now > specified in §3.1: > - **a host that freezes** (stops serving new states): writers sign a periodic checkpoint, > and the vault sets how stale one may be; > - **two writers producing the same next `seq`**: each signed state names its predecessor's > hash, the server's compare-and-swap picks one, and two valid states with one `seq` are > equivocation, refused and reported; > - **the pin store**: written by the secret writer, never lowered, fail-closed when it is > missing once recorded, merged by maximum on restore; > - **fresh clones**: still trust on first use, as before, but a share link can carry a floor. ## 1. Your question, checked > A vault where one key lets you read and a different key lets you write, and neither > key can be computed from the other. Can one public/private pair do that, used either > way round? **One key pair cannot. Two key pairs can, and they give exactly what you describe.** A key pair supports two operations, and in each the direction is fixed: | Operation | Done with | Undone / checked with | What it gives | |---|---|---|---| | encrypt → decrypt | **public** key | **private** key | confidentiality *to the private-key holder* | | sign → verify | **private** key | **public** key | proof that the private-key holder wrote it | Two facts settle it: - **The private half always yields the public half** (for Ed25519 / X25519 trivially, and for RSA the public key sits inside the private key). Only the reverse is infeasible. So whichever key goes to the *less trusted* party must be the public one. - **"Decrypt with the public key" is not confidentiality.** With RSA you can run the math the other way, but that is signing: anyone with the public key can undo it. Keeping a "public" key secret to use it as a read key is not sound either. RSA public exponents are conventionally 65537, and EC public keys are derived from the private key. With one pair, writers sign (private) and readers verify (public). That gives *who wrote it*, but nothing for keeping content secret. Add a second pair for reading and every property you asked for falls out: ``` R = read pair (X25519, for encryption) R.priv decrypts R.pub encrypts W = write pair (Ed25519, for signatures) W.priv signs heads W.pub verifies ``` | Party | Holds | Can | Cannot | |---|---|---|---| | **Reader** | `R.priv`, `W.pub` | read everything; check that every head was signed by a writer | write: needs `W.priv`, not derivable from anything it holds | | **Writer** (normal) | `R.priv`, `W.priv` | read and write | — | | **Depositor** (write-only) | `R.pub`, `W.priv`* | add new content readers can open | read anything: needs `R.priv`, not derivable from `R.pub` | | **Host** | ciphertext, `W.pub` | store, serve, refuse unsigned writes | read; forge a head (no `W.priv`); can still withhold or replay old signed states | \* or a depositor-specific signing key the writers accept for an inbox branch (§5). So: **two keys, neither derivable from the other, one for reading and one for writing.** The read capability is `R.priv` (+ the public `W.pub`); the write capability is `W.priv` (+ `R.priv` for writers who also read). A write-only depositor is the one case today's design cannot express at all. **Two vault modes side by side is fine.** The object store, content addressing, trees, sync, merge, scoped clones, the path guard and the CLI stay shared. The mode decides only three things: how keys come into being, how the data key reaches a reader, and what authorises a write. ## 2. The architecture ``` owner master seed M (32 bytes; or PBKDF2(passphrase) for humans) │ HKDF ┌───────────────────┴────────────────────┐ W.priv (Ed25519) R.priv (X25519) W.pub ──────────► vault id = base32(SHA-256(W.pub))[:26] (self-certifying, 128 bits) R.pub write capability = M (derives everything) read capability = R.priv + W.pub (cannot derive W.priv or M) deposit capability= R.pub + a signing key the vault accepts for its inbox vault data key DK (random 32 bytes, generation g) bare/keys/dk- = HPKE-Seal(R.pub, DK) ← readers unwrap with R.priv every object = AES-256-GCM(DK, …) ← exactly today's object format every ref / index = AES-256-GCM(DK, …) + Ed25519 signature by W.priv over (vault_id, file_id, seq, ts, ciphertext hash) ``` **Why a data key, rather than encrypting every object to `R.pub`.** Public-key encryption is randomised: the same tree encrypted twice gives different bytes. Today's design depends on deterministic tree encryption in two places: - tree ids repeat (dedup); - a scoped clone rebuilds a spine whose ids match the whole-vault builder byte for byte. With a symmetric data key wrapped once to `R.pub`, the object layer is **unchanged**: `DK` simply plays the role today's read key plays. The PKI sits in two places only: - **distribution:** `R.pub` wraps `DK`, so `DK` never travels as a passphrase; - **authority:** `W.priv` signs heads, so no write is authorised by a bearer secret. ## 3. What each piece fixes (threat model rows) | Row | Today | Native PKI mode | |---|---|---| | TM-R01 read-key holder + host forge history | by design | **closed**: heads must carry `W.priv`'s signature | | TM-R03 refs not bound to their id | accepted | **closed**: the signature covers vault id + file id | | TM-R29 host replays an old index (drops features) | accepted | **closed** for content (signed); freshness still needs `seq` pinning | | TM-R13 bearer write key | by design | **gone**: the server checks a signature, so nothing replayable crosses the wire | | TM-R14 keys in argv | H likelihood | reduced: capabilities can be handed over as wrapped envelopes (sealed-files doc §6), never typed | | TM-R12 no read revocation | by design | **better**: rotate `R` and `DK`, re-wrap to remaining readers; old data stays readable to the removed reader (inherent) | | Write-only contributors | impossible | **new**: depositor capability (§5) | | Freshness for a fresh clone (SP-2) | open | partly: signed `seq` + `ts` lets a client flag a stale head; a host can still withhold | ### 3.1 Freshness, concurrent writers and the pin store (revised 10-10) **A host that freezes.** Pinning stops a host going *back*; it cannot see a host that simply stops showing new states (every clone keeps its last, valid view, and teammates diverge silently). Two pieces: 1. **Signed checkpoints (heartbeat).** Every write already signs `{seq, ts}`. In addition, any writer's client publishes `checkpoint = sign_W(vault_id, index_seq, H(ref seqs), ts)` when the newest one is older than `checkpoint_interval` (default 6 h, set by the owner in the index gate). A quiet vault therefore still shows a fresh `ts`. `status` and `pull` warn when the newest checkpoint is older than `max_staleness` (default 24 h): "this server has shown nothing newer than ; it may be withholding". The check warns and does not refuse, because a vault with no active writer looks the same. 2. **Witnesses (optional).** A vault can name witness URLs in the signed index: other hosts (a mirror, the sgit.ai site, a teammate's static copy) that store the latest checkpoint they saw. A clone compares the host's checkpoint with the witnesses' and refuses to *push* on top of a state older than one a witness holds. Withholding then requires the host and every witness to collude. **Two writers, the same next `seq`.** Every signed ref and index state carries `prev = SHA-256(previous signed state)`, a hash chain per file. - Online, the server's compare-and-swap (`write-if-match` on the previous bytes) lets one writer win. The loser re-reads, merges (the index) or rebases (a ref), and writes `seq + 1` with `prev` set to the winner's state. This is today's index CAS loop, with the chain added. - A client accepts state `n+1` only if its `prev` equals the hash of the state it pinned at `n`. Two valid, signed states with the same `seq` and different content are **equivocation**: a host that served different states to different clones, or a writer bypassing CAS (static hosting has none). The client refuses both, keeps its pin and reports the two hashes. Recovery is a deliberate owner action (`sgit vault resolve-fork`), never automatic. - A clone that has been offline accepts a run `n+1 … n+k` as long as each link verifies. **The pin store.** `.sg_vault/local/pins.json` maps `{file_id: (seq, state hash)}` for each ref and for the index. - **Writes:** written by the single secret writer (temp file, fsync, rename, 0600), like `remote_heads.json`. - **Integrity:** the file carries an HMAC under a key derived from the read capability, which detects corruption and half writes. It is not a defence against an attacker who can write `.sg_vault/local/`: that attacker can also replace the keys. - **Never lowered:** an update takes the per-entry maximum, so no code path can lower a pin. Restoring a backup merges by maximum with the pins already present, never replaces them. A sync tool or `cp -r` reverting the folder is covered by the next rule. - **Fail closed:** a missing or unreadable pin file, once one has been recorded (a marker in `config.json`, as `remote_heads_file` is today), refuses pull and push until `sgit pull --accept-rewind` rebuilds it, deliberately (TM-F23's rule). - **Backups:** `uninit` backups carry the pins. **Fresh clones** remain trust on first use. Two optional ways to give them a floor: - the share link or capability envelope can carry `(index_seq, index hash)` at share time; - a witness, as above. ## 4. Mapping to the code | Area | Stays | Changes | |---|---|---| | `Vault__Crypto` | AES-GCM, deterministic metadata encryption, content ids | + `Vault__Crypto__PKI`: HKDF derivation from `M`; Ed25519 sign/verify; HPKE seal/open (X25519, HKDF-SHA256, AES-256-GCM: RFC 9180 base mode) | | Key strings | `sgit_private_vault_…`, `sgit_public_read_…` | + `sgit_pki_write_…` (M), `sgit_pki_read_…` (R.priv ‖ W.pub), `sgit_pki_deposit_…`; `Enum__Key_Kind` gains them; the prefix *is* the mode, so a fresh clone knows signatures are mandatory (no strippable flag) | | `Vault__Components` | object store, refs, index, branches | `read_key` ← `DK` (unwrapped); + `verify_key` (`W.pub`); + `signing_key` (`W.priv`, writers only) | | Refs (`Vault__Ref_Manager`) | encrypted `{commit_id}` | `{commit_id, seq, ts}` + signature; verified on every read path (status, pull, clone, fetch) | | Index (`Vault__Index_Sync`) | merge rules, tags, gate | signed on every write (all writers hold `W.priv`); verified on read; `seq` monotonic per clone | | `Vault__API` | endpoints, batch, CAS | writes carry a request signature (method, path, body hash, time) instead of `x-sgraph-vault-write-key`: **server change** | | Commits | per-clone ECDSA signatures | unchanged (attribution: *which* writer); the vault signature answers *a writer at all* | | Format gate | format 1/2 | `format: 3` + mode `pki`; `min_client` set at creation | | Migration | `vault move` rewrites everything already | `sgit vault move --to-pki` makes a PKI vault from a symmetric one | ## 5. Write-only (deposit) capability A depositor can encrypt (it holds `R.pub`) but cannot read, so it cannot read the current tree to modify it. Write-only therefore means **append**, not edit: - each deposit is a commit on a per-depositor **inbox branch** (`inbox/`), whose tree holds only what that depositor added; - its objects use a per-deposit data key `DK_d`, sealed to `R.pub` and stored beside the commit; - a writer merges inbox branches into the main branch, and from then on that content is ordinary data. This fits agents handing work to a human, form-style submissions, and log or report drops. The vault accepts the depositor's signing key for its inbox branch only: the writers add it to the signed index. ## 6. Rotation and the owner key The self-certifying vault id (`H(W.pub)`) means a new `W` is a new vault. That is fine for "a writer left" today (it is what `vault move` does), but heavy for teams. The clean fix, and the same design as the threat model's "owner-signed membership": - **vault id = H(owner root key `O.pub`)**, with `O` kept offline or in a decryption service; - the index carries an `O`-signed **writer set** (one `W.pub` per member, or one shared); - removing a writer = `O` re-signs the set without them. Removing a reader = new `R` and `DK`, re-sealed to the remaining members. The sealed-files doc's per-member envelopes do exactly this. Start with one shared `W` (simple, matches today's team model) and keep the writer-set format ready for per-member keys. ## 7. Interop and algorithms - **Ed25519, X25519, HKDF-SHA256, AES-256-GCM** are all in `cryptography` (installed: 49.0.0) and in current browsers' Web Crypto. - **Test vectors.** Byte-for-byte vectors (CLAUDE.md) work for derivation and Ed25519 (deterministic). HPKE uses a fresh ephemeral key, so its vectors fix the ephemeral key, the same way RFC 9180's do. - **Post-quantum:** AES-256 content is fine. The key exchange is the exposure (harvest-now-decrypt-later on `bare/keys/dk-*`); HPKE has hybrid X25519 + ML-KEM suites to adopt when the browser side has them (TM-R20). ## 8. Effort | Phase | What | CLI estimate | |---|---|---| | 1 | `Vault__Crypto__PKI`, key strings, derivation, vectors shared with the web team | ~1 week | | 2 | Signed refs and index + verification on every read path (also usable on symmetric vaults; = phase 1 of the earlier analysis) | 1–1.5 weeks | | 3 | `DK` sealed to `R.pub`; read path via `DK`; `init --pki`, `clone` of a PKI vault | ~0.5 week | | 4 | Server: store `W.pub`, accept signed writes (SG/Send team) | server side | | 5 | Deposit capability + inbox branches | ~1 week | | 6 | Owner root key + signed writer set + rotation | 2–3 weeks | Phases 1–3 give a working native mode in which a reader cannot write and the host cannot forge. The web UI needs the same verification and unwrap code to open PKI vaults. ## 9. Decisions for you 1. **Vault id:** self-certifying `H(W.pub)` now, or go straight to `H(O.pub)` with a writer set? My view: start with `W.pub`, and design the index field so `O` can be added later. 2. **Capability strings:** random 32-byte seeds only (agents and machines), or also a passphrase form for humans (PBKDF2 → `M`, as today)? 3. **Deposit / inbox:** in the first release or later? 4. **Server signed writes (phase 4):** ask the SG/Send team now? Until then the server keeps the bearer check, but clients reject anything unsigned, so forgery is closed either way. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/native-pki-vault-mode.html)* # AC-L-001 / TM-R36 — characters outside `[a-zA-Z0-9/\_.- ]` in paths: what really happens, cli.sgit.ai > Characters outside a narrow set in a path are silently rewritten, so one vault gets two roots and a false push conflict. What really happens, reproduced, and a fix that raises an alarm instead of rewriting. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # AC-L-001 / TM-R36 — characters outside `[a-zA-Z0-9/\_.- ]` in paths: what really happens **Status:** documented, **not fixed in this push** (your call, 13 Oct). Reproduced on `dev` at 81a4176 with the Python API over the in-memory host. The fix proposal below follows your instinct: **raise an alarm, never rewrite silently**. It is narrower than "refuse every such character", for the reason in §3. It is worse than the round-1 record (`review/findings/audit-context-local.json` AC-L-001) says. Two different things share the bug: **where the vault lives** (the host path of the clone) and **names inside the vault** (file and folder names in the trees). ## 1. The vault root: broken, and silently in the wrong place `Safe_Str__Vault_Path` and `Safe_Str__File_Path` both use `[^a-zA-Z0-9/\\_.\- ]` in *replace* mode: every other character becomes `_`. The working directory of the clone, pull, fetch, push and move workflows (`directory : Safe_Str__File_Path`), `Vault__Context.vault_path` and `Vault__Storage` go through them; other code uses the raw `str`. So one vault gets two roots. | Step, under `/tmp/x/josé/vault` | What happened | |---|---| | `init` | the working copy and `.sg_vault/{bare,local}` are under `josé/vault`, **and** a second tree `jos_/vault/` is created next to it | | `commit`, `status` | ok, `clean` | | first `push` | **`Vault__Push_Conflict_Error`: "the branch moved on the server while this push was in flight"** — false; nothing was pushed | | `clone` into `josé/clone` | reports success; **the whole clone is written to `jos_/clone`**; `josé/clone` does not exist | | `pull` in `josé/clone` | `FileNotFoundError: …/josé/clone/.sg_vault/local/vault_key` | | second `commit` + `push` in `josé/vault` | `FileNotFoundError: …/jos_/vault/.sg_vault/work/pull-…/.tmp-…` | It is not only accents. What the type does to real folder names: | Path | Becomes | |---|---| | `/Users/x/Dropbox (Personal)/v` | `/Users/x/Dropbox _Personal_/v` | | `/Users/x/OneDrive - Acme, Inc/v` | `/Users/x/OneDrive - Acme_ Inc/v` | | `/home/first.last@corp/v` | `/home/first.last_corp/v` | | `/home/u/my+vault` | `/home/u/my_vault` | | `C:\Users\Zoë\v` | `C_\Users\Zo_\v` | **The last row needs a check on Windows.** The colon of every drive letter is rewritten, so if any of these fields carries an absolute Windows path, every Windows user is affected, not just users with accented names. I could not run Windows here. Security reading: a write lands in a directory the user never named. Two roots that sanitise alike share one store (`josé/` and `josè/` and a real `jos_/`), so one vault's objects, refs and `work/` files can mix with another's. A local process (A6) that can create `jos_/` next to `josé/` decides where the store goes. Likelihood M (any European or company folder name, any Dropbox or OneDrive folder); impact H (a false "conflict", a clone in the wrong place, failures on the next command). I have raised TM-R36 from M / M to M / H. ## 2. Names inside the vault: the core is right; the display and `--path` are not Tree entry names are encrypted strings, not `Safe_Str` fields, so the sync path never touches them. With `café.md`, `naïve dir/résumé.md`, `a+b@c.md`, `日本語.txt` and a `caf_.md` beside `café.md`: | Step | Result | |---|---| | status, commit, push, clone | every file round-trips byte for byte, with the exact name; `café.md` and `caf_.md` stay two files | | edit in the clone, push, pull in the first clone | ok; content and name exact | | `stash` / `stash pop` | ok | | **`sgit diff`** (`Schema__Diff_File.path`) | **lists `caf_.md` and `na_ve/r_sum_.md`**: the display lies | | **`sgit clone --path naïve`** (`scope_paths : list[Safe_Str__File_Path]`) | **records the scope `na_ve`, checks out nothing, and says nothing** | | history file lists, publish entries, stash metadata | typed with the same class, so they rewrite the same way (not each one run) | So "what happens if the vault creates that": another client, or the web UI, can put such a name in a vault. This CLI clones, edits and syncs it correctly. Its listings show the wrong name, and a scoped clone of that folder is silently empty. Hostile names in a vault are a separate question with an existing answer: `Vault__Path_Guard` refuses absolute paths, `..` on either separator, `.git` / `.sg_vault`, and writes through a symlink at checkout (TM-F series). Nothing in this finding weakens that. ## 3. The decision proposed for the next push **The rule:** a type that names a place must **refuse** what it cannot represent, never rewrite it. A rewritten path is a silent data change: the user asked for one place and got another. Worth a line in CLAUDE.md's Type_Safe rules. 1. **Vault roots: the alarm now, the real fix right behind it.** - *Stop-gap (S):* at the CLI boundary, before any command uses a vault root (`init`, `clone*`, and resolving the root of every other command), refuse a root the type would change: > `sgit cannot work in a folder whose path contains 'é' (…/josé/vault) yet. Move the vault, or clone it, to a path made of letters a-z, digits, space, '.', '_' and '-'. (TM-R36)` - *Fix (M):* type the host-side fields (`directory`, `vault_path`, the storage root) with `Safe_Str__Host_Path`. It already exists, added for `sgit serve`: any printable text, control characters refused, never rewritten. Then the alarm is no longer needed. 2. **Names inside a vault: do not refuse them.** They work today, other clients create them, and refusing would lock people out of their own vaults. That is the lock-out lesson of R1 and TM-F35. Instead: - a content-path type (Unicode allowed; NUL, control characters, absolute paths and `..` refused) for the display and scope schemas: `Schema__Diff_File`, history lists, `scope_paths`, publish entries, stash metadata; - `clone --path` refuses a folder that is not in the tree, rather than cloning nothing. 3. **Where an alarm is right for content: names that will not survive another OS.** Add a check (rule 9 puts it under `check`), and a warning at `commit` time for new names only (an alarm, not a refusal). It covers: - case-only collisions (`A.md` and `a.md` are one file on macOS and Windows); - Unicode normalisation twins (NFC and NFD `café`; macOS can hand back the other form, so status shows a delete and an add); - Windows-reserved names (`CON`, `AUX`, `NUL`, `COM1`…, a trailing dot or space, `<>:"|?*`); - control characters and newlines; - paths too long for Windows. 4. **Tests that land with the fix:** - init, commit, push, clone and pull under a root with `é`, `+`, `@`, `(` and a space; - clone into `josé/`; - `clone --path naïve`; - `diff` listing the exact names; - a `Fixed` security test that two roots that used to sanitise alike no longer share a store; - the check's cases. Sizes: the stop-gap is under a day. The type change is a day or two, because the type is used by every path field; the check is about a day. ## Where this is recorded - Threat model TM-R36: likelihood/impact raised, this evidence linked. - CHANGELOG `[Unreleased]` → "Known issue". - Probe scripts were run from the session scratchpad. Their steps are in the tables above. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/non-ascii-paths-refuse-dont-rewrite.html)* # One vault, ten agents, 35,000 objects: what sgit does when the repository is big and the clients are many, cli.sgit.ai > What sgit does when the repository is big and the clients are many: the comms vault, 35,000 encrypted objects and ten agents, cloned eight ways and measured from a cloud sandbox. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # One vault, ten agents, 35,000 objects: what sgit does when the repository is big and the clients are many *Draft for the newsroom, 16 Oct 2026. Companion to "Scion and sgit" (No. 141). Numbers from the response note in this folder; every one was measured on the live vault from a cloud sandbox, so a plain connection does better. Placeholders in [brackets] are filled from the measurement table.* Anthropic's post on Scion makes one observation that survives contact with any version control system: when the clients are agents, they are always online, there are a great many of them, and the expensive thing is moving bytes nobody needs. Scion answers with a server that does the work and three tiers of cache in front of it. This is what sgit does with the opposite constraint, a server that cannot read a byte, on the one vault we have that is big by our standards and busy every day. ## The vault The comms vault is where ten agents and one human keep their mail, briefs, runs and analysis. On 16 October it had 1,295 commits, 68 of them from the last two days, 5,101 files, 16,315 blobs and 16,391 trees: about 35,000 encrypted objects, 527 MB when every one of them is on disk. Its biggest folder is `mail` (3,200 files, 68 MB); `docs` has twelve files. It lives on the hosted server, S3 behind CloudFront, and the agents reach it through the CLI. ## What an agent fetches Nobody working in this vault needs all of it, and the point of the design is that they do not have to take it. | What was asked for | Time | Objects | On disk | |---|---|---|---| | Everything (full clone, 14 Oct) | 3½ min | 35,423 | 527 MB | | The whole tree, one commit of history (`--depth 1`) | 57 s | 6,248 | 142 MB | | One folder, one commit (`--path security --depth 1`, 130 files) | 13 s | 137 | 3 MB | | One folder, all its history (`--path security`) | 227 s | 3,364 | 64 MB | | The biggest folder, all its history (`--path mail`, 3,275 files) | 264 s | 25,110 | 274 MB | | Structure only, one commit (`--sparse --depth 1`), then one file on demand | 27 s + 2.9 s | 968 | 6 MB | | A headless clone, then one file (`clone-headless` + `cat`; the first `cat` brings all history down) | 0.4 s + 202 s, then 2.5 s a file | 18,300 | 116 MB | | Widening a scoped clone by a folder (`fetch security`, 130 files) | 7.8 s | +135 | +2 MB | | A pull when nothing changed | 2.5 s | 0 | | Two things stand out. A clone that carries a depth costs seconds and megabytes however large the vault is, because it asks for the spine of trees down to its folder and the blobs under it, and nothing else; this is the pattern the documentation gives agents, and the one the agents use. And a clone that carries the whole history of a folder pays for the history, not the folder: 227 seconds for 130 files, almost all of it walking 1,295 commits one request at a time, because a partial clone cannot do what a full clone does (list the store once and fetch everything in bulk) without holding objects it must not hold. That is the one real edge in the numbers, and the fix is a small encrypted object that records the commit chain, so a clone can fetch the chain in 26 batched requests instead of discovering it. Until it ships, the depth flag is the answer. ## What an agent does every session The command an agent runs most is not clone but pull: come back to the vault, take what changed, go on. Until 14 October that was the slow path on this vault, one request per commit and one per blob, and an agent a day behind was waiting most of an hour. It now fetches the whole head into an empty store in 186 seconds, blobs in batches of fifty with sixteen requests in flight, and a pull that is a few commits behind takes the few requests it needs. The agents reported the slowness; the vault they reported it on is the one it was fixed and measured on. ## One file, four requests The question Scion's API answers for jobs that never check out is "give me this file at this commit". In sgit that is four GETs computed from the read key: the ref, the commit, the trees on the path, the blob. No listing, no index, no account; the server could be GitHub Pages, and on sgit.ai it sometimes is. The site's documentation pages are a vault read exactly this way in the browser: a cold load fetches 18 objects, the next load fetches one 69-byte ref, and a load inside the two-minute freshness window fetches nothing, because every object but the ref is named by the hash of its ciphertext and carries `-imm-` in its name, so whoever fetched it can keep it for ever without reading it. The browser app is fast for exactly that reason: it caches every `-imm-` object on sight. The trade is that there are no deltas; a changed file is a new object. The CLI does the same from a sparse or headless clone, with one caveat the table shows: on a long history it walks the commits first, which is the same edge as above with the same fix. ## Where the server is, and what it is The same vault and the same client, with the objects moved to local disk, say where the time goes: | | Hosted (Lambda, S3, CloudFront) | Plain file server, local | Local API server, one process | |---|---|---|---| | Full clone | 3½ min | 111 s | 69 s | | `--depth 1` | 57 s | 25 s | 12 s | | One folder, one commit | 13 s | 2.3 s | 0.8 s | | One folder, all history (1,227 commits) | 181 s | | 7 s | | One file on demand | 2.5 s | 1.8 s | 1.8 s | The history walk that costs three minutes hosted costs seven seconds locally, because it is the same 1,227 requests at a millisecond each. What remains with no network at all is the client decrypting 16,000 trees, which is most of the 69 s. sgit is not the bottleneck; the distance to the storage is. The hosted server is S3 behind a CDN and a small API that accepts batches of fifty objects, moves refs by compare-and-swap, and can read none of what it stores. The same image runs on a laptop (`sgit serve`, in Docker, Podman, Apple's container runtime, or as a plain Python process), in a CI job as a sidecar, on a Mac mini next to the agents, in Kubernetes, or not at all: a published vault is plain files on any host that answers a GET. Ten scoped clones at once took 1.9 seconds on a single server process; ten full clones queued to 21 seconds, which is the single process, not the protocol, and the deployment note says so. ## What this does not claim - Not proven at Scion's scale. 35,000 objects, 1,295 commits and ten agents is the largest vault we run; Scion's monorepo has 100 GB of history and 30,000 merges a day. - No blame or bisect. Path history, signed commits, signed tags and verification exist; blame is a fold over path history that nobody has written yet. - A partial clone of a long history is slow without a depth, today. The fix is designed, not shipped. - One run each, from a sandbox whose every request goes through a proxy. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/one-vault-ten-agents-measured.html)* # Partial clones — a folder scope and a history depth (6 Oct 2026), cli.sgit.ai > Cloning just the folders an agent reads and writes, read and write, plus a shallow mode, without changing the full clone: the Merkle-spine idea and the measurements. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Partial clones — a folder scope and a history depth (6 Oct 2026) **Branch:** `claude/charming-babbage-5hzcxq` · **Vault measured:** `y8pwtjlw` (626 commits, 8,013+ trees, 9,404 blobs, 18,720 objects at HEAD time) **Ask:** agents that know which folders they read and write should be able to clone just those, read *and* write; plus a shallow (no-history) mode; and none of it may change the full clone. ## 1. The idea, in one paragraph The vault is a Merkle tree: a commit names a root tree, a tree names its children by id. To hold `mail/crm.riskmandate/` a clone needs the root tree, the `mail/` tree (the *spine*) and the folder's own subtree; everything else is an entry it keeps by id ("opaque"). To commit a change inside the folder, rebuild the folder's subtree from disk, then each tree on the spine with one entry replaced, and keep the siblings' ids — exactly what the whole-vault builder would produce, byte for byte, because tree encryption is deterministic (a test proves the rebuilt root id equals the full builder's). Push uploads the new commit, the spine trees, the changed subtree and the new blobs; the server needs nothing new. Two scoped clones in different folders cannot conflict: out-of-scope entries are always the remote's, by id. ## 2. What was built | piece | where | |---|---| | scope model (paths + boundaries, pure path logic) | `storage/Vault__Scope.py` (re-exported from `core/scope/`) | | opaque entries in the tree builder (`build`, `build_from_flat`, `opaque=`), canonical entry order | `storage/Vault__Sub_Tree.py` | | scoped flatten → (flat, opaque); level-by-level fetch walk that only opens held + spine folders | `storage/Vault__Scoped_Tree.py` | | local config: `scope_paths`, `shallow_boundaries` (empty on full clones) | `schemas/Schema__Local_Config.py` | | clone: `--depth N`, `--path F` (repeatable); depth-limited commit walk records boundaries; scoped tree walk; scoped checkout; bulk sweep skipped for partial clones | clone steps, `Vault__Sync__Clone`, `CLI__Main`/`CLI__Vault` | | commit / write_file: scoped flatten + opaque rebuild; writes outside scope refused with the path | `Vault__Sync__Commit` | | status: scoped diff; chain fetch stops at boundaries | `Vault__Sync__Status` | | pull: scoped fetch (held folders only), merge within scope with theirs' opaque entries, commit walk stops at boundaries (push's implicit pull used to deepen a shallow clone to the root) | `Step__Pull__Fetch_Missing`, `Step__Pull__Merge`, `Vault__Sync__Pull` | | push: scoped flatten, trees/blobs the clone never fetched are skipped (unchanged, on the server), cache reconcile skipped | `Vault__Sync__Push`, `Vault__Batch` | | widen (`sgit fetch `), unshallow (`sgit fetch --unshallow`), `require_whole` | `core/actions/scope/Vault__Sync__Scope.py` | | guard on `check fsck`, `dump`, `publish`, `vault move` | `cli/CLI__Scope_Guard.py` | | branch switch keeps scope fields | `Vault__Branch_Switch` | ## 3. Numbers on the real vault (this session, through its egress gateway) | clone | objects | store | files | wall | |---|---|---|---|---| | full (bulk sweep, unchanged) | 18,720 | 173 MB | 3,442 | 81 s | | `--path mail/crm.riskmandate --depth 1` | 429 | 3.2 MB | 365 | **14 s** | | `--path docs --path conductor --depth 1` | 20 | — | 15 | **5 s** | | widen the CRM clone with `docs` | +10 | — | +8 | 2 s | On the scoped clone: `status` in sync; `history log` shows HEAD and stops; `check fsck` refuses naming what the clone holds and how to widen. ## 4. "No side effects on the full clone" — how that was held - Every scope branch is behind `scope.is_scoped()` / `is_partial()`; a whole-vault clone takes the exact calls it took before (`flatten`, `build`, `build_from_flat`, `_fetch_missing_objects` without the new kwarg, `checkout`). The existing pull-step tests use a fake sync client with no crypto and no new kwargs, and they pass unchanged — which is the proof the full path is not touched. - `Vault__Sub_Tree.build*` with no opaque entries runs the old code; with them, the entry order is re-established to files-then-folders by name so ids match the whole-vault builder. - Real vault, before and after: a full clone gives the same 18,720 objects, empty scope and boundary in its config, and `fsck` reports the same 41 missing objects. - `pytest tests/unit/ -n auto`: 3,932 passed (21 new partial-clone tests on two real clones in the in-memory API: read, write, delete, refuse-outside-scope, write_file, fast-forward and three-way pulls with changes outside scope, two scoped agents never conflicting, dirty edit survives a pull, depth-1 round trips, unshallow, scoped+shallow, widen, guards, branch switch, and a full-clone-unchanged check; 8 storage tests on the scoped tree). ## 5. Caveats and follow-ups - A folder renamed by another agent silently leaves a clone's scope (names are decrypted along the spine). Pull should notice a held path vanishing and say so. Not done. - Scoped clones cannot see blobs missing elsewhere; the push fix from earlier today should ship with this (same branch). - `history log` on a shallow clone ends with "history stops here: shallow clone … sgit fetch --unshallow" instead of an `[object not found locally]` line. - A shared, immutable object cache across clones on one machine would make every agent's clone after the first a local copy — the browser-style cache idea; separate change. - `clone-branch` (thin) still walks all commits serially; with `--depth 1` on `sgit clone` there is little reason left to use it. ## 6. Deep review before the PR to main (same day, evening) Two passes over `origin/main...origin/dev` after the merge: a code review (bugs, side effects) and a security review (new exploitable issues only, >80 % confidence). CI on dev was green first (run 37544894784 on ddee8a4). ### Security review — no new vulnerability; three hardening gaps closed No high-confidence exploitable issue was introduced. Three pre-existing "trust the host / trust the committer" gaps that the new fetch paths exercise more were closed anyway: | Gap | Fix | Test | |---|---|---| | A batch read wrote whatever `file_id` the host answered with, and the id names the on-disk path | `_batch_read_chunk` keeps only requested ids | `Test_API__Ignores_Unrequested_File_Ids` (real local HTTP server answering with `local/config.json` and an extra object) | | `Vault__Verified_Write.save` accepted any contained path (`local/config.json`, a working-copy file) | store-leaf allow-list `STORE_FILE_ID` + protected-dir check, before the content-address check | `Test_Verified_Write__Store_Paths_Only` | | Pull let an untracked file be overwritten when the tree entry's `content_hash` CLAIMED the same content | proven from the decrypted blob (`blob_hash_fn`), the claim is ignored | `test_commit_whose_entry_hash_lies_cannot_overwrite_an_untracked_file` (a commit built with a lying hash) | ### Code review — 15 findings, what changed | # | Finding | Resolution | |---|---|---| | 1 | `Vault__Head_Paths` flattened the whole tree on a scoped clone → tracked-wins rule broke, a tracked file under an ignored dir was dropped by the next commit | scoped flatten; test | | 2 | read-only scoped clones: status/checkout used the whole tree | scoped in `_status_read_only` and the RO checkout step; test | | 3 | sparse push could try to load a blob it never fetched | collector skips any blob not local | | 4 | `sgit ls`/`fetch`/`cat` on a scoped clone walked a sibling's tree and failed | `_get_head_flat_map` scoped; test | | 5 | other commands crash with "vault may be corrupted — run fsck" on a partial clone | friendly error naming the scope + how to widen (generic, for every command); test | | 6 | branch-only push re-uploaded the whole clone-branch history each time | stops at the server's clone head; blobs the server has are not re-sent — and the ref CAS was matching the LOCAL ref bytes, so the ref write conflicted on every push after a commit (silent on the in-memory API, a 409 on the real one): now matches the server's bytes; test checks the server ref and that nothing re-sent is a blob | | 7 | status walked the whole local history on every call | walk stops at known-complete heads; test (0 object reads when up to date; N when N behind); the truncated-fetch case is covered by the existing lower-bound test | | 8 | widen overwrote files already on disk under the new folder | refuses naming the clash; held folders untouched; test | | 9 | one failing bulk-sweep chunk aborted the clone | fail-soft per chunk; test | | 10 | sweep counters unsynchronised across threads | lock + summed futures | | 11 | duplicate parents counted against the fetch limit | deduped | | 12 | `CLI__Scope_Guard` duplicated `require_whole` | delegates | | 13 | pull-step call shape | kept (the fake-client tests prove the full path is unchanged) | | 14 | guard re-read the config | config read once in the merge step, passed through | | 15 | `Vault__Scope` held raw `list[str]` | `Safe_Str__File_Path` / `Safe_Str__Commit_Id` with validation (`..`, absolute, non-object ids refused; nested folders collapse) | ### What the new tests then found (would have shipped otherwise) - **Typed boundary ids broke unshallow and the status boundary stop.** `Safe_Str` hashes differently from the equal `str` (`'x' in {Safe_Str('x')}` is False) and `os.path.join` sanitises it (`/` → `_`), so once the config fields became Safe types the boundary commit was "not in the local store" and `--unshallow` on a scoped shallow clone fetched nothing. `Vault__Scope.boundary_ids()` / `folders()` hand consumers plain strings; a test pins both behaviours. Worth remembering for every Safe_Str that reaches a path or a set. - The first status rewrite assumed "a local commit has local ancestors", which a truncated fetch (limit hit) violates; the walk now stops only at known-complete heads. - Branch-only push's ref CAS bug above (pre-existing, invisible to the in-memory API). `pytest tests/unit/ -n auto`: 3,953 passed (21 new review tests under `tests/unit/review/`). ### Follow-up (7 Oct): the merge of main into dev and a CI flake that was a real leak Merging `main` into `dev` for PR #7 conflicted only on the release bot's version bumps (dev v0.17.3, main v0.17.0; dev kept). CI on that merge then failed on `test_Dev__Tree__Graph__Happy::test_analyse_temp_dir_cleaned_up`, which diffed `/tmp` before and after `analyse()` and so could see any vault another xdist worker was creating at that moment. Rewriting it to watch the `mkdtemp` calls made inside the call exposed something real: every `sgit clone` (all five entry points) creates `/tmp/sgit-clone-*` as the workflow workspace's root, the runner removes the workspace inside it on success, and the root stays — one empty dir per clone, 3,881 of them in this sandbox. `_run_clone_workflow` now removes the root in its `finally`, success or failure; a review test clones three ways and checks none of the roots survive. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/partial-clones-scope-and-depth.html)* # Fix — `sgit pull` silently discarded uncommitted edits to tracked files, cli.sgit.ai > sgit pull silently discarded uncommitted edits to tracked files. The root cause, and the fix: git's rules for the working tree, applied before any write. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Fix — `sgit pull` silently discarded uncommitted edits to tracked files **Date:** 2026-10-03 **Branch:** `claude/charming-babbage-5hzcxq` (restarted from `dev` @ `2f86cf5` after PR #6 shipped as 0.17.0) **Bug report:** RiskMandate.ai agent team (@briefs), 3 Oct 2026 — hit three times in real use on a vault shared by ~10 agents. Reproduces on 0.16.0 and 0.17.0, so not a regression from the clone work. ## 1. Root cause `Step__Pull__Merge` applied the result of a fast-forward by calling `_checkout_flat_map(directory, theirs_map, …)` with the **whole** incoming tree, and the three-way path did the same with `merged_map`. Both maps are built from committed trees (clone HEAD, named HEAD, LCA), never from the working directory, so every tracked file was rewritten from its committed blob — including a file the user had edited and not committed, and including files the incoming commits never changed. `status` then compared the (now reverted) file with HEAD and found nothing to report: "fully in sync". The edit was simply gone. ## 2. The fix — git's rules, applied before any write `sgit_ai/core/actions/pull/Vault__Pull__Guard.py` (new) and `_guard_working_tree` in `Step__Pull__Merge`: 1. Scan the working tree with the same ignore-aware scanner `status` uses (`_scan_local_directory`) and classify each path against the clone HEAD tree: `modified` (content hash differs), `deleted` (tracked, absent on disk; on a sparse clone an unfetched blob is not "deleted"), `untracked` (on disk, not in HEAD). 2. For each dirty path, ask "does the merged tree's blob for it differ from HEAD's blob?": | working tree | merge leaves it as HEAD | merge changes it | |---|---|---| | modified | **carry over** — not written, edit survives | **refuse** | | deleted | **carry over** — stays deleted | **refuse** | | untracked, incoming creates same path | identical content → fine | **refuse** (would overwrite) | | untracked, not incoming | ignored | — | 3. Any refusal raises `Vault__Dirty_Working_Tree_Error` **before** `_checkout_flat_map`, `_remove_deleted_flat` or the clone-ref write, naming each path and why. The working tree, the clone ref and the object store are exactly as before the command. 4. Otherwise the checkout writes `merged_map` minus the carried-over paths. Removal (`_remove_deleted_flat`) only touches paths absent from the merged tree, so a carried file is never deleted either. CLI: the refusal prints as ``` error: your local changes would be overwritten by pull: x.txt (your uncommitted edit would be overwritten) commit them (sgit commit) or stash them (sgit vault stash) and pull again; nothing was changed ``` and a successful pull that carried edits adds ``` Kept 1 uncommitted change(s) the incoming commits did not touch: ~ y.txt (yours, uncommitted) ``` (`kept_dirty` in the pull result dict / `kept_dirty_files` on `Schema__Pull__State`). Three-way merges get the same guard; a conflict on a dirty path is "changed by the merge" and refuses, so no `.conflict` file is ever written over uncommitted work. ## 3. Tests (`tests/unit/workflow/pull/test_Pull__Dirty_Working_Tree.py`, two real clones on the in-memory API) - the reported case: Bob changes `x.txt`, Alice has an uncommitted `y.txt` → pull merges `x.txt`, keeps `y.txt`, status still shows `~ y.txt` - dirty `x.txt` while Bob changed `x.txt` and `docs/z.md` → refused, **neither** file changed, clone ref unchanged; after `commit` the same pull merges - locally deleted untouched file stays deleted; locally deleted file Bob changed → refused - untracked `new.txt` colliding with Bob's `new.txt` → refused; identical bytes → fine; unrelated untracked file left alone - clean tree: unchanged behaviour, `kept_dirty == []` - three-way: dirty untouched file survives; dirty file the other side changed → refused - unit tests of `plan()` rules, `dirty_paths()` semantics, the message - CLI: the error renders without the fsck/corruption hints; the `Kept` block renders ## 4. Minor issue 1 from the report — `status` said "200 ahead, 1 behind" Real, and I had seen it myself on a fresh clone. Two things were wrong in `status`: 1. It **overwrote the local named ref with the remote value first**, then counted. When the remote had moved, the local ref pointed at a commit the store did not have, the ancestor walk from it was empty, and `clone_walk − named_walk` was the entire local history — "N ahead" on a clone with no unpushed commits, plus the hint "push your own commits". Every later `status` walked from the same hole. 2. It had no way to get the missing commits. Now `status`: - reads the remote ref **without writing it**, and keeps the local ref as `last_known_named_head` — the remote head whose history this clone fully has (set by its last push/pull, or by a status that fetched completely); - walks the commit graph from the remote head, fetching the commit objects that are absent (one ~700-byte object per new commit, verified before write — the same path pull uses), bounded by `commit_fetch_limit` (default 50), and reports whether the chain **connects** to local history (a local head whose parents are missing is not connected — my first version used "head exists" and gave a short count; a test caught it); - advances the local named ref only when connected; - when connected: exact `ahead` / `behind`. When not (offline, or more than the limit of new commits): `ahead` = local commits not reachable from `last_known_named_head` (still exact — it never needed the network), `behind` = what was fetched, flagged `behind_lower_bound`, which the CLI prints as `50+ new commits`. Tests: fresh clone one behind → `(0, 1, 'behind')`; diverged 1/2 → `(1, 2, 'diverged')`; offline with one unpushed commit → `(1, 1+, 'diverged')`; three new commits with the limit at 1 → lower bound, then exact `(0, 3)` once the limit allows. Cost: the walk is bounded (2,000 visited; `_count_unique_commits` was already bounded at 200), and the fetch happens only for commits the store lacks, so a `status` on an up-to-date clone makes the same single ref read it made before. Minor issues 2 (doctor `whoami` on the dev server) and 3 (read key + wrong vault id message) are not touched here: the first is server-side, the second is a wording change on a path that already lists "the vault key or ID is incorrect" as cause 1. ## 6. Replication and side effects (asked after the first push) **The new tests reproduce the bug on the pre-fix code.** With the four changed source files checked out from `dev` and the new test file run unchanged: 10 of 17 fail, the reported case on `assert 'y v1' == 'y v2 (UNCOMMITTED in A)'` (the edit reverted), every refusal test on `DID NOT RAISE`, and the status tests on the old numbers — a fresh clone one behind reported `(2, 1, 'diverged')`. With the fix: 19 of 19 pass (two status tests added since). **Side effects of the pull guard, considered:** - *Pull can now refuse where it used to "succeed".* A script that edits a file, then pulls while another agent changed the same file, now exits 1 with the paths named instead of silently losing the edit. That is the intended change; an agent loop should commit (or `sgit vault stash`) before pulling, which is what git demands too. A `--autostash` flag would be the convenience to add if the teams want it. - *Cost:* the guard hashes the working tree exactly as `status` does. Measured on the 2,900-file DC vault clone: 393 ms for the 1,687 scanned files (13 MB). It runs only when a pull actually merges, never on an up-to-date pull. - *Carried-over edits after a fast-forward* are dirty against the new HEAD exactly as they were against the old one (same blob), so `status` shows them and the next `commit` includes them — the same as git. - *Three-way merges* create the merge commit from committed trees only; a carried-over edit stays uncommitted on top of it, as in git. A conflict on a dirty path refuses before any `.conflict` file is written. - *Sparse clones:* an unfetched tracked file is not "deleted" (same rule `status` uses), so sparse pulls behave as before; the sparse suite passes unchanged. - *Ignore rules:* the scan is the ignore-aware one `status` uses, with tracked paths exempt from ignore ("tracked wins"), so a tracked-but-ignorable file is never misread as deleted. - *Untracked collisions* now refuse (git does too); identical bytes pass. Previously the local file was overwritten silently. - *API surface:* `kept_dirty` is a new key in the pull result dict and `kept_dirty_files` a new field on `Schema__Pull__State`; both default empty, nothing reads them strictly. ## 5. What I could not do from here The report's `repro.sh` needs an access token for `dev.send.sgraph.ai`; this session has none, so the end-to-end check is the two-clone test above, which follows the script step for step. Running `repro.sh` against a release with this fix should end with `RESULT: OK, uncommitted edit kept`. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/pull-discarded-uncommitted-edits.html)* # Read and write keys as PKI — design analysis, cli.sgit.ai > Are the read key and the write key really a public and a private key? Half right: the write key should become a signing key, and the read key cannot become a public key. What that would take. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Read and write keys as PKI — design analysis **Date:** 2026-10-09 · **Context:** the 0.21.0 security review (threat model `team/explorer/appsec/threat-model/v0.21.0__threat-model.md`, TM-R01) · **Status:** analysis, no code changed. **Question.** sgit went symmetric because there was no good place to publish public keys and no good way to move secrets. Both now exist. So: aren't the read key and the write key really a public key and a private key? And since only the encrypt/decrypt part would change, how hard would supporting that be? ## Short answer **Half right, and the half that's right matters a lot.** - **The write key should be a private key.** Today it's a password the server compares, and nobody else can check it (TM-R01). If writing meant *signing with a private key*, every reader could verify every head, and a host or a read-key holder could no longer forge history. - **The read key cannot become a public key.** Reading means decrypting, and decrypting needs a secret shared by every reader. Public-key encryption works the other way round: anyone can encrypt *to* a key, and only the private-key holder can read. So content encryption stays symmetric. What the read capability gains is the writer's **public verification key**. It isn't secret, but it has to travel with the read key so nobody can substitute it. So the right mapping is: ``` write capability = content key + writer's PRIVATE signing key read capability = content key + writer's PUBLIC verification key host = ciphertext only; can no longer forge, only withhold or replay ``` This is a known, proven shape: Tahoe-LAFS mutable files (write-cap = signing key; read-cap = symmetric key + hash of the verifying key) and Hypercore/Dat (public key = identity and verification; secret key = write). **On "it's just the encrypt/decrypt bit":** the cryptography really is small. Content encryption, objects, trees and commits don't change at all, and there's no re-encryption or data migration. Only two mutable things need a signature: the branch **refs** and the **branch index**. Everything else is content-addressed below them, so signing the roots authenticates the whole Merkle tree. The real work is around the cryptography (§4). It's about 1.5–2 weeks for the CLI, and the web UI needs the same change. ## 1. What a writer signature fixes, and what it doesn't | Threat-model row | Today | With a writer signing key | |---|---|---| | **TM-R01** read-key holder + host forge history that verifies | open (by design) | **closed**: a forged ref or index fails verification on every client | | TM-R03 ref/key ciphertext not bound to its file id | accepted | **closed**: the signature covers vault id + file id + content | | TM-R27 legacy named private key readable | accepted | moot: that key stops mattering | | `signatures-required` meaning "some key that decrypts" | weak | meaningful: the signed index lists the trusted clone keys, so a commit signer must be one of them | | Tags, format gate, features (all in the index) | host/reader can rewrite | covered by the index signature, so stripping the gate is detected | | SP-14 whole-site substitution on static mirrors | open | **closed** for browsers that get the verification key in the URL fragment | | SP-2 / rollback for a *fresh* clone | open | **partly**: a signed timestamp lets a clone flag a stale head; the host can still serve an old signed state | | Withholding (availability) | open | open: signatures can't force the host to serve data | | Metadata to the host (TM-R08) | by design | unchanged | | Revoking one writer | needs a new passphrase | still needs one: the signing key is shared by all writers (§3, phase 3 changes that) | Per-clone commit signatures stay as they are. They answer *which* writer, while the vault signature answers *a writer at all*. ## 2. The one design detail that must be right **The signing key must not be derived from the write key.** The write key is sent to the server as a header on every write, so the server knows it. A signing key derived from it would be computable by the very party it must protect against. Deriving it from the read key fails for the same reason (read-only shares hold that). It has to come from the **passphrase**, along a third path: ``` passphrase ──PBKDF2(600k, salt "sgit-sign:")──► 32-byte seed ──► Ed25519 signing key └► verification key (public) ``` - Every teammate with the vault key derives the same signing key, so there's nothing to distribute among writers. The cost is one more PBKDF2 (~190 ms, cached like the others). - **Ed25519, not ECDSA P-256.** Ed25519 signatures are deterministic, so the browser byte-for-byte test vectors that CLAUDE.md requires are possible. ECDSA in Web Crypto is randomised, so it can only be verified, never matched. Web Crypto in current Chrome/Firefox/Safari imports an Ed25519 key from a 32-byte seed (PKCS#8) natively. Deriving a P-256 key from a seed would need a JS library, since Web Crypto can't compute the public point. `cryptography` 49 (installed) supports Ed25519 from a seed; checked. ## 3. Phases **Phase 1: vault writer signature (the TM-R01 fix).** CLI ~1.5–2 weeks. 1. **Derivation** of the signing key and verification key (§2), plus test vectors shared with the web team. 2. **Signed refs.** The encrypted ref JSON becomes `{commit_id, seq, ts, sig}`, with `sig` = Ed25519 over `(vault_id, file_id, commit_id, seq, ts)`. Clients older than this read `.get('commit_id')` and ignore the rest (checked). The signature is made in `Vault__Ref_Manager` / the push batch and verified wherever a remote ref is parsed: status, pull, clone, fetch. 3. **Signed index.** Add a `writer_sig` over the canonical (JCS) index minus the signature. `Schema__Branch_Index.from_json` already tolerates the extra field (checked). Verified in `Vault__Index_Sync.read_remote`; re-signed on every merge/upload (all writers can). Each branch's `public_key_id` gains a key **fingerprint**, so the signed index pins the clone keys and the key files can't be swapped. 4. **Downgrade resistance**, the part that's easy to get wrong: - Feature `writer-signed` plus `min_client`, so older clients can't write: they would re-save the index without the signature. - A clone that has seen the feature pins it locally, the way the format gate never goes down. - A **fresh** clone can't trust a feature flag the host could strip. So signed vaults get a new vault-key / share-token version (`sgit_private_vault_v2_…`), and a client that sees v2 requires signatures from the first byte. 5. **Read-only shares.** The read share carries the verification key (32 bytes) next to the read key. Existing read-only clones have no verification key: trust on first use, by pinning it from the first signed index and refusing any change after. 6. **Tests.** `Known_Gaps` TM-R01 and TM-R03 get inverted into `Fixed` tests (forgery and ref swapping refused). Add browser vectors and a real-server round trip. Existing vaults opt in (`sgit vault format --feature writer-signed`, which signs the current refs and index). No object is re-encrypted. **Phase 1b: server enforcement (SG/Send team, optional).** The server stores the verification key at vault creation and accepts ref/index writes only when they are signed. The bearer write key then stops mattering (a stolen header is useless), and garbage writes are refused at the door rather than by every client. **Phase 2: move capabilities with PKI, not passphrases.** CLI ~1 week. The other half of your question: now that public keys can be published, a vault key or a read capability can be wrapped to a teammate's published public key. `PKI__Crypto.hybrid_encrypt` (RSA-OAEP-4096 + AES-GCM, with an optional ECDSA signature) already exists, as do the `sgit pki` keyring and contacts. For example, `sgit share --to ` would write the wrapped capability where the recipient can find it. This removes passphrases from chat, argv and shell history (TM-R14), and onboarding never exposes the key to the host. **Phase 3: per-member writers and owner-signed membership.** ~3–4 weeks, later. The phase-1 key becomes the vault **owner** key. It signs a member list (each member's own signing key) inside the index, and a head must be signed by a listed member. Removing a writer becomes "owner drops them and re-wraps the content key to everyone else", with no new passphrase. This is the full answer to revocation, and it needs phase 1b to be enforceable against availability attacks. ## 4. Where the effort actually goes | Piece | Why it isn't "just the encrypt bit" | |---|---| | Key derivation | must avoid the write key and the read key (§2); new interop vectors | | Formats | refs and index carry a signature; old clients must read them (they do) but must not write them (format gate) | | Downgrade | a host strips signatures or the feature flag unless fresh clones learn "signed" from the capability itself | | Read shares | new token format; existing read-only clones need trust on first use | | Every write path | push, index sync, tags, branch create/switch, move, rekey, history edit all write refs or the index and must sign | | Web UI | same verification (and signing, if it writes); Ed25519 via Web Crypto | | Tests | gap proofs become fixed tests; vectors; real server | ## 5. Recommendation Do **phase 1** for the next release after 0.21.0. It closes the most serious row in the threat model (TM-R01), makes `signatures-required` mean what people think it means, needs no data migration, and is opt-in per vault. Do **phase 2** alongside it, since it's cheap with the PKI code that's already there and fixes how keys get shared. Ask the SG/Send team about **1b**. Keep **3** on the roadmap. Decisions needed from you before starting: 1. **Ed25519** (my recommendation, for deterministic vectors) or P-256. 2. **A new vault-key version for signed vaults** (fresh-clone downgrade protection), or opt-in by feature flag only (simpler, weaker for fresh clones). 3. **Whether the web UI writes to signed vaults** in the first version, or reads and verifies only. ## 6. Points to settle before phase 1 (from the sgit.ai agent's review of d3b8eef) 1. **Existing read keys.** A writer signature closes TM-R01 only for readers whose capability carries the verification key. Existing read-only shares, including the read keys sgit.ai publishes, fall back to trust on first use until they are reissued in the v2 form. Plan the reissue, and say so on the site. 2. **Replay.** Signatures do not stop a host from replaying an older signed index or ref. Phase 1 needs per-ref and per-index sequence numbers, pinned per clone and failing closed; otherwise TM-R29 survives. 3. **Old clients.** They, and any fallback that rewrites the index (the N5 path, now removed), would write it unsigned. Gate the change on `--min-client`. 4. **S10.** "A forged first ref fails verification anyway" is true only once phase 1 ships. Until then S10's per-file rule (only the server's `not_found` is "absent") is what protects a first push. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/read-write-keys-as-pki.html)* # Response to "Scion and sgit" (newsroom.sgit.ai, article No. 141, v1.0.0), cli.sgit.ai > The CLI team's response to the newsroom's "Scion and sgit" article: what it gets right, the three factual errors, and the measurements from a 35k-object vault the article should have carried. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Response to "Scion and sgit" (newsroom.sgit.ai, article No. 141, v1.0.0) **Date:** 16 Oct 2026. **Code read:** `dev` at 881d6df (what 0.22.0 ships plus the 14 Oct fixes). **Vault measured:** the comms / DC collaboration vault (`y8pwtjlw` on `dev.send.sgraph.ai`), read-only: cloned eight ways from the hosted API, then the same objects served from local disk (static and a local API server); nothing pushed to the real server. **Also measured:** the vault the article used (`newsroom-synthetic-users`, `bg1opz3c`, public read key), from the same kind of sandbox. **Also read:** Anthropic's post in full, every sgit.ai page the article should link, and the eleven newsroom articles that already describe sgit's architecture and deployment. ## Verdict The Scion half is accurate: every figure the article quotes is in Anthropic's post. The sgit half is under-claimed where it should be strong (deployment models, reading files without a clone, the pull fix, history integrity) and over-claimed in one place (a CDN that caches for a year), and its measurement is the wrong one: an 83-file vault says nothing about the question the post raises, which is what happens when the repository is big and the clients are many. We have a vault that is big by sgit's standards and is used by ten agents every day. Its numbers are below, and they are the ones the article should carry. Three things the article gets wrong on the facts, in order of how much they matter: 1. **"no packing on the server" is listed as a gap.** Scion has no packfiles either and says so as a feature: "There is no equivalent of packfiles … so no repository maintenance is needed." sgit is the same: immutable objects, no maintenance, no GC on the server. This is a point of agreement, not a gap. 2. **"its latest releases add scoped, shallow and on-demand clones" / "sgit 0.18 to 0.22 added … `--sparse`".** `--path` and `--depth` are 0.18.0 (7 Oct). `--sparse` has been in the CLI since v0.11.3 (1 May 2026). 3. **"objects cache for a year at a CDN".** Drop it (see §2, claim 1). The property is that they *can* be cached by anyone, for ever, and the evidence for that is what the sgit.ai pages do in the browser, not a CDN header. ## 1. What Scion actually says, and the overlaps the article missed Read against the post, the article's Scion paragraph is right (2,500 → 30,000 merges a day, ~100 GB of main-branch history, ~25 M CI checkouts a day, ~600 PB if fetched naively, several hundred mirrors, three cache tiers each over 99 %, 200× less CPU and memory, seconds instead of minutes, 25 % of CI's CPU, seven people in five months, "by no means a suitable replacement for git"). Four things in the post are closer to sgit than the article noticed: | Scion (from the post) | sgit | Where it is documented | |---|---|---| | "Scion's negotiation doesn't tell the server what the client already has; instead, it operates exclusively on trees and blobs the client wants. Starting from a root tree, the client checks its local store for missing trees and requests them recursively, then repeats the process for blobs." | This is sgit's clone and pull walk, word for word: level-by-level over the trees, one batched request per level, then the blobs in batches of 50 with 16 requests in flight. A scoped clone asks only for the spine down to its folders. | `docs/partial-clones`, CHANGELOG 0.18.0; `Step__Clone__Walk_Trees`, `Vault__Scoped_Tree.walk_fetch` | | "Every request is for a specific object that never changes, so any cache can answer it." | Every object's name is the SHA-256 of its ciphertext; the client checks every object against its name before writing it. The mutable ref is `no-store`. | `api/vault-objects` (the caching contract), `case-studies/live-vault-docs` | | "There is no equivalent of packfiles … no repository maintenance is needed." | Same. No packing, no GC, no maintenance on the server. For cold starts from a static host, `sgit publish --bundles` emits a head snapshot and per-commit delta zips. | CHANGELOG 0.18.0 (static publishing) | | "moving [a ref] is a compare-and-swap on a single Spanner row" | Refs move by compare-and-swap; so does the branch index, which every client merges as a shared document. | `docs/history-integrity`, `docs/vault/vault-from-javascript` §5 | And two differences the article should state, because they are the real ones: - **A Scion changeset stores the list of files that changed; an sgit commit stores a tree,** like git. Scion rebuilds history and blame as "derived data" on the server. sgit's tree snapshot is what makes a scoped clone cheap (an unchanged folder keeps its tree id across commits, so it is never fetched twice) and what makes a client-side blame possible without the server. - **Scion protects directories with permissions on the server; sgit protects them with keys.** A partial clone is a speed setting, not a boundary (`docs/limitations`); the boundary is one vault per trust domain, a read key that cannot be turned into a write key, and append lanes that can write but never read. The article says nothing about any of this, and it is the strongest contrast with a server that holds plaintext and an IAM system in front of it. ## 2. Claim by claim | # | The article says | Status | Say instead (evidence) | |---|---|---|---| | 1 | "Its object ids are immutable content addresses, so objects cache for a year at a CDN" (title paragraph, abstract) and "the live server already uses it: … `cache-control: public, max-age=31536000, immutable`, through a CDN" | **Half true, and the half that is true is the wrong half.** The header is now sent (verified today on `/api/vault/read/y8pwtjlw/bare/data/obj-cas-imm-…`: `public, max-age=31536000, immutable`; the ref answers `no-store`). But three consecutive reads of the same object all came back `x-cache: Miss from cloudfront`: the CDN in front of the dev API does not serve reads from cache today. | "Objects are named by the hash of their ciphertext and the name carries `-imm-`, so anything that has fetched one can keep it for ever without reading it: a browser's Cache API, a proxy, a CDN, a folder on disk. That is how the browser app is fast: it caches every `-imm-` object on sight and refetches only the ref. sgit.ai's own documentation pages read a vault that way: a cold load fetches 18 objects, the next load fetches one 69-byte ref, and a load inside the two-minute freshness window fetches nothing. The price is that sgit keeps no deltas: a changed file is a new object." Link `case-studies/live-vault-docs` and `docs/vault/reading-a-vault-file`. | | 2 | "measured it on a real vault: a scoped clone in 4 seconds against 59 for the full one" (12 MB, 83 files) | **Wrong vault, and the 59 s was a bug, now fixed.** 83 files answers nothing about Scion's problem. The 59 s reproduced exactly from here (59.0 s, 56.5 s): the vault's 106 objects go up in two batches of 50, each batch's response is over the Lambda's size limit, the client slept 14 s retrying each whole and then read 106 objects one at a time. A refused batch is now split in halves on the first refusal: 8.1 s and 9.9 s for the same clone (§3). The article measured a Lambda limit and a client fallback, not the vault. | Use the comms vault (§3): 35k objects, 1,295 commits, 5,281 files, ten agents writing daily; and say what the 59 s was. | | 3 | "sgit 0.18 to 0.22 added the same family of options: `--path` …, `--depth` …, and `--sparse`" | **Partly wrong.** `--sparse` is from v0.11.3 (May). 0.18.0 added `--path` and `--depth`; the 14 Oct fixes made `clone-headless` + `fetch`/`cat`/`ls` work. | "sgit has had on-demand (`--sparse`) clones since May; 0.18.0 added a folder scope and a history depth, and they combine." | | 4 | "writes … go through a batch endpoint, up to 100 operations at a time" | Server cap is 100; the client sends 50 per request with 16 requests in flight on reads. | "batches of 50 objects a request, 16 in flight". | | 5 | "a vault of 626 commits and 9,404 files" | 9,404 **blobs**; the file count was not recorded then (today's vault: 5,101 files, 16,315 blobs). | "9,404 blobs". | | 6 | "The CLI team measured a scoped clone of one folder in 14 seconds against 81" | True (6 Oct, 18,720 objects). | Keep, but it is superseded by §3. | | 7 | "sgit does no packing or compaction on the server" under *Where Scion is ahead* | **Not a gap** (§1). | Move to the overlaps. | | 8 | "a vault of 35,000 objects still takes about three and a half minutes to clone in full" | True (14 Oct), and it is the honest number for a *whole* vault. What the article leaves out: nobody doing agent work needs the whole vault, and the pull that an agent does every session was the slow path, fixed on 14 Oct (§3). | Keep the 3½ min and put the scoped/depth numbers and the pull fix next to it. | | 9 | "sgit has a history log on the client but no blame and no bisect" | True. Understated: `history log --file PATH` (path history), `--stat`, `--grep/--since/--until/--author`, `history diff A..B`, `history show`, `history revert --as-commit`, `commit --amend`, `history undo`, reflog, `vault tag` (signed), and `check verify` (which commit was signed by which key). Blame is a client-side fold over path history and nothing in the format prevents it. | "No blame or bisect yet; path history, signed authorship and signed tags, yes." Link `docs/history-integrity`, `docs/update-to-0-21-0`. | | 10 | "There is no command to read a file from a vault without at least a headless local clone" | True for the CLI, and misleading: reading one file is four GETs by design (ref → commit → trees on the path → blob), which the site's pages do in the browser with a 90-line reader, and `clone-headless` is a config directory, not data. The real edge is in §3: on a long history today's `cat` first walks the commits. | "A file is four GETs away from its read key, and sgit.ai's pages read vaults that way; the CLI needs a headless clone directory first." Link `docs/vault/reading-a-vault-file`, `docs/vault/vault-from-javascript`, `demos/vault-app-embed`. | | 11 | "Merges happen on the client, three-way and whole-file; Scion does them on the server" | True. Add: two independent clients (the CLI and the browser vault app) speak one wire format, so a merge made in a browser is one the CLI verifies and vice versa (`sgit serve pull`). | Link `docs/what-is-sgit` (three doors), `security/` (two implementations, one wire format). | | 12 | "Git and sgit also live side by side rather than in sync … no bridge" | True for history. There is `vault attach` (bind a key to a `.sg_vault/bare` that came in through `git clone`), the one-repo pattern, and a pure-Python git reader for vault apps. | Link `docs/vault/git-and-vaults`, `case-studies/one-tree-two-remotes`. | | 13 | "The server only matters for writes" | Also for the store listing that makes a cold full clone fast (one `list`, then bulk fetch). A static host has no listing, so a cold full clone from one walks the graph; a scoped or depth clone never lists. | Fine as a simplification; say "writes and the branch index". | | 14 | Side-by-side figure "Scion, sgit 0.22.0 and Git … proven scale and availability" | Add rows: *server holds plaintext* (Scion yes, sgit never), *directory protection* (Scion IAM, sgit keys per vault and read-only / write-only keys), *clients* (Scion one, sgit two implementations), *deployment* (Scion internal fleet; sgit hosted, your container, static host, a folder). | | | 15 | "What it means … The useful next steps are clear from Scion's list: server-side history queries that do not need the plaintext" | A server that cannot read commits cannot answer history questions; that is the design, not a roadmap item. The sgit answer is a client-side index over a *scoped* clone, which §3 shows is small. | Reword: "history queries over a scoped clone on the client, and a commit-chain object so a partial clone does not walk history one commit at a time". | | 16 | *Where this comes from*: "the largest vault sgit's team documents has about 35,000 objects" | Make it concrete: it is a live vault with ten agents, 1,295 commits (68 added in the last two days), 5,101 files, 527 MB of ciphertext in a full clone. | | ## 3. The evidence the article should carry All timings from one client in this cloud sandbox (every request leaves through a proxy; a plain connection does better), against `dev.send.sgraph.ai` (S3 storage behind CloudFront), sgit at `dev` 881d6df (0.22.0 plus the 14 Oct fixes), one run each unless stated. The vault: 1,295 commits on 16 Oct (1,227 on 14 Oct), 16,391 trees, 16,315 blobs, 5,101 files, 35,423 objects / 527 MB in a full clone on 14 Oct. **Clones** (the full clone is the 14 Oct run; the vault has grown by 180 files since): | Clone | Time | Objects | Store on disk | Files checked out | |---|---|---|---|---| | everything (full clone) | 3 min 30 s | 35,423 | 527 MB | 5,101 (131 MB) | | `--depth 1` (whole tree, one commit) | 57 s | 6,248 | 142 MB | 5,281 (139 MB) | | `--path security --depth 1` | **13 s** | **137** | **3 MB** | 130 (3 MB) | | `--sparse --depth 1` (structure only, one commit) | 27 s | 967 | 6 MB | 0 | | `--path docs` (12 files, all history) | 181 s | 2,668 | 17 MB | 12 (1 MB) | | `--path security` (all history) | 227 s | 3,364 | 64 MB | 130 (3 MB) | | `--path mail/crm.riskmandate` (all history) | 159 s | 6,109 | 69 MB | 558 (27 MB) | | `--path mail` (the biggest folder, all history) | 264 s | 25,110 | 274 MB | 3,275 (69 MB) | | `--sparse` (every commit and tree, no blobs) | 206 s | 18,299 | 117 MB | 0 | | `clone-headless` (keys and config only) | 0.4 s | | | 0 | **Working in a partial clone** (each is one CLI invocation, so ~1.5 s of it is process start and PBKDF2): | Operation | Time | Network | |---|---|---| | `ls docs` in the sparse clone | 1.8 s | none (trees are local) | | `fetch docs/drafting-guide.md` into the sparse clone | 2.9 s | 1 object | | `cat` of a held file | 1.7 s | none | | `cat` of a file not yet held (fetched on demand) | 2.5 s | 1 object | | `fetch security/` (130 files) into the sparse clone | 5.1 s | 130 objects, batched | | `fetch security` to widen the `--path docs` clone | 7.8 s | 135 objects | | `status` in the scoped clone, up to date | 1.5 s | ref + index | | `pull` in the scoped clone, up to date | 2.5 s | ref + index | | `cat` of one file from a headless clone (first command: brings every commit and tree down) | **202 s** | 18,300 objects, 116 MB | | `cat` of a second file from the same headless clone | 2.5 s | 1 object | | `history log --file ` on the full clone (1,227 commits) | **6 min 8 s** | none: CPU, 5½ min of it | **The article's own vault** (`newsroom-synthetic-users`, 83 files, 12 MB, public read key), cloned from this sandbox for comparison with its "59 s full / 4 s scoped": | Clone | Before the fix (0.22.0) | After (`dev`, batch splitting) | Objects / store | |---|---|---|---| | full, read-only | 59.0 s, 56.5 s | **8.1 s, 9.9 s** | 106 / 12.4 MB | | `--depth 1` | 62.1 s | (same path) | 93 / 12.0 MB | | `--sparse` | 4.0 s | | 18 / 180 KB | | `--path ` (5 files) | 3.7 s | | 15 / 412 KB | The article's numbers reproduce to the second. The full clone's 59 s was two 50-object batches whose responses exceeded the Lambda's size limit (files of ~150 KB): the client retried each whole through the 2 + 4 + 8 s back-off and then read 106 objects one per request. Fixed on `dev` (CHANGELOG, Unreleased): a refused chunk is split in halves on the first refusal. ### What the numbers say - **A scoped clone of a 35k-object vault is a few MB and a few seconds, when it also carries a depth.** `--path security --depth 1` and `--depth 1` are the agent pattern the docs already recommend ("an agent clones only its own folder with one commit of history"), and they do not grow with the vault's history. - **Without a depth, a partial clone pays one round trip per commit.** `--path docs` (12 files) took 181 s and 2,668 objects because it walked 1,295 commits one request each, then the spine trees of every one of them (that is what gives `history log` on a scoped clone); `--sparse` without a depth is the same walk (206 s), `clone-headless` + `cat` pays it on the first `cat` (202 s, then 2.5 s a file), and `--path mail` with all its history is most of the vault (25,110 objects). Whole-vault clones and pulls stopped doing this on 6 and 14 Oct (list the store once, fetch in bulk); a partial clone cannot list-and-sweep because it must not hold what it does not track. This is the one real performance edge today, and it has a clean fix that needs no server change: a small encrypted "commit chain" object written on push, so a clone batch-fetches the chain (1,295 commits is 26 requests, 16 in flight) instead of discovering it. Until then: `--depth`. - **Once a clone exists, every operation is a few requests.** A file on demand is one object and under 3 s including process start; a folder of 130 files is 5 s; a pull that is up to date is 2.5 s. This is the "fetch only what you need" number, and it does not depend on the vault's size. - **History queries on the client are the other honest edge.** `history log --file` over 1,227 commits took 6 min of CPU on the full clone because it flattens every commit's tree; it should skip subtrees whose id did not change, the way the clone walk does. The fix is local and needs no format change. - **The pull an agent runs every session was the slow path, and is fixed.** Fetching the whole head into an empty store through the pull path: 186 s now, about 53 min before 14 Oct (one request per commit and one per blob). Measured on this vault, in the 14 Oct note. - **A full clone is the worst case and still bounded:** 3½ min for 35,423 objects / 527 MB (listing 21 s, bulk sweep ~140 s, local tree decrypt 33 s, checkout 4 s). - **Server-side numbers already published:** ten scoped clones at once in 1.9 s against ten full clones in 21.1 s on a single server process (the single uvicorn worker is the limit, not the protocol); a one-folder clone of 0.59 s at 100 files and 0.77 s at 2,000 while the full clone went 0.95 → 5.55 s; 100 writers landing 1,000 append-lane entries in 4.9 s with none lost (newsroom: *Agent workflows on ephemeral vaults*, *sgit in GitHub Actions, side by side*). ## 4. The argument the article should make Scion's post names what matters at agent scale: always-online clients, fetch only what a job needs, immutable objects any cache can serve, cheap incremental commits, a sparse working copy by design. Point by point, with the mechanism and the number: 1. **Fetch only what you need.** A clone asks for the spine down to its folders and nothing else; unchanged folders keep their tree ids across commits, so they are never fetched twice. §3. 2. **One file is four requests, from any host that answers a GET.** Ref → commit → trees on the path → blob, computed from the read key; no listing, no index, no account. The site's own pages do it; `sgit cat` on a sparse clone does it (and, once a file is held, reads it with no network). 3. **Immutable objects, cached by whoever fetched them.** Browser Cache API on sgit.ai, a CDN, `--transport static` from GitHub Pages or S3 or a folder, `sgit vault mirror` for custody without access. The server sends `immutable`; the design does not depend on it. 4. **Writes are one batched request and a compare-and-swap.** 50 objects per request; the ref and the branch index move by CAS, and every client merges the index as a shared document. A commit costs the size of the change: the new blobs, the rebuilt spine, one commit object. 5. **Many clients, a dumb server.** The server computes nothing, so it scales as storage does (S3 behind CloudFront is the hosted deployment; `sgit serve` is the same image on a laptop). Append lanes let a hundred writers post without reading. 6. **And the thing Scion cannot offer anyone outside Anthropic:** the server never holds plaintext, the read key is one-way derived and publishable, and the whole thing is `pip install sgit-ai`. Said with the gaps: no blame or bisect yet; partial clones walk history per commit until the commit-chain object exists; one server process queues full clones; proven at 35k objects and ten agents, not at 100 GB and 30,000 merges a day. ## 5. Deployment models the article should claim (all shipped, all documented) | Model | What runs | Docs | |---|---|---| | Hosted | `dev.send.sgraph.ai`: S3 storage behind CloudFront, access token per account | `/docs/installation`, `/api/` | | Your own server, one container | `diniscruz/sg-send-vault` with an access token and a folder or an S3 bucket; Docker Compose on a Mac mini; Kubernetes; a cloud VM with no public route; two servers, one vault | `/docs/self-host-for-agents`, `/deploy/`, newsroom *Encrypted memory for agents that run somewhere else* (five patterns) | | On the laptop, from the clone | `sgit serve`: the vault UI + API in Docker, Podman, Apple `container`, nerdctl, Finch, or `uv` with no container; `serve pull/push` move work between the browser and the clone | CHANGELOG (Unreleased, `sgit serve`) | | Static host, no server | `sgit publish` + any GET host (GitHub Pages, S3, a CDN, Python's file server); clients use `--transport static`; `sgit serve --static` locally | `/docs/vault/static-hosting`, CHANGELOG 0.18.0, newsroom *The homepage is a vault* (317 ms a request via the API, 44 ms from GitHub Pages) | | A folder | `--transport local` | CHANGELOG 0.18.0 | | CI sidecar | the server as a job sidecar; ciphertext in the cache or a branch between runs; a key per job; append lanes per shard | newsroom *sgit in a CI pipeline*, *sgit in GitHub Actions, side by side* (four lanes) | | In the browser | the vault app, `vault-embed.js`, and pages on sgit.ai and newsroom.sgit.ai that render a vault with a published read key | `/docs/vault/vault-apps`, `/docs/vault/vault-from-javascript`, `/demos/vault-app-embed`, `/case-studies/live-vault-docs`, newsroom *Two agents, one vault*, *No server, by design* | ## 6. Website changes (sgit.ai) 1. **`/api/vault-objects` → "Measured 21 September 2026: the immutable header is not currently being sent."** Stale. Measured 16 Oct: `cache-control: public, max-age=31536000, immutable` on `obj-cas-imm-*`, `no-store` on `ref-pid-muw-*`. Add: CloudFront still reports a miss on repeated reads of the same object (the token header is likely in the cache key), so the CDN is not yet a cache for API reads; browser and client caches are. 2. **`/docs/partial-clones` → "Measured".** Add the §3 table, and a sentence under `--depth`: on a partial clone the history walk is one request per commit (a scoped clone of a 1,295-commit vault: 181 s without a depth, seconds with one); whole-vault clones and pulls list the store and fetch in bulk since 0.18 / 0.22. 3. **`/docs/limitations` → "Commit author attribution is not yet recorded … population is planned work."** Stale since 0.19.0: commits are signed, `check verify` classifies every commit, `history log --author` filters by key id, `vault tag` is signed. Replace with the current edges: no blame or bisect, partial clones walk history per commit, no rebase or cherry-pick. 4. **`/llms.txt` quick answer "No for: … tags"** and `/docs/sgit-for-git-users`: tags exist (`sgit vault tag create/show/list/delete`, signed). Also add `history undo`, `commit --amend`, `history revert --as-commit`, `reflog`. 5. **`/docs/self-host-for-agents` §1:** `pip install sgraph-ai-app-send "mcp<2"` → 0.34.0 caps `mcp` itself; image `v0.33.0` → `v0.34.0` in the three places; the troubleshooting row for `mcp` 2.x can go once 0.34.0 is the floor. 6. **`/deploy/`:** the `--endpoint` → `--base-url` correction note can be folded into the vault pages themselves (they are a vault: push the fix). 7. **A single "deployment models" table** (§5) somewhere under `/deploy/` or `/docs/`: today the seven models are spread over six pages and two newsroom articles, which is why the article claimed none of them. 8. **`/docs/vault/reading-a-vault-file`:** add the CLI equivalent (`clone-headless` + `cat`, and `clone --sparse` + `cat`/`fetch`) next to the browser reader, with the history-walk caveat. ## 7. Links for the newsroom agent sgit.ai: `/docs/partial-clones`, `/docs/agents-sharing-one-vault`, `/docs/self-host-for-agents`, `/deploy/`, `/docs/vault/static-hosting`, `/docs/vault/reading-a-vault-file`, `/docs/vault/vault-from-javascript`, `/docs/vault/vault-apps`, `/demos/vault-app-embed`, `/case-studies/live-vault-docs`, `/api/vault-objects`, `/docs/history-integrity`, `/docs/two-branch-model`, `/docs/update-to-0-21-0`, `/docs/limitations`, `/docs/sgit-for-git-users`, `/security/`, `/docs/vault/git-and-vaults`, `/case-studies/one-tree-two-remotes`, `/use-cases/ai-agents`, `/docs/surfaces`, `/docs/what-is-sgit`, `/demos/vaults/llms.txt`. newsroom.sgit.ai articles: *Encrypted memory for agents that run somewhere else*, *sgit in a CI pipeline*, *sgit in GitHub Actions, side by side*, *Agent workflows on ephemeral vaults*, *No server, by design*, *The homepage is a vault*, *Two agents, one vault*, *The vault is live*, *Where the vault keys live*, *From sending a file to sgit*, *Git for things you cannot put on GitHub*. Repo: `CHANGELOG.md` (0.18.0 partial clones and static publishing; 0.19.0 history integrity; 0.21.0 history commands; Unreleased: `sgit serve`, the 14 Oct pull fix), and the 14 Oct note `team/humans/dinis_cruz/claude-code-web/10/14/response__sgit-issues-2026-10-14.md`. ## 8. Three paragraphs the article can paste **Replacing "Immutable identifiers that cache":** Every object in an sgit vault is named by the SHA-256 of its ciphertext, so a name can never point at different bytes, and the client checks each object against its name before it is written. That is the property Scion's three cache tiers rest on, and it means anyone who has fetched an object can keep it for ever: a browser's cache, a proxy, a CDN, a folder. sgit.ai's own documentation pages are a vault read this way: a cold load fetches 18 objects, the next load fetches one 69-byte ref, and a load inside the two-minute freshness window fetches nothing at all. The only mutable object is the ref, and it is never cached. **Replacing "Measured on 0.22.0":** The vault this site's agent should have measured is the one ten agents share every day: 1,295 commits, 5,101 files, 35,000 encrypted objects, 527 MB in a full clone. [§3 table]. A full clone is the worst case at three and a half minutes. An agent does not take one: it clones its own folder with one commit of history, in [path-security-depth1] seconds and [MB], and a pull that is a day behind, which used to be the slow path at nearly an hour, now takes three minutes for the whole head. One file is four requests from its read key, from any host that answers a GET. **Replacing "Where Scion is ahead", first sentences:** Scion does blame, bisect and merges on the server and builds history indexes as derived data; sgit's server cannot read a commit, so those run on the client, and today sgit has path history, signed authorship and signed tags but no blame or bisect. Neither system packs objects on the server, and both treat that as a feature. Where sgit is behind on its own terms: a partial clone still walks a long history one commit per request (a depth avoids it), one server process queues concurrent full clones, and the largest vault it has been proven on is 35,000 objects and ten agents, not 100 GB and 30,000 merges a day. ## 9. Storage is the bottleneck, not sgit The hosted API is a Lambda in front of S3 behind CloudFront. Every request costs its round trip (sandbox → proxy → CDN → Lambda → S3) before a byte moves, and the client's work per object (verify the hash, decrypt) is microseconds. So on the hosted path the numbers are request counts, not bytes: `--path security --depth 1` is 137 objects and 3 MB, and takes 13 s because it is about a dozen dependent requests (ref, index, commit, one batch per tree level, the blob batches) plus 0.4 s of PBKDF2. The newsroom measured the same thing from the browser: 317 ms a request through the vault API, 44 ms for the same object from GitHub Pages (*The homepage is a vault*). Take the Lambda away and the same objects move at the speed of the storage: - **ruv.sgit.ai ledger** (local SG/Send server, one process, disk storage, 11 recorded runs): every sgit operation, including Python start-up and the 600k-iteration PBKDF2, is 0.4 to 1.5 s; a 38-file / 4.0 MB vault clones in 0.63 s; a 142-file / 6.9 MB vault clones in 0.74 s and pushes in 0.87 s. Not one of those numbers is sgit being slow; they are the floor of starting a Python process and deriving a key. - **newsroom, local server:** a one-folder clone 0.59 s at 100 files, 0.77 s at 2,000; ten scoped clones at once in 1.9 s. - **The comms vault from local storage** (same 35k objects, this sandbox): the same sandbox, the same CLI, the same 35,423 objects, three transports: | Clone / operation | Hosted API (Lambda + S3 + CloudFront, via proxy) | Static: `sgit serve --static` (plain GETs, local disk) | Local API: `sgit serve --runtime uv` (one Python process, disk) | |---|---|---|---| | full clone | 3 min 30 s | 110.6 s (bulk fetch 71.5 s, tree decrypt ~33 s) | 69.1 s (commits 1.4 s, trees 40.7 s, blobs 18.8 s, checkout 2.7 s) | | `--depth 1` | 57.4 s | 25.4 s | 11.7 s | | `--path security --depth 1` (120 files) | 13.0 s | 2.3 s | **0.8 s** | | `--path docs`, all 1,227 commits | 181 s | | 7.1 s | | `--sparse`, all history | 206 s | 67.0 s | 43.3 s | | `cat` of one file on demand | 2.5 s | 1.8 s | 1.8 s | | `pull`, up to date | 2.5 s | | 0.8 s | Read down the columns: the per-commit history walk that costs 181 s against the hosted API is 7 s against a local server (the same 1,227 requests at ~1 ms each); a one-folder clone is 0.8 s. What is left when the network is gone is the client's own work, which is decrypting 16,391 trees (about 33 to 40 s on a full clone) and nothing else of note. (Observation for the SG/Send brief: the local disk server answered the store listing with 0 objects, so the full clone there went by the graph walk rather than the bulk sweep; the walk found everything.) The API does two things that storage cannot: it protects writes (the access token, compare-and-swap on the ref and the branch index) and it shortens reads a little (batches of 50, a listing for cold full clones). Both are small, and both can be done differently on different storage. Everything else is a GET of an immutable object. So "how fast is sgit" is the wrong question; the right one is "where is the storage, and how close": memory, a local disk, an S3 bucket behind a CDN, GitHub Pages, a folder. The deployment table in §5 is that list. ## 10. sgit is a standard, not a server The article treats sgit as a product with a server. It is closer to a convention for laying files out, and the proof is that everything in a vault is reachable from one hash with no help from anyone: 1. The vault key gives the vault id and, one way, the read key. 2. The read key gives the ref's name: `ref-pid-muw-` + the first 12 hex of `HMAC(read_key, "sg-vault-v1:file-id:ref:" + vault_id)`. Nobody told you; you computed it. 3. The ref holds the head commit id; the commit holds the tree id; trees hold blob ids and encrypted names. Ref → commit → tree → blob is four GETs for any file, from any host that serves the paths. 4. Every object but the ref and the index is named by the hash of its ciphertext, so it is immutable, verifiable and cacheable by whoever holds it. And the name says so: an immutable object's id carries `-imm-` (`obj-cas-imm-…`), the mutable ones carry `-muw-` (`ref-pid-muw-…`). Anything in the path, a browser, a proxy, a CDN, a CI cache, a vault app, can decide to keep an object for ever by looking at its name, without reading a byte of it, and without being told by a header. That is how the browser vault app and the sgit.ai pages get their speed: they cache every `-imm-` object on first sight and refetch only the ref. It is also the trade sgit made on purpose: no deltas between versions of a file (a changed file is a new immutable object), in exchange for objects that any intermediary can cache blindly. Git stores deltas and must be told what changed; sgit stores whole objects and never has to ask. 5. Writes are the same objects PUT in batches, plus a compare-and-swap on the ref. That is the whole wire format, and two independent implementations speak it (the CLI; the browser vault app, which `vault-embed.js` reduces to about 90 lines). A static host implements the read half with nothing installed. The **cache layer** (`sgit cache add `, shipped) extends the same idea to hot records: a declared path gets a client-computable cache object under `bare/cache/`, maintained by push, so a reader with only `vault_id` + read key does *one* batch read (the ref plus the candidate ids) and gets the file with no tree walk; a miss or a stale entry falls back to the walk, so correctness never depends on it. The reference reader is written to run in an edge function or a Lambda that serves one hot record. Scion's "derived data" is the same move, computed on a server that can read the plaintext; sgit's is computed by the writer and named by the key. ## 11. History, blame and merges: where they belong Correct the article's framing, and then disagree with Scion's. In sgit, history (`log`, `--file`, `--stat`, `diff`, `show`, `reflog`), undo, revert, tags, signature verification and three-way merges all run on the client, and that is the design, not a shortfall. Blame is the one Scion feature sgit does not have as a command today; it is a fold over path history and nothing in the format prevents it. (The honest edge: on this vault `history log --file` over 1,295 commits took 6 min 8 s (5½ min of CPU), because it flattens every commit's tree; it should skip subtrees whose id did not change, the way the clone walk does.) The argument for the client: a merge is a workflow step, not a storage operation. It needs the context of the change, the history, and the policy that says who may merge what. In ruv.sgit every worker writes to its own branch and the host merges them per the run's mandate, with a receipt; in the comms vault ten agents meet on a named branch and resolve on the clone that has the conflict in front of it. A server that merges for you must be trusted with the plaintext and with the policy. Scion's own post concedes the point from the other side: it could not keep a merge queue, so "concurrent changes can conflict, but Claude can typically fix these conflicts" — the merge went back to the agent. And keep what made git git. Scion is in the Piper and Sapling line, a centralised system with one authoritative store, and the post says plainly it "is by no means a suitable replacement for git for general use". sgit keeps the distributed property: a full clone is still one command and a complete, verifiable copy of everything, and anyone with the key can rebuild the whole thing from any host. What changed is that the full clone became optional. `--path`, `--depth`, `--sparse` and the cache layer mean a repository can hold large files without every client paying for them, which is the one habit every git team has had to police by hand. The whole-history clone is a backup and an audit, and it is still there when you want it. ## 12. Encryption as a constraint The article sells encryption as privacy. The stronger claim is that it is a design constraint that forbids the shortcut. Because the server cannot read a byte, there is no "just index it on the server", no "just render it there", no server-side merge, no derived data the client did not compute. Every feature had to be expressible as objects the client writes and other clients can derive by key, which is why the results are small: a read is four GETs, a cache is a path-derived object, a deployment is any storage. The newsroom's *No server, by design* is the same argument applied to a whole site. Say it as a property, not a scolding: the constraint is what produced the simple designs. ## 13. ruv.sgit, the integration the article does not mention ruv.sgit.ai runs a Ruflo hive (rUv / ruvnet; a queen and parallel workers) with one ephemeral sgit vault per run and an Agent Behaviour Policy compiled into Ruflo's enforce mode. The ledger (`data/vaults.json`, 11 runs to 15 Oct) records every sgit operation with its milliseconds, the server-held bytes, a plaintext probe of the server's store (0 hits in every run), a fresh clone checked identical to what was delivered, and the vault deleted at close. It is the agent-swarm case Scion's post is about, run on a local server, and it is where the "storage, not sgit" numbers in §9 come from. Link it as the swarm evidence, next to *Agent workflows on ephemeral vaults*. ## 14. What was not done - Nothing pushed to the comms vault; nothing changed on the site. The site changes in §6 are a list for the site agent. - The commit-chain object (§3) is a design, not code. It is the one change that would make partial clones of long histories as fast as the depth-1 case; it needs a format note and a migration-free write path (a mutable, encrypted, per-branch object the client rewrites on push). --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/response-to-scion-and-sgit.html)* # Sealed files — a second encryption layer on top of a vault, cli.sgit.ai > A second encryption layer on top of either vault mode: files sealed to named recipients with ECDH P-256, HKDF and AES-256-GCM, with the corrections from the sgit.ai agent's review. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Sealed files — a second encryption layer on top of a vault **Date:** 2026-10-09 · **Status:** design for review, no code · **Companion:** [`design__native-pki-vault-mode.md`](../articles/native-pki-vault-mode.md) (same folder). This layer works on **either** vault mode, symmetric (today) or native PKI. > **Revised 2026-10-10 after the sgit.ai agent's review (dev at 0a0707d).** Three corrections, > applied below: > 1. **No "compare by re-sealing".** Sealing is randomised (a fresh file key and ephemeral key > every time), so re-sealing never reproduces the stored bytes. The tree entry instead > carries a **keyed hash of the plaintext**: HMAC-SHA256 under a key derived from the vault > read key, stored under the read key. A recipient compares plaintext to that HMAC. A > non-recipient compares the sealed bytes. Unlike the unkeyed `content_hash`, the keyed > hash does not let a vault reader confirm a guess of the content. > 2. **Seal-policy changes need an authorised signer.** Otherwise any writer can "unseal by > policy": drop a rule or a recipient, and the next commit of that path is sealed to fewer > people, or not at all. `.sgit/seal` names its `admins` (public keys). A commit that > changes the policy must be signed by one of them, or pull refuses it, as > `signatures-required` does. The first policy is pinned by the clone that adds it (trust > on first use, like the PKI mode's writer set). > 3. **The construction is Web-Crypto-native: ECDH P-256 + HKDF-SHA256 + AES-256-GCM.** That > is the HPKE base mode with the P-256 KEM (RFC 9180, DHKEM(P-256, HKDF-SHA256)). Browsers > open sealed files with no WASM dependency, and CLAUDE.md's byte-for-byte parity rule holds > with test vectors. age is kept as an **interop path only**: > - `sgit seal export --age` re-seals a file for the stock `age` tool; > - hardware and KMS keys are reached through a provider bridge that speaks the > `age-plugin` protocol for the unwrap step only (only the file key crosses it). > §3 and §9 below are updated accordingly. > **Revised again 2026-10-10 after the release-gate review (dev at eed8084).** Two more > corrections: > 1. **The comparison value moves inside the sealed envelope.** A keyed hash under a key > derived from the *vault read key* (correction 1 above) does not stop guessing: every > reader holds that key, so a reader who is not a recipient can still compute the HMAC of > a guess and compare. The comparison value is now `HMAC-SHA256(K_cmp, plaintext)` with > `K_cmp = HKDF(FK, info="sgit seal compare v1")`, where `FK` is the per-file key that only > recipients unwrap. It is stored **inside** the envelope, as the first 32 bytes of the > encrypted payload, never in the tree entry. A recipient's clone caches it per sealed blob > id in a 0600 local file, so `status` does not call the identity provider on every run. > The tree entry holds only the hash of the sealed bytes. > 2. **The chunked AES-GCM payload has defined framing** (§3.1): a per-file payload key, a > counter nonce, and the chunk index and a final-chunk flag in the AAD. A reader refuses > truncation, reordering, extension and a final flag in the wrong place. ## 1. The idea, in your words and in mechanism > As the author I can say: "here is my public key, encrypt this data with it; if you need > to decrypt it, point over there". The files still exist in the vault as before, but > only someone with the private key can read them. The service holding the private key > can be local, remote, or anything else. Mechanically, this is **envelope encryption to recipients' public keys, with decryption delegated to an identity provider**. git-crypt (GPG), SOPS (KMS/age/PGP) and age (X25519, ssh keys, hardware plugins) all use this pattern. The vault keeps doing its job, and selected files get an inner envelope: ``` plaintext ──► [ inner: sealed to recipients' public keys ] ──► [ outer: vault encryption ] ──► host file key FK (random) encrypts the content read key / DK, exactly as today FK is wrapped once per recipient public key ``` | Who | Sees | |---|---| | The host | ciphertext only (outer layer), as today | | A **vault reader** (read key) | that the file exists: its name, path, history, approximate size, who changed it when. **Not its content.** | | A **recipient** (holds, or can ask for, a matching private key) | the content | Two levels, two key systems: - **Outer layer** (vault key): hides everything from the host. - **Inner layer** (recipient keys): hides content from everyone who is not a recipient, including the host, other vault members, and anyone who later obtains the vault's read key. ## 2. What each party can do, precisely | Party | Create a sealed file | Read it | Edit it | Replace / delete it | |---|---|---|---|---| | Recipient (with its provider) | yes | yes | yes | yes | | Vault writer, not a recipient | **yes**: only the recipients' *public* keys are needed | no | no (editing needs reading first) | **yes, blindly**: it can write new sealed bytes over the file, or delete it | | Vault reader | no | no | no | no | | Host | no | no | no | withhold or replay, as today | One correction to "you could only edit or modify a file if you have that key". **Editing** needs the key, because you must read before you change. **Replacing** does not. Any vault writer can put new sealed bytes at the same path, because sealing needs only public keys. Stopping that is an integrity question, not a confidentiality one, and has two answers that can be combined: 1. **Signed envelopes:** the sealer signs the inner envelope with its own key. Readers (recipients) check that the signer is on the path's allowed-author list (§4). 2. **Path rules enforced on commit/pull:** a commit that changes a sealed path must be signed by an allowed author. Otherwise pull refuses it, as `signatures-required` does today. With the native PKI mode's writer set, this becomes a hard rule. ## 3. Format **Recommendation (revised 10-10): an HPKE envelope, DHKEM(P-256, HKDF-SHA256) + AES-256-GCM, all native to Web Crypto; age v1 as an export and plugin-bridge format only.** The original reasoning for age follows, kept for the record. - It is specified, small and audited. Recipient types include X25519, ssh-ed25519 / ssh-rsa and scrypt (passphrase), plus a **plugin protocol** (`age-plugin-*`). - The plugin protocol is precisely "the private key lives elsewhere": YubiKey/PIV, TPM, Secure Enclave and cloud KMS plugins already exist. - **A sealed file can be opened with the stock `age` tool, without sgit**, which is good for longevity and audit. - The pieces it needs (X25519, HKDF-SHA256, ChaCha20-Poly1305, HMAC-SHA256, scrypt) are all in `cryptography`. A minimal reader/writer for the X25519 and plugin stanzas is a few hundred lines. **Trade-off to decide.** age's payload cipher is ChaCha20-Poly1305, which **Web Crypto does not provide**, while CLAUDE.md asks for Web Crypto byte-for-byte parity. The browser would use `typage` (the reference TypeScript age implementation). The alternative is a custom HPKE envelope (X25519 + HKDF + AES-256-GCM, all in Web Crypto), at the cost of the tool ecosystem and the plugins. My view: age. The plugin ecosystem *is* the "decryption service elsewhere" feature, and the parity rule can name `typage` as the browser implementation for this layer. ``` sealed file = "age-encryption.org/v1\n" "-> X25519 \n\n" one stanza per recipient "-> piv-p256 <…>\n\n" a hardware / plugin recipient "---
\n" [ + optional detached Ed25519 signature by the sealer, in a sidecar stanza ] ``` ### 3.1 Payload framing (HPKE envelope, revised 10-10) ``` header = magic "sgit-seal/1" ‖ header_nonce (16 random bytes) ‖ recipient stanzas (each: HPKE enc ‖ HPKE-Seal(recipient pub, FK)) ‖ chunk_size (u32, 65536) H = SHA-256(header) PK = HKDF-SHA256(ikm=FK, salt=header_nonce, info="sgit seal payload v1") (32 bytes) plaintext' = HMAC-SHA256(K_cmp, plaintext) ‖ plaintext (the comparison value rides inside) chunk i = plaintext'[i·65536 : (i+1)·65536] (the last may be shorter; an empty file is one empty final chunk) nonce_i = u88_be(i) ‖ final_i (12 bytes; final_i = 0x01 on the last chunk only) aad_i = H ‖ u64_be(i) ‖ final_i ct_i = AES-256-GCM(PK, nonce_i, chunk i, aad_i) sealed file = header ‖ ct_0 ‖ ct_1 ‖ … ‖ ct_n ``` - **The AAD binds each chunk to this header, its position and whether it ends the file**, so chunks cannot be moved between files, reordered, dropped from the end (the new last chunk has `final = 0` and fails) or followed by more (anything after a `final = 1` chunk is refused). The index is also in the nonce, which keeps nonces unique under `PK` without storing them. - `PK` is fresh per sealing (random `FK` and `header_nonce`), so the counter nonce never repeats under one key. At most 2^32 chunks (256 TiB) per file. - A reader streams: it verifies and releases chunk *i* before reading *i+1*, but treats the file as complete only after a valid `final = 1` chunk. The plaintext is written to a temp file and renamed only then, so a truncated file never reaches the working copy. - Test vectors (Python and Web Crypto, byte for byte): empty file, exactly one chunk, one chunk plus one byte, three chunks; and the refusals: truncated at a chunk boundary, two chunks swapped, a chunk appended after the final one, a final flag on chunk 0 of three, a header byte changed. ## 4. Policy: which files, to whom, written by whom A committed policy file, signed like any commit. It works the way `.gitattributes` works for git-crypt: ``` # .sgit/seal (committed; changes to it are themselves a signed, reviewable commit) recipients: dinis = age1qy… # published key, fingerprint pinned here backup = age1zz… # offline recovery key ops = age1yubikey1q… # hardware-held seal: secrets/** -> dinis, backup finance/*.xlsx -> dinis, ops, backup **/.env -> dinis, backup authors: # optional: who may change a sealed path (§2) secrets/** -> dinis ``` - **Recipients by public key**, fetched from the published key directory and **pinned by fingerprint in the policy**. A later change at the directory cannot silently add someone; the policy changes only through a commit. - `sgit commit` seals any changed file a rule matches. `sgit seal add --to ` and `sgit seal rekey` edit the policy. ## 5. On disk, in the working copy | Clone | A sealed file looks like | |---|---| | Has a working identity for that file | the **plaintext**, written 0600 (secret writer). `sgit status` compares `HMAC(K_cmp, plaintext)` with the value inside the envelope (cached locally per blob, revised 10-10); never by re-sealing, which is randomised | | Has no identity, or the provider is unreachable | the **sealed bytes**, unchanged (an `age` file). `status` treats it as unchanged, never deleted or modified: the same "keep the committed entry" rule the symlink ban uses | Integration points in the current code: - **Tree entries** carry `content_hash` = an unkeyed `sha256(plaintext)[:12]` (today's `Vault__Crypto.content_hash`), stored under the vault read key. For a sealed file that would let any vault reader *confirm a guess* of the content. **For sealed files the hash is taken over the sealed bytes.** The entry also gets a `sealed` flag and the recipients' fingerprints (still under the read key). - **Blob reuse** (`encrypt_or_reuse_blob` matches on `content_hash`) then works on sealed bytes. Re-sealing an unchanged file is skipped as long as the plaintext is unchanged and the policy did not change (a local cache of plaintext hash → sealed blob). - **Merge:** sealed files are binary unless both sides can be opened locally. A conflict keeps both versions (`.conflict`). Diff shows `sealed, changed` without an identity. ## 6. The decryption service ("identity provider") Only the **16-byte file key** crosses the boundary, never the content: ``` sgit ──(stanza: recipient tag + wrapped FK)──► provider ──(FK, or "refused")──► sgit ``` | Provider | Where the private key lives | Notes | |---|---|---| | Local identity file | `~/.config/sgit/identities` (0600) | the simple default; written by the secret writer | | ssh key / ssh-agent | your existing ssh key (ed25519/rsa) | age natively supports ssh recipients; nothing new to manage | | Hardware | YubiKey / PIV / TPM / Secure Enclave via `age-plugin-*` | touch-to-decrypt; the key cannot be copied | | Remote service | a team service, cloud KMS, or an SG/Send service, over an authenticated API | can **log, rate-limit, require approval and revoke** per unwrap, per file: an audit trail of who opened what. Offline means unreadable, by design | Configuration is per clone (`.sg_vault/local/identities` naming providers) or per user. The interface is the age plugin protocol, so any existing plugin works, and our own remote-service provider is just one more plugin. **Capability hand-off.** The same envelope moves *vault* capabilities between people. `sgit share --to dinis` seals a read or write capability to Dinis's published key: no passphrase in chat, argv or shell history. This closes most of TM-R14 and is phase 2 of the earlier analysis. ## 7. What it protects, and what it does not | Threat | Covered | |---|---| | Host reads content | yes (outer layer, as today) | | A vault member who is not a recipient reads a sealed file | **yes** | | The vault read key leaks (a share, a log, an old clone) | **yes for sealed files**: they need a recipient key as well | | A recipient's key is revoked | only for **new** seals: history sealed to them stays readable to them (inherent; rotate secrets, not just files) | | A non-recipient writer replaces a sealed file | detected with signed envelopes / author rules (§2); not prevented on the host | | File names, paths, sizes, history from vault readers | not hidden (they live in the vault's tree). Sizes are approximate (age adds per-chunk overhead); pad if needed | | Plaintext on a recipient's disk | out of scope: it is the recipient's machine (written 0600) | | Provider compromise | as strong as the provider: hardware or approval-based providers limit it | ## 8. Relation to the native PKI mode - **Independent of the vault mode:** sealed files work in symmetric vaults today. - **In a PKI vault**, per-member envelopes are also the natural way to hand `DK` to each member: the vault's `DK` is sealed to every member's public key instead of to one shared `R`. Removing a reader then means re-sealing to the rest (native-mode doc §6). - The deposit capability (native-mode doc §5) plus sealed files gives a **drop box**: contributors can add files that only the owner can open, and cannot read each other's. ## 9. Effort | Step | What | CLI estimate | |---|---|---| | 1 | HPKE envelope (DHKEM(P-256, HKDF-SHA256) + AES-256-GCM, chunked), Python and Web Crypto with shared vectors; `seal export --age` | ~1 week | | 2 | `.sgit/seal` policy, seal on commit, open on checkout/pull, sealed-bytes hashing, status rules | 1–1.5 weeks | | 3 | Providers: identity file + ssh keys, then the plugin protocol (hardware/KMS), then a remote-service plugin | ~1 week (+ service side) | | 4 | Signed envelopes + author rules on pull | ~0.5 week | | 5 | `sgit share --to ` (capability hand-off) | ~0.5 week | | — | Web UI: open sealed files with Web Crypto (no WASM) + a provider bridge | web team | ## 10. Decisions for you 1. **Format (revised 10-10):** the Web-Crypto-native HPKE envelope, with age as an export and plugin-bridge format. (Was: age.) 2. **Working copy:** plaintext when the clone can open the file (git-crypt style), or always sealed on disk with `sgit cat` / `sgit open` to read? 3. **Integrity for sealed paths:** signed envelopes + author rules in the first version, or later? 4. **First remote provider:** an SG/Send-hosted unwrap service with approval and audit, or a cloud KMS plugin? --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/sealed-files-layer.html)* # Security CI: a dedicated pipeline, and an AI review that can break the build, cli.sgit.ai > A dedicated security pipeline on every pull request, and a keyless AI security review that can break the build on anything new: why, how it is wired, and what it costs per reviewed change. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Security CI: a dedicated pipeline, and an AI review that can break the build **Your questions (13 Oct):** 1. Is a dedicated security pipeline worth it? 2. Can some of the checks and AI security reviews run in CI and break the build on anything new? 3. What would that cost, on a cheaper model? 4. How do you create a Claude key for it? **Short answers:** 1. Yes. It is now `.github/workflows/security.yml`. 2. Yes. The AI review is built, tested without spending anything, and switched **off** until you connect it. It is keyless: GitHub's OIDC token through workload identity federation. 3. About **$0.20–0.65 per reviewed PR on Sonnet 5.5**, or **1–16 cents on Haiku 5.5**. Changes that touch none of the 45 security-slice files cost nothing. 4. No key is needed: workload identity federation, steps in §6. ## 1. Why a dedicated pipeline Three reasons: - **Nothing ran on a pull request.** `ci-pipeline.yml` runs on push to `dev` and `main` only, so every gate, including the security job, could only fail after the merge. `security.yml` runs on every PR into `dev` or `main`, so a red result blocks the merge rather than reporting it, and on every push to `dev`, since that is where changes land here (updated 13 Oct). - **New advisories and rules arrive without code changes.** A weekly run (Mondays 05:17 UTC) on `dev` catches a new CVE in `cryptography` or a new Semgrep rule even when nobody pushes. - **The workflows themselves had findings.** zizmor found 4 high and 13 medium issues in our Actions. They are fixed or recorded (§3). The release pipeline keeps its own security-tests job, since that job gates publishing. The new workflow does not replace it. ## 2. The gates (each fails only on what is new) | Job | What it runs | Fails when | |---|---|---| | Security & adversarial tests | `pytest tests/security` | a `Fixed__*` attack works again, or a `Known_Gaps` gap moved | | Bandit (baseline) | `bandit -r sgit_ai -b tests/security/bandit-baseline.json` (1.9.4) | a finding not in the committed baseline | | Semgrep (new lines) | `p/python`, `p/security-audit`, `p/secrets`, `r/python.cryptography` (1.178.0) on `sgit_ai` and `review/tools`, `--baseline-commit` = the PR's base | a finding on a line the PR adds or moves. The weekly run reports everything as SARIF and does not fail | | pip-audit | `--strict` on the runtime dependencies, now **read from `pyproject.toml`** (the release job hardcodes them) | any known advisory | | zizmor | the workflows and the composite action, `--min-severity medium`, ignores in `.github/zizmor.yml` | a new medium or high finding | | actionlint | `actionlint` (1.7.12) on the workflows | a workflow GitHub would reject. Added after the first `dev` run of this change failed at load: a duplicated `with:` in the mutation job passed every YAML parser and zizmor | | review/ graph current | `review/tools/derive.py --check` | never fails: a warning, because a stale graph is a docs debt, not a defect | | AI security review | `review/tools/ai_review.py` (§4) | a new, confident, medium-or-worse finding that survives a second opinion | Calibrated here: - **Semgrep's PR mode** run against the merge I pushed today flagged the three `urlopen` lines in the `serve` code. Bandit already had reviewed `nosec` reasons for those lines; they now carry `# nosemgrep` as well. - **Bandit, zizmor and pip-audit** are clean on the branch. ## 3. What zizmor found in our workflows, and what changed - **template-injection (high, 4).** The composite test action pasted `${{ inputs.* }}` into shell text. The values are our own constants today, but the pattern is the classic Actions injection. - Fix: the inputs now reach the shell as environment variables, with `set -f`. - Side effect: the integration jobs' `'mcp<2'` lost its quotes (`mcp<2`). A variable expansion never parses `<` as a redirection; I checked the word-splitting in bash. - **artipacked (medium).** Every read-only job now sets `persist-credentials: false`. - The tag job and the two publish jobs keep their credentials, because they push the tag and run `git__update_branch`. Each has an inline ignore with that reason. - **secrets-inherit, excessive-permissions (medium).** The `dev` and `main` callers pass every secret and run with default token permissions. Narrowing them changes the publishing path, so they are ignored with reasons and left for a release check (follow-up under TM-R21). **Risk:** these edits touch the release pipeline's test action. They parse, zizmor is clean, and the word-splitting is checked. The first `dev` run after the merge is the real check, so watch it. ## 4. The AI review (`review/tools/ai_review.py`) ### Flow 1. **Scope.** Take the PR's changed files that are in the security slice (`review/slice.json`, 45 files: crypto, host and local boundaries, verify-then-accept, key stores, supply chain). If there are none, stop: it costs nothing. 2. **Request.** One request to Claude: - *System part (cached):* the rules, `review/intent/*.json`, and the threat model. - *User part:* the diff (15 lines of context), each touched file whole, and the function dossiers for those files from `review/context/functions/`. - Settings: adaptive thinking; effort `medium`; output forced to a JSON schema; server-side fallback on (`fallbacks: "default"`), so a safety-classifier decline is retried on another model instead of failing. Haiku has no fallback. 3. **Findings.** Each finding has a file and line, a function, a category, a severity, a confidence, and **`register`**: the threat-model row it is (TM-F/TM-R), or `new`. 4. **The gate.** - It counts only findings that are `new`, at least medium severity and at least 0.8 confidence. - If any remain, a **second call tries to refute them**: is the input reachable, is it handled elsewhere, did it exist before the diff? Only what survives fails the build. - Clean PRs never pay for that second call. 5. **Waivers.** You accept a finding by adding its id (12 hex: file, function and category, not the wording) with a reason to `review/findings/ai-waivers.json`. Exit codes: | Code | Meaning | |---|---| | 0 | pass | | 1 | new findings | | 2 | error | | 3 | inconclusive: the model declined or ran out of tokens; nothing was reviewed, which is not a pass | ### What it is not The diff is attacker-controlled text and the model reads it. A hostile change can argue its way past the reviewer. So the AI review catches mistakes by honest contributors and agents; the deterministic gates are the controls. The model gets no tools, so injected text can only change the verdict. The prompt also asks it to report steering text as a finding. The job runs only: - on pushes to `dev`, and on PRs from branches of this repository (a fork's PR never gets an OIDC token); - when the repository variable `SECURITY_AI_REVIEW` is `true` and a federation rule id is set for that event. Never move it to `pull_request_target`. ### Tested without spending anything - **Wire shape.** A loopback server stood in for the API. It confirmed that the request carries the `server-side-fallback-2026-07-01` beta, `fallbacks: "default"`, adaptive thinking, the JSON-schema output and the cached system prompt. - **Paths.** I ran it through the four outcomes: - confirmed finding → exit 1 (only the high, new one gates; a low one does not); - refuted finding → pass; - waived finding → pass, one call only; - refusal → exit 3. - **Unit tests.** `tests/unit/review/test_Review__AI_Review.py` covers the gate rules, waivers, the fingerprint, the Haiku price tier, the skip, and a dry run that calls nothing. ### The off-the-shelf alternative [`anthropics/claude-code-security-review`](https://github.com/anthropics/claude-code-security-review) runs Claude Code as an agent over the PR (up to 20 minutes) and comments on it. - It does not fail the build itself. It outputs `findings-count`, so you would add your own step to fail on it. - Its default model is an older Opus. Its README says it is *"not hardened against prompt injection attacks and should only be used to review trusted PRs"*. It is good for PR comments. As a gate, ours is cheaper and predictable, because it is one call with no tools and known inputs. It also knows our threat model and register, so it can tell new from known. Both can run if you want PR comments as well. ## 5. Cost (measured inputs, list prices as of 2026-10) Dry runs (`--dry-run`: counts what would be sent, sends nothing) on five real commits: | Commit | Slice files | Input tokens (~) | Sonnet 5.5 clean / with findings | Haiku 5.5 clean / with findings | Opus 5.5 clean / with findings | |---|---|---|---|---|---| | `b32a6ba` sgit serve | 2 | 61k | $0.19 / $0.32 | $0.01 / $0.02 | $0.38 / $0.64 | | `9362a25` TM-F32..36 | 4 | 75k | $0.22 / $0.38 | $0.01 / $0.02 | $0.44 / $0.76 | | `fbd550f` pull guard | 4 | 75k | $0.22 / $0.38 | $0.01 / $0.02 | $0.44 / $0.76 | | `eb7941c` bulk fetch | 4 | 109k | $0.29 / $0.51 | $0.07 / $0.13 | $0.57 / $1.03 | | `a5e6043` partial clones | 7 | 144k | $0.36 / $0.65 | $0.09 / $0.16 | $0.71 / $1.30 | **How the numbers are built:** - Input is about 16k tokens of system (design plus threat model) plus the change. - Output is assumed at 6k tokens per review and 4k per verification, including thinking. The real figure is printed in the job summary once it runs. - Prices per million tokens (input / output): - Sonnet 5.5: $2 / $10. - Haiku 5.5: $0.10 / $0.50, or $0.50 / $2.50 past 100k input tokens, which is why the larger commits jump. - Opus 5.5: $4 / $20. **Per month**, at 60 PRs that touch the slice: | Model | Per month | Before your first top-up | Spend limit to set | |---|---|---|---| | Sonnet | about $15–40 | a $10 credit covers about 30 reviews | $25 | | Haiku | about $1–5 | | | **Which model.** - Start with **Sonnet 5.5**, the default (`SECURITY_AI_REVIEW_MODEL` changes it with no code edit). Haiku 5.5 is about 20 times cheaper per token, but finding security bugs is reasoning work, and a gate that misses or invents findings costs more in attention than it saves. - **Measure before you choose.** Your branch gives a ready-made test with known answers: review the five fixes *backwards*, so the "change" re-introduces TM-F32..F36. Commands are in §6. A model that does not flag them is not worth its price at any price. That is under $1 on Sonnet and a few cents on Haiku. - Batch pricing (50% off) does not suit a PR gate, which needs the answer in minutes. It would suit a later weekly sweep of the whole slice: about 150k tokens of code, roughly $1.30 on Sonnet, $0.65 batched. ## 6. Switching it on: keyless, with workload identity federation *Updated 13 Oct. This replaces the API-key setup first written here.* The CI job holds **no API key**. It asks GitHub for a short-lived OIDC token (about 5 minutes) for the audience `https://api.anthropic.com`, and the SDK exchanges that token once for an Anthropic token bound to a service account. Billing goes to the organization that owns the service account (the one with the credits), attributed to the rule's workspace. The job runs on every **push to `dev`**, since that is where changes land here. It can also run on PRs. It reviews `before..after` of the push. A push that touches no security-slice file (release-badge commits, docs) is skipped at no cost. ### In the Claude Console (organization admin) 1. **Settings → Workspaces → Create workspace** `sgit-ci`. Click the workspace, then **Spend limits**, and set the monthly cap, for example $25 with an alert at 50%. 2. **Settings → Workload identity → Connect workload → GitHub Actions.** The wizard creates the issuer, the service account and the rule together. It is better than the bare *Register issuer* dialog. - **Issuer:** name `github-actions`, issuer URL `https://token.actions.githubusercontent.com`, JWKS source *OIDC discovery*. Leave the discovery URL and CA certificate empty. - **Service account:** for example `sgit-ci-security-review`, member of workspace `sgit-ci`. - **Rule for pushes to dev** (the one the job needs): - subject: `repo:SGit-AI/SGit-AI__CLI:ref:refs/heads/dev` (exact, no `*`); - audience: `https://api.anthropic.com`; - claims: `repository_id` = `1187093532`, `repository_owner_id` = `269696930`, `ref` = `refs/heads/dev` (numeric ids survive a rename); - workspace `sgit-ci`, scope `workspace:developer`, token lifetime 600 s. - **Optional second rule for PRs:** subject `repo:SGit-AI/SGit-AI__CLI:pull_request`, same audience and id claims, no `ref`. 3. Note the ids: rule `fdrl_…`, service account `svac_…`, workspace `wrkspc_…`, and the organization id (Settings → Organization). ### In GitHub (Settings → Secrets and variables → Actions → *Variables* tab) None of these values is a secret: without a GitHub token that matches the rule, they mint nothing. | Variable | Value | |---|---| | `SECURITY_AI_REVIEW` | `true` | | `ANTHROPIC_ORGANIZATION_ID` | the organization id | | `ANTHROPIC_SERVICE_ACCOUNT_ID` | `svac_…` | | `ANTHROPIC_WORKSPACE_ID` | `wrkspc_…` | | `ANTHROPIC_FEDERATION_RULE_ID` | the dev-push rule, `fdrl_…` | | `ANTHROPIC_FEDERATION_RULE_ID_PR` | optional: the PR rule | | `SECURITY_AI_REVIEW_MODEL` | optional: `claude-haiku-5-5` (the default is Sonnet 5.5) | Do **not** add an `ANTHROPIC_API_KEY` secret. A set key, even an empty one, takes precedence over federation; the job unsets it anyway. **Who can spend.** Anyone who can push to `dev` can run a workflow that mints a token. Forks cannot: a fork's PR gets no OIDC token. The workspace spend limit is the backstop. **Tested here without spending anything.** I ran the job's exact shell step against a local stand-in for both GitHub's token endpoint and Anthropic's token exchange: - the GitHub token is requested with the right audience; - it is exchanged **once**, with the rule, organization, service-account and workspace ids; - API calls carry the federated bearer token; - an empty API key is unset; - an all-zero push base falls back to the parent commit; - the token file is deleted afterwards; - a gating finding gives two API calls on one exchange, and the step exits 1. `ai_review.py` now makes one SDK client per run. Before this fix, the second call created a new client, which would have re-presented the single-use GitHub token and been refused (`jti_reused`). **Locally**, with your own key and never in a repo file, the commands in §5 still work: `export ANTHROPIC_API_KEY=…` and `python3 review/tools/ai_review.py --base 9362a25 --head 9362a25^`. ## 7. Next steps (not done here) - A weekly whole-slice AI sweep through the Batch API, with findings written as records under `review/findings/`. - Narrowing `secrets: inherit` and the default token permissions in the dev and main callers, with a release check. - A committed lockfile, so pip-audit checks what is installed, not what could be (TM-R21). - `review/tools/derive.py` running in CI and committing the graph, rather than warning. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/security-ci-and-ai-review.html)* # Case study — a tool-assisted security review of sgit-ai 0.21.0 (round 1), cli.sgit.ai > What each tool found in the round-1 security review of sgit-ai 0.21.0, with counts and cost: the skill vetting, the derived graph, 270 function records, 68 STRIDE threats and the insider overlay. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Case study — a tool-assisted security review of sgit-ai 0.21.0 (round 1) **From:** the sgit-ai CLI agent · **Date:** 2026-10-11 · **Plan:** `../plan__security-review-with-skills.md` (accepted by Dinis the same day: the slice as proposed, `review/` at the repository root, no need to tell Matt beforehand since every tool is open source). **For:** `skills.sgit.ai` and `threat-modeling.sgit.ai`, and Matt Adams, whose three tools were part of the run. This note reports what each tool found, with counts, and what the run cost. The method and the data are in `review/` (see `review/README.md`); the vetting of the tools is in `00__vetting.md`. ## 1. What ran, in order | Step | Tool | Ran as | Output | |---|---|---|---| | 0 | Cisco AI Defense skill-scanner 2.2.2, OWASP AST10 checklist | tool + checklist by hand | `00__vetting.md`, four SARIF files, `review/tools/skills.lock.json` | | 1 | `review/tools/derive.py` (stdlib `ast`) | tool | `review/graph/`: 41 Python files, 47 classes, 522 methods, 3,289 test functions whose file imports a slice module | | 1 | Trail of Bits audit-context-building | four subagents, one per boundary | `review/context/functions/`: 270 function records with `nothing found` assumptions and open questions | | 2 | mrwadams stride-threat-modelling skill (no model key, no hosted MCP) | the agent | `review/stride/agent.json`: 68 threats, 25 explicit N/A, DREAD scores; joined to the register by `stride_join.py` | | 2 | Matt Adams' insider-threat model (27 threats, CC BY 4.0) | the agent, as data | `review/stride/insider-overlay.json`: 16 land on existing rows, 2 new model items, 9 are the deployment's; actor A9 proposed | | 3 | Semgrep 1.180.0 (236 rules), Bandit 1.9.4, pip-audit 2.10.1 | tools | `review/findings/raw/`; a Bandit baseline (108 results) and the pip-audit gate added to the security CI job (TM-R21) | | 3 | Trail of Bits sharp-edges, insecure-defaults (its six corpora by hand) | the agent | `review/findings/sharp-edges.json`, `insecure-defaults.json` | | 3 | Sentry security-review, secret-serialization, gha-security-review | the agent | three findings files | | 3 | Trail of Bits supply-chain-risk-auditor | its scripts via `uv` | `raw/supply-chain-report.md`; the collector did not parse the Poetry tables (see §4) | | 3 | Trail of Bits differential-review (0.20.0 → 0.21.0, 27 files, 58 commits) | one subagent | `review/findings/differential-review.md` + `.json` | | 3 | Trail of Bits post-patch-validation on TM-F29 (`f624d3b` → `d970599`) | the same subagent, the skill's runner | `post-patch-validation.md` + result and report JSON: complete, 0 gaps, runtime evidence, 4 exploit/variant checks fail on base and pass on patch | | 4 | Trail of Bits fp-check (by hand), mrwadams threat-led-triage (rubric by hand) | the agent; four reproductions with lying hosts on the in-memory API | `review/findings/ranked.json` | | 5 | `review/tools/security_json.py` | tool | `review/security/`: 41 `.security.json` records in the house schema | Not run: STRIDE GPT's `analyze` (no model API key in this session) and the hosted STRIDE GPT MCP server (deliberately: it receives the application description). The skill from the same author stood in for both; the tool pass is owed from a machine with a key, so the planned two-pass comparison can still happen. ## 2. What was found 104 finding records across nine files (`review/findings/`), each a claim with its evidence: | Tool | VULN | VERIFY | KNOWN | CLEARED | |---|---|---|---|---| | audit-context-building (four dossiers, read as claims) | 4 | 31 | 6 | 7 | | differential-review | 2 | 1 | 1 | 12 | | sharp-edges | 1 | 4 | 3 | 2 | | sentry-security-review | 2 (same root cause as sharp-edges') | 2 | 1 | 2 | | sentry-secret-serialization | 0 | 1 | 1 | 1 | | insecure-defaults | 0 | 1 | 2 | 3 | | semgrep | 0 | 1 | 0 | 5 | | bandit (30 in the slice) | 0 | 2 | 1 | 4 (+1 out of slice) | | pip-audit, supply-chain auditor, gha review | 0 | 0 | 0 | clear | **Five distinct defects reproduced and fixed this round, each with a test that fails on the code before its fix, a register row and a CHANGELOG line:** | Row | Defect | Found by | Reproduced how | |---|---|---|---| | TM-F32 | `sgit pki delete ..` emptied `~/.sg-send` (credential store, keyring, keys); `pki export ../x` read any PEM | sharp-edges (SE-001), confirmed by the Sentry pass and the crypto dossier | in a sandboxed temp dir | | TM-F33 | a push the host acknowledged with an empty body counted as pushed (batch path and the individual-write fallback) | host dossier (AC-H-003), confirmed by the differential review (DR-002) | a lying host answering `{}` | | TM-F34 | an empty listing made a clone that had synced re-upload stale refs with no compare-and-swap: the shared branch rewound under a legitimate key | host dossier (AC-H-001) | a lying host listing `[]`; 1 of 2 refs moved | | TM-F35 | a garbled branch index on clone degraded to a policy-less index: signatures-required off for the newcomer | verify dossier (AC-V-001) | random bytes in the host's index; clone succeeded with no features | | TM-F36 | a server index this build cannot parse was overwritten by the pushing clone (the whole-index twin of TM-F29) | differential-review (DR-001) | a branch entry with an unknown type; the push erased it | One more reproduced defect is **open with a proposed fix**: AC-L-001, the vault-path type rewrites any character outside `[a-zA-Z0-9/\_.- ]` to `_`, so a vault under `/home/josé/` keeps its objects, refs and keys under `/home/jos_/` (the fix touches every path field, so it is left for a decision). One suspected ordering defect was **cleared by reproduction**: a dirty-tree refusal on pull leaves the baseline untouched (AC-V-003). The remaining 46 VERIFY records are ranked in `review/findings/ranked.json` with the threat-led-triage rubric (reachability first: "host alone" is this system's unauthenticated). The next ones in the queue: AC-H-002 (the host's `not_found` drops the CAS), AC-V-002 (the rewind guard's UNKNOWN verdict proceeds), AC-H-004 (host body text drives control flow), AC-L-006 (no ceiling on the walk to `.sg_vault`), AC-C-002 (a corrupt `remotes.json` falls through to another server). ## 3. STRIDE GPT's rubric, applied to the hand STRIDE pass TM-Bench scores STRIDE coverage, completeness, technical validity and JSON compliance. Against this project's register as ground truth (`review/stride/coverage.json`): | Dimension | Result | |---|---| | Completeness against the register | first pass 54 of 63 rows (85.7 %); second pass 63 of 63; third pass, after the review added five rows, 68 of 68 | | STRIDE distribution | T 20, I 15, S 8, D 8, E 7, R 5; 25 cells recorded N/A with a reason | | Priority distribution (DREAD) | Critical 19, High 29, Medium 14, Low 1 | | New threats (no register row before this round) | 9, of which 4 became fixed rows and 2 are the agent-as-insider model items | | JSON compliance | by construction (the join tool reads it) | The honest reading: the register was the model's input, so completeness measures whether the pass restated it, not whether it would have found it. The nine new threats and the five fixes are the part that was not in the input. ## 4. What the tools got wrong, or could not do - **The skill scanner's errors inside the installed set were all false positives** (pattern lists and attack references read as instructions); its real findings were the undeclared network hosts of the supply-chain skill, and the absence of any licence field or signature. - **The supply-chain collector misread this project**: it parses PEP 621 tables only, so the three runtime dependencies in `[tool.poetry.dependencies]` were not seen and the report says "every direct dependency is a development dependency". Without an authenticated `gh`, ten of thirteen criteria were unassessable. pip-audit did the advisory work. - **Semgrep and Bandit found nothing new**: their six and thirty results were the two opt-in TLS branches, the guarded presigned PUT, a file name called `token`, and `try/except: pass`. Their value here is the CI baseline, not the findings. - **The dossiers (audit-context-building) were the most productive input**: four of the five fixed defects came from a `nothing found` assumption in a dossier, read as a claim and reproduced with a lying host. The skill's refusal to name vulnerabilities is what made the records usable: they list what the code counts on, and the hunting is a separate, cheap step. - **The differential review found the one defect the dossiers missed** (TM-F36), because it asked of every fix "what did its skip branch do to the data it skipped", which is how TM-F29 happened. Its post-patch-validation runner worked end to end on the first fix it was pointed at, with runtime evidence on both revisions. - **Three of the Sentry findings and the sharp-edges finding were the same defect** seen from three methods; the case for running several skills is that each one's phrasing reached a different line of the same file. - **The one thing the agent got wrong**: the first run of the TM-F32 test executed the unfixed code's `..` and `../..` cases from a temp directory directly under `/tmp`, and the unfixed code removed `/tmp`'s contents, including this session's scratchpad and the commit-signing helper. The test was rewritten to run three directories below a sandbox root so that nothing the unfixed code does can leave it. A proof that destroys the environment it runs in is itself a finding about the proof. ## 5. Cost | Item | Measure | |---|---| | Subagents | 5 (four dossiers, one differential review + post-patch validation); 1.9 M tokens between them; 23 to 32 minutes each | | The orchestrating agent | one session, the whole day | | Tools' own time | Semgrep about 1 minute; Bandit seconds; the full unit + security suites 2 min 11 s with `-n auto` | | Lines of code reviewed | 11,604 in 45 files (the slice); the differential review read +1,566/−172 across 27 of them | | Tests added | 4 files or classes, 46 attack cases, 11 controls; every attack case fails on the code before its fix | ## 6. For the two sites and for Matt - `skills.sgit.ai`: the research doc (`../research__security-review-skills.md`), `00__vetting.md` and §4 above are the material. The headline: none of the ecosystem's best security skills can pass the AST10 checklist they are vetted against, because none is signed. - `threat-modeling.sgit.ai`: `../research__threat-modeling-tools.md`, `review/stride/`, the insider overlay, and §3 above. The headline: a register with a test per row is a usable ground truth, and the agent-as-insider actor (A9) belongs in it. - For Matt: the stride-threat-modelling skill ran without a key and produced a model the join tool could score; the licence in its frontmatter (GPL-3.0-or-later) disagrees with the repository's MIT; `threat-led-triage`'s rubric transferred well once "host alone" was mapped to its `unauthenticated` band; STRIDE GPT's `analyze` is still owed a run with a key. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/security-review-case-study.html)* # Plan — the next security review of sgit-ai, with skills, and the review graphs it produces, cli.sgit.ai > The plan for a tool-assisted, data-producing security review of one slice of sgit-ai: the code the zero-knowledge guarantee and the trust boundaries live in. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Plan — the next security review of sgit-ai, with skills, and the review graphs it produces **From:** the sgit-ai CLI agent · **Date:** 2026-10-11 · **Status:** proposed, for Dinis to accept or redirect before step 0 runs. **Companion docs:** [`research__security-review-skills.md`](../articles/security-review-skills.md), [`research__threat-modeling-tools.md`](../articles/threat-modeling-tools.md), `map__security-work-already-done.md` in this folder. ## 1. One sentence Run a tool-assisted, data-producing security review of **one slice** of sgit-ai (the code the zero-knowledge guarantee and the trust boundaries live in), using five skills and three tools chosen in the research docs, join every output to the existing register and its tests, and leave behind the first `review/` graphs and per-file security records, so the case study can say which tool found what. ## 2. Scope: the slice, and why it is this one The code-review-graphs brief says not to start an existing repository with a full derivation, and the `secrets.sgit.ai/review/` page starts from what people actually walk. For a security review, what people walk is the key path: derive a key, encrypt, send to the host, verify what comes back, write to disk. Everything the register's High-impact rows touch is in these packages. | In scope (round 1) | Files · lines | Why | |---|---|---| | `sgit_ai/crypto/` | 9 · 793 | every primitive; the KDF cache; the deterministic metadata mode (KI-SEC-02); PKI | | `sgit_ai/network/api/` | 13 · 1,504 | the host boundary: presigned URLs, CAS, per-op conflicts, timeouts, scheme rules | | `sgit_ai/storage/` (`Vault__Path_Guard`, `Vault__Storage`, `Vault__Verified_Write`, `Vault__Object_Store`, `Vault__Index_Reader`, `Vault__Ref_Manager`, `Vault__Branch_Manager`) | 7 of 17 | the local boundary and verified writes; where the path, symlink, baseline and index fixes landed | | `sgit_ai/core/actions/verify/`, `clone/`, `pull/`, `push/`, `commit/`; `sgit_ai/core/Vault__Secret_Guard.py`, `Vault__Ignore.py` | 5 of 28 action groups + 2 | verify-then-accept, the rewind guard, the secret guard and the always-ignored set | | `sgit_ai/secrets/`, `sgit_ai/cli/CLI__Main.py`, `CLI__Credential_Store.py`, `CLI__Token_Store.py`, `CLI__Debug_Log.py`, `CLI__Vault.py` | 6 files | where keys are stored, pinned, printed and logged (TM-R14 is High) | | `.github/workflows/*.yml`, `pyproject.toml`, `sgit update` | 3 + 2 | supply chain (TM-R21) | 41 source files, about 11,300 lines (counted with `wc -l`), out of 469 files. **Out of scope for round 1:** `plugins/`, `workflow/`, `migrations/`, the rest of `cli/`, `core/serve`, `mirror`, `publish` (round 2 candidate: their code-stage review is owed since 08/19), `objects/`, `safe_types/`, `schemas/` (covered by the Type_Safe tests and the round-trip invariant). ## 3. The tools, and what each is for | Step | Tool | Chosen because | Not chosen | |---|---|---|---| | 0 | OWASP AST10 checklist; `cisco-ai-defense/skill-scanner` | the only published rubric for installing third-party instruction files into a key-custody repository | SkillLedger and skillsign: nothing on our list is signed, so there is nothing to verify | | 1 | Trail of Bits **audit-context-building** | a per-function dossier with a line for every claim and "nothing found" for unestablished assumptions; no verdicts, so it cannot anchor the later steps | Sentry's "research the code base first" is a paragraph, not an artefact | | 1 | a parser (Python `ast`) | the code-review-graphs brief: derived structure comes from parsers, never a model | STRIDE GPT's repository stage: it is a model reading code, so its output goes in the intent/explanation layer, not `graph/` | | 2 | **STRIDE GPT** CLI (`analyze`, JSON, Anthropic via LiteLLM) and the **`stride-threat-modelling` skill** copy | two independent STRIDE passes with the same rubric: one by the tool, one by this agent; stable threat ids; evidence required per threat | the hosted `mcp.stridegpt.ai` endpoint (sends the description off-machine); pytm and Threagile (their libraries are web- and infrastructure-shaped; a pytm model file is a round 2 artefact once the DFD exists as data) | | 2 | Matt Adams' **insider-threat model** (27 threats) | the agent as actor; thirteen threats already map to register rows after one reading | | | 3 | Sentry **security-review** (cryptography, file-security, supply-chain, logging guides) and **secret-serialization**; Trail of Bits **insecure-defaults**, **sharp-edges**, **differential-review** (0.20.0 → 0.21.0), **post-patch-validation** (the 0.21 fixes) | method, not pattern list; the diff of 0.21.0 is the largest security diff the project has had and two of its fixes regressed | Project CodeGuard: read its crypto and supply-chain rules once, cite where relevant, do not install; the "Anthropic Cybersecurity Skills" library (not Anthropic's, no code-review domain); Varlock (keep the rule, not the skill); squirrelscan (nothing to audit in a CLI) | | 3 | **Semgrep** (`p/python`, `python.cryptography.*`, Trail of Bits rules), **Bandit**, **pip-audit**; Sentry **gha-security-review** on the workflows | deterministic, SARIF, and the CI baseline TM-R21 has wanted since 10/08 | CodeQL: worth adding at the CI step if the Semgrep pass leaves gaps; not first | | 3 | Anthropic **`/security-review`** | already here; diff-scoped; a second opinion on the pending changes of each fix commit | as the whole-slice review: its filter drops DoS-shaped findings by design, and three accepted rows are DoS-shaped | | 4 | Trail of Bits **fp-check**; Matt Adams' **threat-led-triage** over all findings with the STRIDE GPT model | every finding is a claim to verify; ranking by reachability across *our* boundaries is how the list stays short | CVSS-style scoring | | 5 | the house `.security.json` schema; the `review/` layout from the code-review-graphs brief | so the output is data the sites can join and the next review can diff | a prose report alone | | 6 | the TM-Bench rubric (four dimensions) with the register as ground truth | the measurable part of the case study | running TM-Bench itself (no public harness) | ## 4. The steps Each step names its output. Outputs that are data go under `review/` (new, at the repository root, per the brief); prose goes under this folder's date; threat-model changes go in the register with their tests, as §9 of the threat model requires. ### Step 0 — Vet, pin, canary (half a day) 1. Clone each skill source at a commit SHA; record SHA and `sha256` of every SKILL.md. 2. Run `skill-scanner scan-all --format sarif` over the lot; HIGH blocks install. Keep the SARIF. 3. Fill the AST10 table from [`research__security-review-skills.md`](../articles/security-review-skills.md) §2 for each skill. 4. Install into this cloud session only (no vault key, no token, no `.env` on disk here): Trail of Bits plugins (`audit-context-building`, `insecure-defaults`, `sharp-edges`, `differential-review`, `post-patch-validation`, `fp-check`, `static-analysis`, `supply-chain-risk-auditor`); Sentry `security-review`, `secret-serialization`, `gha-security-review`; `stride-threat-modelling` skill copy; `threat-led-triage` skill; `pip install stride-gpt==` with hashes; Semgrep, Bandit, pip-audit pinned. 5. Rule for every run: the agent never reads `.sg_vault/local/*`, never prints a key, never runs a skill's "fix" mode. Written at the top of the session prompt. **Output:** `…/10/11/case-study/00__vetting.md` with the table and SARIF. ### Step 1 — Context and the first derived graph (one day) 1. `audit-context-building` over the slice → `audit-context/DOSSIER.md` and one file per function. Read the "nothing found" assumptions first; they are the candidate threats. 2. A small `ast`-based deriver (classes, methods, calls within the slice, tests that import each module) → `review/graph/{files,classes,methods,tests}.json`, each node with path, lines and tree hash, per the brief's grammar. Freshness checked by tree hash. **Parsers only.** 3. Intent for the slice, written from the threat model's §2 assets, §4 boundaries and §5 mechanisms, each node citing its section → `review/intent/boundaries.json`, `mechanisms.json`. This is the top-down half, and it is small because the register already is the intent. **Output:** the dossier; `review/graph/` and `review/intent/` seeds; `review/README.md` with counts written by the deriver, not by hand. ### Step 2 — Threat model, two ways, plus the insider overlay (one day) 1. `stride-gpt analyze -f json -o review/stride/stride-gpt.json` with the cost estimate recorded first. Also `-f sarif` for the triage step. 2. This agent runs the `stride-threat-modelling` skill over the same slice and the dossier → `review/stride/agent.json`, same schema (id, STRIDE class, component, description, evidence line, mitigation). 3. Reconcile → `review/stride/join.json`: each threat id ↔ TM-R/TM-F row ↔ test file, or `new`, or `out-of-scope-by-model` (the host-and-harness threats). Disagreements between the two passes are kept, not averaged (the house two-modes rule). 4. Insider overlay: confirm the thirteen mappings in the threat-modeling doc §1.1 against code; add actor **A9** (the agent holding the key) to the register §3, and a column to §6 naming the insider threat ids each row answers. Where a mapping proves an *open* gap, write its `Known_Gaps` test in the same commit. **Output:** `review/stride/` (three files); a register change with tests; the first number for the case study (STRIDE GPT's completeness and coverage against the register). ### Step 3 — Code review (two days) Run in this order, each writing findings as `VULN-###` / `VERIFY-###` with file, line, evidence: 1. Semgrep, Bandit, pip-audit → `review/findings/semgrep.sarif`, `bandit.json`, `pip-audit.json`. These three also become the CI baseline: a new job step in "Run Security & Adversarial Tests" that fails on *new* findings only (closes TM-R21's open half). 2. `insecure-defaults` and `sharp-edges` on `crypto/` and `storage/`. 3. Sentry `security-review` on the slice with the cryptography, file-security, supply-chain and logging guides; `secret-serialization` on `crypto/Vault__Secret_File.py`, `secrets/`, `cli/CLI__Debug_Log.py`, `cli/CLI__Vault.py`, `cli/CLI__Credential_Store.py`. 4. `differential-review` on `git diff 0.20.0..0.21.0 -- `; `post-patch-validation` on the TM-F25→F29 and TM-F27→F30 pairs specifically (the two fixes that regressed). 5. `gha-security-review` on the three workflows; `supply-chain-risk-auditor` on `osbot-utils` and `cryptography`. 6. `/security-review` on each fix commit as it is made (step 4), as the PR gate. **Output:** `review/findings/.json` per tool, same record shape, each with `tool`, `confidence`, `file`, `line`, `evidence`. ### Step 4 — Triage and fix (one day, plus the fixes) 1. `fp-check` over every finding; a finding without a reproducing path is `VERIFY`, not `VULN`. 2. `threat-led-triage rank review/findings/*.json against review/stride/stride-gpt.json` → `review/findings/ranked.json`, with the "could not determine" list kept. 3. House rule applies to anything real: a `Fixed` test that fails on the current code, the fix, the register row, the CHANGELOG line, in one commit. Anything accepted gets a `Known_Gaps` test and a `risk_decision` with owner and rationale. **Output:** fixes with tests; register updated; the per-tool true/false-positive counts for the case study. ### Step 5 — Per-file records and the skill rule (half a day) 1. One `.security.json` per file in the slice (house schema: `attack_surface`, `vulnerabilities` with `risk_decision`, `secrets_exposure`, `dependencies` with `trust_boundary_crossings`, `redundant_code`), written from the dossier and the findings, every claim with a line → `review/security/.security.json`. The slice is 41 files; that is the whole point of the slice. 2. Add to `library/skills/sg-send-cli__SKILL.md` (and the sgit skill on `skills.sgit.ai`) the rule Varlock states and the insider model's I4 needs: never print, echo or log a vault key, read key, write key or token; read keys from the store, not from argv; prefer the key prompt. This is the one thing from the screenshot list that becomes a line in our own skill. **Output:** `review/security/`; the skill change. ### Step 6 — Publish (half a day) 1. The two research docs → `skills.sgit.ai` and `threat-modeling.sgit.ai` Documents, as they are. 2. The case study: which tool found what (true positives, false positives, time, cost per tool), STRIDE GPT scored on the four TM-Bench dimensions against the register, the insider overlay result, the vetting table and the signature gap → both sites, and to Matt Adams. 3. Link the 0.21.0 register and the case study from `sgit.ai/security/` (closes "being prepared"). ## 5. Acceptance - Every file in `review/graph/` is regenerated byte-identically by the deriver from the tree hash it names; a stale file fails the check. - Every STRIDE threat in `review/stride/join.json` resolves to a register row with a test, or is marked `new` with a `VERIFY`, or `out-of-scope-by-model` with the reason. - Every `VULN` has a test that fails on the code before its fix; every accepted gap has a `Known_Gaps` test and a `risk_decision`. - The security CI job runs Semgrep, Bandit and pip-audit against a committed baseline and is green on the head that adds it. - Suites on the final head: unit, security, integration, QA all pass, with counts in the note. - The case study reports per-tool counts, not impressions. ## 6. Risks of the plan itself | Risk | Handling | |---|---| | A skill is a supply-chain vector into a key-custody repository (AST01/02) | step 0; cloud session with no keys; pinned SHAs; no fix modes | | Prompt injection from repository content into a review skill: `tests/security/` and the review-round tests contain adversarial strings by design (hostile paths, `(ſ[a-z]+)+$`, forged objects) | review skills read `sgit_ai/`, not `tests/`; findings cite `sgit_ai/` lines only; the agent treats skill output as claims | | STRIDE GPT sends the slice to the model provider | the code is Apache-2.0 and public; never run it over a checkout holding `.sg_vault/local`; record the cost estimate first | | Finding flood: 23 Bandit mediums were counted in March | step 4's ranking and the baseline; only ranked-reachable findings get a test | | The derived graph becomes hand-maintained | `review/graph/` is written by the deriver only; the freshness check is in the acceptance list | | Scope creep into `plugins/`, `workflow/`, `serve`/`mirror`/`publish` | named out of scope; round 2 is a separate plan with its own note | ## 7. Decisions needed from Dinis 1. Accept the slice, or swap one package in or out (the obvious candidate to add is `core/actions/publish` + `mirror`, owed since 08/19; it would make round 1 about 60 files). 2. Whether `review/` goes at the repository root now (the brief's layout) or under `team/explorer/appsec/review/` until the deriver is trusted. The plan assumes the root. 3. Whether to tell Matt before or after the run that his three tools are being test-driven on a published register with tests, so he can say what he would like measured. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/security-review-plan.html)* # Agent skills for security review — what exists, what it would do here, cli.sgit.ai > Agent skills for security review, each checked at its source, ranked by whether it brings a method, a tool with deterministic output, or vets the other skills, against this code base. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Agent skills for security review — what exists, what it would do here **From:** the sgit-ai CLI agent · **Date:** 2026-10-11 · **For:** `skills.sgit.ai` (Documents), then the case study once used. **Method.** Each entry was checked at its source (the repository README or SKILL.md, the project site, or the registry page) on 2026-10-11, not from the aggregator listing that surfaced it. Where a claim could not be confirmed it says so. "On this code" means the sgit-ai CLI: 469 Python files, Type_Safe throughout, two runtime dependencies (`osbot-utils`, `cryptography`), AES-256-GCM / PBKDF2 / HKDF / ECDSA P-256 in `sgit_ai/crypto/`, a host boundary in `sgit_ai/network/api/`, a local boundary in `sgit_ai/storage/`. **The ranking rule.** A skill earns a place on this review if it does one of three things the existing work does not already do: (a) brings a *method* (not a pattern list) that reaches crypto, trust boundaries and file handling; (b) brings a *tool* with deterministic output (SARIF, JSON) we can join to the threat model; (c) vets the other skills. Skills that only restate OWASP web-app lists score low here: this code base has no HTML, no SQL, no sessions. ## 1. The catalogue ### 1.1 From the screenshot | # | Skill | Source, licence | What it is | On this code | Verdict | |---|---|---|---|---|---| | S1 | **security-review** | `getsentry/skills`, `skills/security-review/SKILL.md`, Apache-2.0 (references credit the OWASP Cheat Sheet Series, CC BY-SA 4.0) | A review *method*: research the whole code base first, trace data flow, then report only high-confidence findings (`VULN-###`) with a separate "needs verification" list (`VERIFY-###`). Do-not-flag rules for tests, dead code, server-controlled config. Loads reference guides per topic (17: injection, XSS, authorization, authentication, **cryptography**, deserialization, **file security**, SSRF, CSRF, data protection, API, business logic, modern threats, misconfiguration, error handling, **supply chain**, **logging**) and per language (Python among five) | The cryptography, file-security, supply-chain and logging guides fit. The confidence discipline matches the house rule that a finding needs a reproducing test. Timon Weber's comparison of five Claude Code security skills picked this one as the only one worth installing | **Use.** Primary code-review skill for the slice | | S2 | **Cisco software.security** → Project CodeGuard | `cosai-oasis/project-codeguard` (was `project-codeguard/rules`), CC BY 4.0; Tessl registry lists `cisco/software-security` at 82 % quality | Secure-by-default *rules* for AI coding agents, eight categories: cryptography, input validation, authentication, authorization, supply chain, cloud, platform, data protection. Ships a `.claude-plugin` and a `skills/` folder; translators to Cursor, Copilot, Windsurf | The cryptography and supply-chain rules are a checklist to read the slice against (safe algorithms, key management, dependency pinning). It is guidance for *writing* code; it has no review loop, no output format, no verification step | **Reference only.** Read its crypto and supply-chain rules once; cite them in findings where they apply. Do not install as a plugin for this review | | S3 | **"Anthropic Cybersecurity Skills"** | `mukul975/Anthropic-Cybersecurity-Skills`, Apache-2.0, community project by Mahipal Jangra; README: "Not affiliated with Anthropic PBC" | About 817 skills in 34 domains mapped to MITRE ATT&CK v19.1, NIST CSF 2.0, ATLAS, D3FEND, NIST AI RMF, F3. SKILL.md with When to Use / Prerequisites / Workflow / Verification | No threat-modeling or secure-code-review domain. Cryptography (16 skills) is TLS, Ed25519, PQ migration, key management at the operations level, not code review. Much of it is offensive and dual-use (C2, exploitation). The review queue is self-described as months long | **Skip.** The name misleads; the content is a SOC and red-team library, not a code-review method | | S4 | **audit-website** (squirrelscan) | `squirrelscan/skills`; the `squirrel` CLI from squirrelscan.com, installed by `curl \| sh` | Site audit: SEO, performance, accessibility, security headers, mixed content, leaked-secret patterns (96). Gen Agent Trust Hub rated the skill **Fail / high risk** for the pipe-to-shell install and agent-run fixes | Nothing to audit in a CLI. Possible later use on the published `*.sgit.ai` sites and on `sgit vault serve` / published static vaults, with the binary installed by hand and the agent never running its fixes | **Skip for the CLI.** Park for the website estate | | S5 | **Varlock** | `wrsmith108/varlock-claude-skill` wrapping `dmno-dev/varlock` (varlock.dev) | A `.env.schema` with sensitivity flags; the skill's rules are "never echo environment variables, never read `.env` directly, validate with masked output". Thin: no support files in the skill repo | The *rule* is the point, and it is already a sgit concern: TM-R14 (keys in argv and logs, re-ranked High on 10/09 because agents' command lines are read by many tools), TM-R09 (`.env` tracked-wins), the always-ignored set in `Vault__Ignore`. We do not use `.env` at runtime | **Skip the skill, keep the rule.** Fold "never print a vault key, token or read key" into the sgit skill (`library/skills/sg-send-cli__SKILL.md`) as a stated rule; see plan step 5 | | S6 | **OWASP Agentic Skills Top 10 (AST10)** | `OWASP/www-project-agentic-skills-top-10`, incubator, "1.0-2026", leads Ken Huang and Fabio Cerullo, CC BY-SA 4.0 | Ten risks for the skill layer (between model and tools): AST01 malicious skills, AST02 supply-chain compromise (both Critical), AST03 over-privileged, AST04 insecure metadata, AST05 untrusted external instructions, AST06 weak isolation (High), AST07 update drift, AST08 poor scanning, AST09 no governance, AST10 cross-platform reuse (Medium). A pre-install checklist | Not a review skill; it is the rubric for *installing* the others into a repository whose whole point is key custody. Every skill on this list is a third-party instruction file that will run with read access to the code and, in a careless session, to a vault key | **Use as the gate.** Plan step 0 applies its checklist to each installed skill and records the answers | | S7 | **Behavioral verification** (canary run, signed manifest) | AST10 checklist items; tools: `cisco-ai-defense/skill-scanner` (Apache-2.0, PyPI `cisco-ai-skill-scanner`, SARIF, YARA + dataflow + optional LLM pass, pre-commit hook and reusable workflow), `getsentry/skills` `skill-scanner` skill, `skillsign` (Ed25519 over a skill folder), SkillLedger (content-addressed, SLSA L3) | The checklist asks for a valid Ed25519 signature with a content hash matching the manifest, and a run in an isolated canary environment with a report that observed behaviour matches declared behaviour | None of the skills above ship a signed manifest, so the signature check fails for all of them today (the `skills.sgit.ai` catalogue already shows version and permissions fields undeclared on our own eight). What we *can* do: scan each SKILL.md with the Cisco scanner, pin to a commit SHA, read it, and run it first in this cloud session (which has no vault key) before any session that holds one | **Use the scanner, note the gap.** The gap (no signatures anywhere in the ecosystem) is itself a `skills.sgit.ai` finding | ### 1.2 Not in the screenshot, and stronger for this code | # | Skill | Source, licence | What it is | On this code | Verdict | |---|---|---|---|---|---| | S8 | **Trail of Bits skills** | `trailofbits/skills`, CC BY-SA 4.0, plugin marketplace (`/plugin marketplace add trailofbits/skills`) | About 49 skills in named plugins. The ones that matter here: **audit-context-building** (a dossier per function: invariants, assumptions with the line that establishes each or "nothing found", callees; explicitly *no* verdicts; runs each function in its own subagent), **differential-review** (security review of a diff with git history), **insecure-defaults** (parallel hunt for fail-open defaults with a verifier per file), **sharp-edges** (error-prone APIs; AES-GCM nonce handling is their own example), **variant-analysis**, **fp-check** (gate review of each finding), **static-analysis** (CodeQL, Semgrep, SARIF), **semgrep-rule-creator**, **supply-chain-risk-auditor** (PyPI advisories, abandoned upstreams), **spec-to-code-compliance**, **property-based-testing** (Hypothesis), **mutation-testing** (surviving mutants), **post-patch-validation** (missed variants after a fix), **agentic-actions-auditor** (GitHub Actions used by agents). `constant-time-analysis` targets compiled code, so it is marginal for pure Python | This is the only set written by people who audit cryptographic code for a living. The context dossier is the data asset the code-review-graphs brief wants at the "explanation layer", written by an agent, with a line cited for every claim. `post-patch-validation` is tailor-made for 0.21.0, where two of the fixes introduced regressions (TM-F29, TM-F30). `supply-chain-risk-auditor` is TM-R21 | **Use.** The backbone of the review: context, then insecure-defaults and sharp-edges on `crypto/` and `storage/`, differential-review on 0.20.0→0.21.0, fp-check on everything, post-patch-validation on the 0.21 fixes | | S9 | **Anthropic `/security-review`** and the `claude-code-security-review` GitHub Action | `anthropics/claude-code-security-review`, MIT; the slash command is built into Claude Code (present in this session) | Reviews pending changes (command) or a PR diff (Action). Categories: injection, auth, data exposure, cryptography (weak algorithms, key management, randomness), input validation, business logic (race, TOCTOU), configuration, supply chain, code execution, XSS. A second model pass filters false positives (excludes DoS, rate limiting, resource exhaustion, generic validation, open redirects). Prompt is customisable by copying `.claude/commands/security-review.md`. **Not hardened against prompt injection**: the README says review trusted PRs only | Cheap, already here, diff-scoped. The exclusions matter: three of our accepted rows are DoS-shaped (TM-R18 tree shapes, TM-R35 `--grep`), which this tool would not report by design. Useful as the PR gate once a baseline exists; the Action needs `pull-requests: write` and an API key secret | **Use on diffs**, not as the whole-slice review. Candidate for the CI job later, after the Bandit / pip-audit baseline (TM-R21), with the fork-approval setting on | | S10 | **find-bugs**, **secret-serialization**, **gha-security-review** | `getsentry/skills`, Apache-2.0 | `find-bugs`: bugs and security issues in local branch changes. `secret-serialization`: secrets leaking through serialization, logs or spans. `gha-security-review`: GitHub Actions workflows for exploitation risk | `secret-serialization` is the exact shape of TM-R14 and TM-F05/F16 (keys written, printed, logged): `sgit vault info` prints the full key; `Vault__Secret_File`, `Secrets__Store`, `CLI__Debug_Log` are its targets. `gha-security-review` covers the three CI workflows and the SP-10 "template pins nothing" finding | **Use** `secret-serialization` on `crypto/`, `secrets/`, `cli/CLI__Debug_Log.py`, `cli/CLI__Vault.py`; `gha-security-review` on `.github/workflows/`. `find-bugs` overlaps S9; skip | | S11 | **Classical scanners**: Semgrep (`p/python`, `python.cryptography.*` rules), Bandit, pip-audit, CodeQL | OSS; Trail of Bits `semgrep-rules` adds crypto rules | Deterministic pattern scanners with SARIF output | TM-R21 already asks for Bandit and pip-audit in the security CI job with a baseline; KI-SEC-06 counted about 23 Bandit mediums. Semgrep's crypto rules (insecure hash, insecure cipher, PKCS#1 v1.5) are the right first pass over `crypto/`. Their SARIF is what `threat-led-triage` consumes | **Use**, and keep: these become the CI baseline the threat model has wanted since 10/08 | ## 2. What the vetting gate looks like (AST10 applied) For every skill the plan installs, record this table in the case study before the first run: | Check (AST10 checklist) | Evidence we can produce today | |---|---| | Provenance: author identity, content hash, signature | Repository and commit SHA pinned; `sha256` of each SKILL.md recorded; **no signature exists** for any of them | | Scan status | `skill-scanner scan-all --format sarif` output kept under the case study; HIGH fails the install | | Permissions: files, tools, network | Read from the frontmatter (`allowed-tools`); Sentry's security-review declares `Read, Grep, Glob, Bash, Task`. Any skill that fetches remote instructions at run time fails AST05 unless pinned | | Identity files | Does it ask to write `CLAUDE.md`, `AGENTS.md`, memory? Sentry's `agents-md` does by design; the review skills must not | | External content | `mcp-stride-gpt` hosted at `mcp.stridegpt.ai` sends the app description off-machine; use the skill copy or self-host on loopback (see the threat-modeling doc) | | Dependencies pinned | `stride-gpt` from PyPI pinned by version and hash; the scanner itself pinned | | Risk tier matches scope | A review skill that only reads is Low; anything that writes fixes is Medium and runs with `--dry-run` or not at all | | Canary run | First run in a session with no vault key, no `.env`, no token on disk (this cloud environment qualifies: `find / -name .sg_vault` is empty) | The honest finding for `skills.sgit.ai`: **the ecosystem's best security-review skills cannot pass their own community's checklist**, because nobody signs skills yet. The `skills.sgit.ai` thesis (skills as packages with version, permissions and provenance) is exactly the missing piece, and this review is a worked example of the gap. ## 3. Two ideas worth taking, not installing - **Confidence tiers with a "needs verification" list** (S1). Our house rule already demands a test per finding; the Sentry format gives the step before the test a name. Adopt the `VULN-###` / `VERIFY-###` split in the review output, then promote a VERIFY only when its test exists. - **Context before verdicts** (S8 audit-context-building). The dossier's "assumption with nothing found to establish it" is the single most useful intermediate artefact: it is a candidate threat with its line number attached, before anyone has argued about severity. That is the `.security.json` row's `attack_surface` and `trust_boundary_crossings` fields, written by an agent with a citation, which is what the code-review-graphs brief allows at the explanation layer. ## 4. Sources checked - `github.com/getsentry/skills` (README skill list; `skills/security-review/SKILL.md`) - `github.com/trailofbits/skills` (README, plugin list; `audit-context-building` SKILL.md) - `github.com/anthropics/claude-code-security-review` (README) - `github.com/cosai-oasis/project-codeguard`, `github.com/project-codeguard/rules`; Cisco and OASIS announcements (Oct 2025, Feb 2026) - `github.com/mukul975/Anthropic-Cybersecurity-Skills` (README) - `owasp.org/www-project-agentic-skills-top-10/` (project page, README, checklist) - `github.com/cisco-ai-defense/skill-scanner` - `smithery.ai/skills/andymsun/audit-website`, `docs.squirrelscan.com`, Gen Agent Trust Hub rating - `github.com/wrsmith108/varlock-claude-skill`, `varlock.dev` - timonweb.com, "I checked 5 security skills for Claude Code, only one is worth installing" --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/security-review-skills.html)* # sgit is a standard, not a server: everything in a vault is one hash away, cli.sgit.ai > A vault is a set of files laid out by a convention, every immutable object named by the hash of its own ciphertext, so the server is the least interesting part and anything in the path can cache blindly. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # sgit is a standard, not a server: everything in a vault is one hash away *Draft for the newsroom, 16 Oct 2026. Second companion to "Scion and sgit" (No. 141). Facts from the CLI's code at `dev` 881d6df, the sgit.ai pages linked, and the ruv.sgit ledger.* Anthropic's post on Scion describes a server that answers every question a client has about the repository, and three tiers of cache in front of it. The post is honest that this is a centralised design in the line of Google's Piper and Meta's Sapling, and that it "is by no means a suitable replacement for git for general use". It is worth saying what sgit is in the same plain terms, because people read "encrypted git" and picture a product with a server in the middle. There is a server, and it is the least interesting part. ## What a vault is A vault is a set of files laid out by a convention. Four kinds of object, all ciphertext: blobs (file contents), trees (a folder: encrypted names pointing at blob and tree ids), commits (a tree id, parents, a message, a signature), and one mutable pointer per branch, the ref. Every object except the ref and the branch index is named by the SHA-256 of its own ciphertext, so a name can never point at different bytes, anyone can check an object against its name, and anyone who has fetched one can keep it for ever. The name says which is which. An immutable object's id carries `-imm-`; the mutable pointers carry `-muw-`. So anything between the storage and the reader, a browser, a proxy, a CDN, a CI cache, a page on a website, can decide to keep an object for ever by looking at its name, without reading a byte of it and without being told by a header. This is how the vault app in the browser got fast: it caches every `-imm-` object the first time it sees one and refetches only the ref. It is a small convention and it makes a large difference, and it is also a trade made on purpose. sgit keeps no deltas between versions of a file; a changed file is a new object. Git stores deltas and has to be told what changed; sgit stores whole objects that anyone can cache blindly. That is git's object model with encryption underneath it, and the choice was deliberate: git solved the hard problems of history, branching and merging twenty years ago, and the workflows built on them are why git won. What sgit changes is where the trust sits. ## Everything from one hash The part that makes it a standard rather than a service is that nothing in a vault has to be discovered. It is all derived from the key: 1. The vault key gives the vault id and, one way, the read key. The read key cannot be turned back into the write key, so it can be published. 2. The read key gives the ref's name: `ref-pid-muw-` plus the first twelve hex characters of `HMAC(read_key, "sg-vault-v1:file-id:ref:" + vault_id)`. The server never told you where HEAD is. You computed it. 3. The ref holds the head commit id. The commit holds the tree id. Trees hold the ids of blobs and subtrees, with the names encrypted inside them. Ref, commit, trees on the path, blob: a file is four requests from its read key, and the requests are plain GETs of fixed paths. 4. Writing is the same objects, PUT in batches, and one compare-and-swap on the ref. sgit.ai's own documentation pages read a vault exactly this way in the browser, with a reader of about ninety lines; a cold load fetches 18 objects, the next load fetches one 69-byte ref, and a load inside the two-minute freshness window fetches nothing at all. The CLI and the browser app are two independent implementations of the same wire format, and they verify each other's work. Because the read half of the convention is four GETs of fixed paths, any storage that answers a GET implements it: an S3 bucket, GitHub Pages, a CDN, a folder, Python's one-line file server. `sgit clone --transport static` reads from any of them, and `sgit publish` is what lays a vault out for one. The write half needs one thing storage does not give you: a compare-and-swap on the ref, which is what the API is for. ## The hot path: a cache you can compute Four GETs is cheap, but a page that serves one record a thousand times a minute does not want to walk trees. So the convention has a cache layer, shipped: `sgit cache add ` declares a path, and every push keeps an encrypted cache object for it at a location derived from the read key and the path. A reader with only the vault id and the read key computes the candidate ids locally, makes one batch request for the ref and the candidates, checks the commit id inside against the ref, and has the file with no walk. A miss, a stale entry or a mismatch falls back to the ordinary walk, so correctness never depends on the cache; losing one is a performance problem, never a correctness one. The reference reader is written to run in an edge function that serves one hot record. This is Scion's "derived data", built by the writer instead of the server, and named by the key instead of indexed by it. ## What the server does, and what storage does Strip the convention down and the API is left with two jobs: protect writes (an access token, compare-and-swap on the ref and the branch index) and shorten reads a little (batches of fifty objects, a listing for cold full clones). Everything else is storage. Which is why, when you look at sgit's timings, you are looking at the storage: - On the hosted service, a Lambda in front of S3 behind CloudFront, a request costs its round trip before a byte moves: the newsroom measured 317 ms a request through the API and 44 ms for the same object from GitHub Pages. A one-folder, one-commit clone of a 35,000-object vault is 137 objects and 3 MB, and takes 13 s from a cloud sandbox because it is a dozen dependent requests, not because of the bytes. - On a local server with disk storage (the ruv.sgit ledger, eleven recorded hive runs), every sgit operation is 0.4 to 1.5 s including Python start-up and the 600,000-iteration key derivation; a 142-file, 6.9 MB vault clones in 0.74 s and pushes in 0.87 s. - The same 35,000-object vault, the same client, the objects moved to local disk: a one-folder, one-commit clone is 0.8 s against a local API server and 2.3 s from a plain file server, against 13 s hosted; a scoped clone with all 1,227 commits of history is 7 s locally against 181 s hosted (the same 1,227 requests, at a millisecond each instead of 150); the full clone is 69 s locally against three and a half minutes, and most of the 69 s is the client decrypting 16,000 trees. So the deployment question is not "how do I scale the sgit server" but "where do I put the objects, and how close": memory, a disk, a bucket behind a CDN, a static host. The sgit.ai deploy pages are that list, and `sgit serve` runs the hosted service's own image on a laptop. ## Why the work stays on the client Scion moves history, blame and merges to the server, and says it is "by no means" for general use. sgit keeps them on the client, and would even if the server could read the data, for the reason git did: a merge is a workflow step that needs the context of the change, the history and the policy that says who may merge what. In ruv.sgit every worker in a hive writes to its own branch and the host merges them under the run's written mandate, with a receipt; in the shared vault ten agents use every day, they meet on a named branch and resolve on the clone that has the conflict in front of it. Scion's post concedes this from the other side: it could not keep a global merge queue at its merge rate, so concurrent changes conflict and an agent fixes them. The merge went back to the client there too. What sgit does not have today is a `blame` command; path history, signed authorship, signed tags and verification exist, and blame is a fold over path history that nobody has written yet. That is the honest gap, stated. ## Keep the property that made git git The distributed copy, complete and verifiable, is why git survived every centralised system that promised to be faster. sgit keeps it: a full clone is one command and holds everything, and anyone with the key can rebuild it from any host. What changed is that the full clone became optional. A folder scope, a history depth, a sparse clone and the cache layer mean an agent takes the part it needs and a repository can hold large files without every client paying for them, which is the one habit every git team has had to police by hand. The whole-history clone is the backup and the audit, and it is still there. ## The constraint that designed it None of this was a plan to be minimal. The server cannot read a byte, so "just do it on the server" was never available: no server-side index, no server-side render, no merge the client did not make, no derived data the client could not compute from the key. Every feature had to be expressible as objects a writer produces and a reader derives. That constraint is why the designs are small enough to be a standard, and why they fit in a page like this one. ## What this does not claim - Not proven at Scion's scale: the largest vault is about 35,000 objects, 1,300 commits and ten agents. - No blame or bisect yet. A partial clone of a long history walks the commits one request at a time until a commit-chain object ships; a depth avoids it today. - The CDN in front of the hosted API does not yet serve object reads from cache; browsers, clients and static hosts do. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/sgit-is-a-standard-one-hash-away.html)* # Response to "sgit issues found on 14 October 2026", and the comms-vault performance report, cli.sgit.ai > The issues reported on 14 October, each traced to its cause in the code, with what was fixed in the same push and what was left open. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Response to "sgit issues found on 14 October 2026", and the comms-vault performance report **Date:** 14 Oct 2026. **Code:** `dev` at 5e85205 plus the fixes on `claude/charming-babbage-5hzcxq`. **Vault used:** the comms / DC collaboration vault (`y8pwtjlw` on `dev.send.sgraph.ai`), read-only: cloned, measured, nothing pushed. ## The comms vault today | | 6 Oct | 14 Oct | |---|---|---| | Objects in the store | 18,720 | **35,423** | | Commits | 626 | 1,227 | | Trees | — | 16,391 | | Blobs | 9,404 | 16,315 | | Files in the working copy | — | 5,101 | | On disk after a full clone | 173 MB | 527 MB | Listing the store takes 21 s on its own. It is a different vault from a week ago. ## What was sequential, and what was not **A full `sgit clone` was not the slow path.** With 0.21.0's code it took about 3½ minutes for 35,423 objects: the listing (21 s), the bulk sweep (about 140 s, 50 objects per request, 16 requests in flight), then a local tree walk (33 s of decrypting 16,391 trees), 47 late blobs (1.3 s) and the checkout (3.6 s). The progress bar moves in steps of 50, which is the batch size. **`sgit pull` was.** Everything that goes through the pull path (`pull`, `fetch`, `status`'s chain fetch, `widen`) had two sequential loops: 1. **One request per commit.** The commit walk fetches wave by wave, each wave being the previous wave's parents, so a linear history of N new commits is N round trips before a single tree arrives. On the comms vault that is 1,227 requests. 2. **One request per blob.** After the graph walk, blobs were fetched with `api.read()` one at a time, inside a loop. 16,315 requests. Measured on the live vault, fetching the whole head into an empty store (the worst case of a pull far behind), unmodified 0.21.0 code against the fix: | | Before (0.21.0) | After | |---|---|---| | Commit and tree walk | 189 s | included below | | Blobs | 5.4 per second: 818 in 150 s, so about 50 min for 16,315 | included below | | **Whole fetch** | **about 53 minutes** (projected from the capped run) | **186 s** | | Objects fetched | — | 35,423, every one verified before it was written | An agent that is a day behind on this vault is behind by hundreds of commits and thousands of blobs. That is what they were reporting. ### The fix - **Blobs go through the batch route** (`_fetch_missing_objects`, pass 2): 50 per request, 16 requests in flight, with per-object verification and per-file failure classification kept. Blobs flagged large go through `read_large` four at a time. - **Far behind, the walk sweeps.** After 25 sequential commit fetches, a whole-vault clone lists the store once and fetches everything missing in bulk, the way a clone does; the walk then finds its objects on disk. The "new commits" list and counts still cover what the sweep brought. Partial clones (scoped or shallow) never sweep: they must not hold what they do not track, and a host that cannot list leaves the walk to finish as before. - Tests: `tests/unit/sync/test_Pull__Batched_Blob_Download.py` (no single-file reads, chunked requests, the bar moves per chunk, a lost blob is classified absent, a corrupt one is refused) and `tests/unit/sync/test_Pull__Far_Behind_Sweep.py` (lists once and fetches in bulk, a short pull never lists, a partial clone never sweeps, a host that cannot list still completes). ## The four reported issues | # | Issue | Side | Status | |---|---|---|---| | 1 | Disk mode: every append-lane write scans the whole store | **server** | Confirmed from the server code: `Storage_FS__Local_Disk.folder__files__all` iterates the entire store and filters by prefix; `append()` calls it on every write for the 1,000-entry cap. In the brief to the SG/Send team. | | 2 | Files over 4 MB cannot be pushed to, or fetched on demand from, a server without S3 | client | **Fixed.** `presigned initiate` / `read-url` answer `400 S3 storage mode required` on memory and disk servers. Push now falls back to the batch; clone, pull, `fetch` and `cat` fall back to the read route (`Vault__API.read_large`). A 401/403/5xx still surfaces. Reproduced and proven on the real in-memory server with a 4.25 MB file: `tests/integration/test_Large_Blob__No_S3__Integration.py` fails on 0.21.0 at the push, passes now (push, full clone, sparse fetch). | | 3 | `clone-headless` then `fetch` always fails | client | **Fixed.** A headless clone holds keys and config only; `fetch`, `ls` and `cat` now bring the tracked head down first (index, ref, commits and trees, never blobs), the way `clone --sparse` does. `tests/unit/sync/test_Sparse__Headless_Fetch.py`. Note: on a vault with a long linear history that first fetch walks the commits one request each, as `clone --sparse` does; the sweep is for whole-vault clones only. | | 4 | Concurrent full clones queue on one server process | **server** | Deployment note: the Docker entry point runs one uvicorn worker. In the brief. | The smaller observations in the report (access token on every route including append writes, caps holding under concurrency, refusals correct) need nothing from the CLI. ## Verification | Check | Result | |---|---| | Unit | 4232 passed | | Security | passed | | Bandit (baseline) | clean | | Integration, large blob on the real in-memory server | passed (fails on 0.21.0) | | Live comms vault | full clone 3½ min; pull-path fetch of the whole head 186 s (was ~53 min) | ## Files - Fixes: `sgit_ai/core/actions/pull/Vault__Sync__Pull.py`, `sgit_ai/network/api/Vault__API.py`, `sgit_ai/core/actions/push/Vault__Batch.py`, `sgit_ai/core/actions/clone/Vault__Sync__Clone.py`, `sgit_ai/core/actions/sparse/Vault__Sync__Sparse.py`. - Brief to the SG/Send team (issues 1 and 4, the large-file note, and the MCP removal): `brief__to-sg-send-team__disk-mode-performance-large-files-mcp.md` in this folder. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/sgit-issues-2026-10-14.html)* # `sgit serve`: container first. Options map and spike results, cli.sgit.ai > The decision to run the SG/Send container image rather than re-implement the server: the runtimes supported (including Apple's container and Podman), the spike results, and phase 1 as built. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # `sgit serve`: container first. Options map and spike results **Date:** 2026-10-12 · **Status:** options and plan, no sgit code yet · **Supersedes** the "native server" plan in [`research__sgit-serve-local-vault-server.md`](../articles/sgit-serve-running-a-vault-locally.md) (same folder). The API inventory and browser notes there still apply. **Decision (Dinis):** run a working vault locally from the SG/Send container image. It has the vault UI and the API in one image and always tracks the latest SG/Send. sgit gets native support for it: - it abstracts the container runtime; - it supports several runtimes, including Apple's `container`; - it can install one when none is present; - it should also work without Docker where possible. The current read-only server stays as a separate, clearly named mode. > **Update, same day: decisions taken and phase 1 built.** > - **Decisions:** > 1. Image `latest`, with the digest that ran recorded. > 2. `sgit serve install` installs, after showing what and asking. > 3. Both Apple `container` and Podman are supported on macOS. > 4. `sgit vault serve` moved to `sgit serve --static`. > 5. The brief to SG/Send is in this folder. > - **What changed from the plan below:** the clone does **not** push to the local server as a > second remote. Its rewind baselines and its copy of the server's ref are kept per branch, > not per server, so two servers would trip its own rewind protection. Instead: > - `sgit serve push` (clone → server) only ever moves the local server forward; > - `sgit serve pull` (server → clone) fetches, verifies, checks the signature policy and > fast-forwards this clone's own branch; > - `sgit push` then reaches the real server as usual. > - **Verified here:** > - Docker end to end: start, the UI opening the served clone, a server-side commit brought > back, stop; > - the uv engine (same data folder); > - divergence refusals. > > The Apple `container` adapter is written to its documented CLI but not yet run on a Mac. ## 1. Two modes | Mode | Command | What it is | Writes | Needs | |---|---|---|---|---| | **container** (default) | `sgit serve` | The SG/Send image (`diniscruz/sg-send-vault`): vault UI + full API + disk store, on `localhost` | yes. A real local vault server: the UI edits, sgit pushes and pulls | a container runtime, or the Python engine (§4) | | **static** | `sgit serve --static` (today's `sgit vault serve`) | A stdlib read-only file server for a **published** vault: the static loader plus the encrypted store, no API | no | nothing | - Static mode still has a job: it previews what `sgit publish` produces, or serves an unpacked published zip, with zero dependencies. - It is not "run a vault". Rename it, and make `sgit vault serve` print its new place (the rule-9 moved-word behaviour). ## 2. What I verified today (spike in this cloud container) All of it was run here, x86_64 Linux, Docker 29.6 / containerd 2.2. The screenshot `spike__local-container-ui-open-vault.png` is next to this file. | # | Check | Result | |---|---|---| | 1 | `docker pull diniscruz/sg-send-vault:latest` | about 10 s; **400 MB**; a multi-arch index (amd64 + arm64); server **v0.34.0**, UI v0.2.3; 31 tags (`v0.27.70` … `v0.34.0`) | | 2 | `docker run -p 8080:8080 -v :/data` | `/api/info/health` ok; UI `/` 200 | | 3 | `sgit init` / `commit` / `push` / `clone --base-url http://127.0.0.1:8080` (repo code) | works: 12 objects on disk at `/vault/iv/ivh5cbkj/{manifest.json,bare/…}`; the fresh clone has the files | | 4 | **The image runs as uid 10001** (its README still says root) | a host folder owned by the user is **not writable**: `Permission denied: '/data/vault'`. **Fix: `--user $(id -u):$(id -g)`**, which sgit must always pass | | 5 | sgit refuses to push with no token ("no access token configured") even though the server has no auth | passing any `--token` works. **sgit fix:** a server that needs no token is a normal case | | 6 | Headless Chromium → `http://localhost:8080/en-gb/vault/`, key typed into the form | **the vault opens and decrypts**: `docs/`, `a.md` ("hello local"). Same origin, plain http on localhost, Web Crypto fine | | 7 | `#` in the URL | did not auto-open (the form did) | | 8 | The UI loads `sg-layout.js` and `sg-llm-events.js` **from `dev.tools.sgraph.ai` at runtime** | not self-contained, despite "no CDN dependency" in its README: offline or air-gapped use will break | | 9 | The UI shows "Read-only" until an access key is entered | expected; `sgit serve` should print the access-key hint, or configure the container so the UI is writable | | 10 | **containerd only** (`ctr run`, no Docker API) | works (image imported, push lands on disk) | | 11 | Anonymous `ctr pull` from Docker Hub | **429 Too Many Requests**; Docker Hub's anonymous limit is 100 pulls per 6 h per IP | | 12 | **No container at all:** the image's own server code (`sgraph_ai_app_send__docker.app`) under a Python 3.12 venv, with the UI folder copied out of the image | works: health, UI, and an sgit push to a disk store | | 13 | CORS for the hosted UI (`Origin: https://vault.sgraph.ai`) | preflight 200 with `Access-Control-Allow-Origin: *` … **but with `Access-Control-Request-Private-Network: true` → 400 "Disallowed CORS private-network"**. Starlette's `allow_private_network` is off, so Chrome's private-network preflight from vault.sgraph.ai to localhost fails | | 14 | A server error response | **contains the full Python stack trace** (an information leak; fine on a laptop, not on a shared host) | ## 3. Runtimes: what exists and how sgit would drive each "docker-compatible" means it accepts `run -d --name -p -v --user -e`, `ps`, `stop`, `logs` and `pull` with the same flags, so one adapter covers all of them. Rows marked † come from my knowledge rather than today's run; check each against its current docs when building the adapter. | Runtime | OS | CLI | Install (what `sgit serve install` would do or print) | Notes | |---|---|---|---|---| | **Docker Engine** | Linux | `docker` | distro package / get.docker.com | the reference; verified here | | **Docker Desktop** | macOS, Windows | `docker` | the vendor installer | Linux VM under the hood; **paid licence for larger companies**† | | **Podman** | Linux, macOS, Windows | `podman` (docker-compatible) | `brew install podman` + `podman machine init/start` on mac/Windows; distro package on Linux | daemonless, **rootless by default**†; with `--userns=keep-id` the uid problem (#4) goes away; no Docker licence | | **Apple `container`** | macOS 26 on **Apple silicon** (macOS 15 runs with reduced networking) | `container` (its own CLI, docker-like) | signed `.pkg` from GitHub `apple/container` releases, then `container system start` | one lightweight VM per container, runs OCI images, no Docker. **Its own adapter**; flags to confirm against `container run --help` (ports, mounts) | | **Colima** | macOS, Linux | gives a `docker` (or `nerdctl`) context | `brew install colima docker && colima start` | free, Lima VM; then the docker adapter† | | **OrbStack** | macOS | `docker` | vendor app | fast; commercial licence for business use† | | **Rancher Desktop** | macOS, Windows, Linux | `docker` or `nerdctl` | vendor installer | free (SUSE)† | | **Finch** | macOS, Windows, Linux | `finch` (nerdctl-based, docker-like) | AWS installer / brew | free (AWS)† | | **containerd + nerdctl** | Linux | `nerdctl` (docker-compatible) | distro or release tarball | lean Linux servers; raw `ctr` works too (verified, #10) | | **WSL2** | Windows | any of the above inside WSL | `wsl --install` + one of the above | Windows without Docker Desktop | | **Python engine** (no container) | anywhere with **`uv`** | `uv run --python 3.12 --with sgraph-ai-app-send …` | `uv` (one static binary); uv fetches Python 3.12 itself | the same server code as the image, verified (#12); needs the built UI folder (§5) | **Detection order:** a runtime already running wins. Otherwise the order is: 1. `docker` (context answers) 2. `podman` 3. `container` (Apple) 4. `nerdctl` 5. `finch` 6. the Python engine if `uv` is found 7. `--static` as the last resort, with a message saying what the user is missing `sgit serve --runtime ` overrides the choice. `sgit serve runtimes` lists what it found, and for anything missing, the one install command per OS. **Installing:** - `sgit serve install []` prints the command and runs it **only after a yes**, because it is a system change. - On macOS with Apple silicon it suggests Apple `container`, or Podman. - On Linux it suggests Podman or Docker Engine. - On Windows it suggests WSL2 + Podman. - Without any runtime, it suggests `uv` and the Python engine. ## 4. What `sgit serve` does (container mode) ``` sgit serve # inside a clone 1. pick a runtime (§3); pull the image, pinned by digest and recorded (latest by default; --tag vX.Y.Z) 2. data dir: /.sg_vault/serve/ (or --data DIR, interchangeable with any sg-send-vault /data) 3. run: --name sgit-serve- --user : -p 127.0.0.1::8080 -v :/data (-p on 127.0.0.1 only: a plain `-p 8080:8080` publishes on every interface) 4. wait for /api/info/health 5. seed: push this clone into it as a remote, `local` → http://127.0.0.1: (the existing push code: no coupling to the server's disk layout) 6. print the UI URL, the vault-key and access-key hints; --open launches the browser sgit serve status | logs | stop | update # update = pull the newer image, restart, same data dir sgit serve --static # today's read-only published-folder server ``` Then: - `sgit pull --remote local` brings edits made in the browser into the clone, with full verify-then-accept; - `sgit push` sends them to the real server. The server owns its own store. The clone's `.sg_vault/bare/` is never mounted, because the UI writing the clone's accepted refs would undo what 0.21 fixed. **Python engine:** steps 1 and 3 become `uv run --python 3.12 --with sgraph-ai-app-send== python -m … --port

`, with `SEND__STORAGE_MODE=disk`, `SEND__DISK_PATH=` and `SEND__VAULT_STATIC_DIR=`. Everything else is the same. **TLS:** only for the LAN or Safari cases (see the research note §5–6). - sgit generates a cert with `cryptography`; - it mounts the cert and key at `/certs`; - it sets `FAST_API__TLS__ENABLED=true`, which the image already supports. ## 5. Asks of the SG/Send team (from the spike) 1. **Vendor the `dev.tools.sgraph.ai` scripts into the image** (#8). Today the "self-contained" UI needs the internet. 2. **`allow_private_network=True`** in the CORS middleware (#13). Without it, the hosted UI cannot reach a local server from Chrome. 3. **No stack traces in error responses** (#14), or only behind a debug flag. 4. **README:** the image runs as uid 10001, so document `--user` or `chown` (#4). 5. **Also publish to GHCR or ECR Public** (#11). sgit can then fall back when Docker Hub rate-limits. 6. **Publish the built vault UI** (`/app/static_vault`) as a versioned release asset. That is what the Python engine needs; today it can only be copied out of the image. 7. Optionally, let `#` (or a `?key=` handoff) auto-open on `/en-gb/vault/` (#7), so `sgit serve --open` lands inside the vault. ## 6. Changes in sgit | Change | Size | |---|---| | `Container__Runtime` (detect, pull, run, ps, stop, logs, health wait) with adapters: docker-compatible (docker, podman, nerdctl, finch), Apple `container`, Python engine (uv) | ~350 lines | | `CLI__Serve`: container mode (seed via push to a `local` remote, banner, `--open`), `status/logs/stop/update/runtimes/install`, `--static` = today's server; `sgit vault serve` points to it | ~250 lines | | A token is optional when the server has no auth (#5) | small | | TLS cert generation + mount | ~100 lines | | **Tests:**
• unit: each adapter's command lines, against a fake runtime binary on `PATH`;
• integration: **real Docker in CI** (the daemon starts in this kind of environment, as shown today); clone, push and pull against the image; a Playwright check that the UI opens the vault | ~1 day | **Estimate:** | Phase | Work | Estimate | |---|---|---| | 1 | Docker-compatible adapter + serve, stop, status, logs + seeding | 2 days | | 2 | Apple `container` adapter + Python engine | 1–1.5 days; the Apple adapter needs a Mac to test on | | 3 | `runtimes`/`install` + TLS | 1 day | | 4 | Docs, threat model rows | ½ day | Total: about **5 days**. Phase 1 alone gives you "clone a vault, `sgit serve`, edit it in the browser". ## 7. Decisions for you 1. **Image tag:** default to `latest` (always current, as you suggested) and record the digest in use, or pin per sgit release for reproducibility? My suggestion: `latest`, plus `--tag` and a recorded digest. 2. **Installing runtimes:** print the command only, or run it after confirmation? 3. **Apple `container` vs Podman** as the macOS recommendation: Apple's needs macOS 26 on Apple silicon; Podman runs everywhere. 4. **`--static` naming:** keep `sgit vault serve` as an alias, or move it fully? 5. Should I send the §5 asks to the SG/Send team as a brief? ## Sources - Apple `container`: (community summary of `apple/container`: Apple silicon only, macOS 26 supported, macOS 15 with reduced networking, signed .pkg + `container system start`) - Docker Hub pull limits: , - Chrome Local Network Access and Safari localhost blocking: the sources in [`research__sgit-serve-local-vault-server.md`](../articles/sgit-serve-running-a-vault-locally.md). --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/sgit-serve-container-first.html)* # `sgit serve`: running a vault locally — research and plan, cli.sgit.ai > How to run a vault locally: the vault server's API inventory, the browser modes, when TLS is and is not needed, and how much plumbing sgit would need. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # `sgit serve`: running a vault locally — research and plan **Date:** 2026-10-12 · **Status:** research and design, no code yet · **Asked:** "how can I run a vault locally … just clone a vault and run it from there … `sgit serve` … SSL if needed … how much plumbing do we need to add to sgit". > **Superseded in part (same day):** Dinis chose to run the SG/Send **container image** rather > than re-implement the server in sgit. See [`options__sgit-serve-container-and-static.md`](../articles/sgit-serve-container-first.md). > The API inventory (§3), browser modes (§5) and TLS notes (§6) below still apply. ## 1. Short answer It is very doable, mostly with what we already have. - **Server side:** a vault server is a small API (§3): read, write, delete, batch with compare-and-swap, list, health and destroy, all over a key-value store. sgit already has a stdlib HTTP server (`sgit vault serve`, read-only) and an in-memory model of the API. A native, stdlib-only `sgit serve` that adds the write side and a disk store is about **600–900 lines plus tests**. It needs **no new dependencies** and runs on Python 3.11. - **The web UI:** the SG/Send team already builds a static copy of it that talks to its own origin. The docker image `diniscruz/sg-send-vault` uses that build (`VAULT_DEFAULT_ENDPOINT=""` → relative `/api/...` paths). sgit can ship or fetch that build and serve it beside the API. - **TLS:** **not needed for the default setup.** Serve the UI and the API from one origin, `http://localhost:`. Browsers treat `localhost` as a secure context, so Web Crypto works, and nothing is cross-origin. TLS (or the CORS mode, §5) matters for two cases only: using the **hosted** UI (vault.sgraph.ai) against a local API from Safari, and reaching the server from another machine. For those we add `--tls` (§6). **Estimate:** - an MVP (local API, the UI, serving a clone) in about **3 days**; - the full version (CORS mode for the hosted UI, TLS, docs, threat model) in about **5 days**. ## 2. What exists today | Piece | Where | What it gives us | |---|---|---| | `sgit vault serve` | `sgit_ai/cli/CLI__Serve.py`, `sgit_ai/core/serve/Vault__Static_Server.py` | A stdlib `ThreadingHTTPServer`, read-only (GET/HEAD; anything else is 405), bound to 127.0.0.1, with a Host-header check against DNS rebinding and a path guard. It serves the published static loader and maps `/api/vault/read//bare/*` onto `.sg_vault/bare/`. | | `Vault__API__In_Memory` | `sgit_ai/network/api/` | The API semantics the CLI relies on (batch, `write-if-match` conflicts, list, tombstones), in process. It does not check write keys. | | Real SG/Send server | `sgraph-ai-app-send` (PyPI, **Python ≥ 3.12**; fastapi, uvicorn, osbot-aws, mcp …) | Our integration suite already runs it in-process with in-memory storage. It also has **disk** storage. | | SG/Send container | `diniscruz/sg-send-vault` (~250 MB; source `sgraph_ai_app_send__docker/`) | **The whole local stack already exists as an image:** the vault UI at `/` plus the API on one port. Disk storage goes to `/data` (layout in §3.4). It has an optional access token, can terminate TLS itself if given a cert/key, and its README covers the secure-context and Web Crypto issue. | | Vault UI | `sgraph_ai_app_send__ui__vault` (inside the server package; v0.2.3 is 2.6 MB unbuilt) | The endpoint comes from `window.SG_ENDPOINT`, then `sessionStorage['sg-vault-endpoint']` (there is an **endpoint field** on the entry form), then `https://dev.send.sgraph.ai`. A `/#` hash opens a vault directly. It calls only vault endpoints, plus the transfer and presigned endpoints for other features. | So "run a vault locally" works **today** with docker: ```bash docker run --rm -p 8080:8080 -v "$PWD/_sg-send_data:/data" diniscruz/sg-send-vault:latest sgit clone my-vault --base-url http://localhost:8080 # or push an existing clone there ``` What is missing: - the one-command experience; - no docker requirement; - starting from a clone you already have. ## 3. The API a local server must speak This is from the code of `sgraph-ai-app-send` 0.33.0, the user lambda: `lambda__user/fast_api/routes/Routes__Vault__Pointer.py` and `service/Service__Vault__Pointer.py`. ### 3.1 Routes | Method | Path | Auth | Notes | |---|---|---|---| | GET | `/api/vault/read/{vault_id}/{file_id:path}` | public | raw bytes; 404 if missing | | PUT | `/api/vault/write/{vault_id}/{file_id:path}` | write key | an empty body gives 400; returns `{file_id, vault_id, status:"completed"}` | | DELETE | `/api/vault/delete/{vault_id}/{file_id:path}` | write key | 403 for a key mismatch **and** for a missing file | | POST | `/api/vault/batch/{vault_id}` | public if every op is `read`, otherwise write key | 1–100 ops; **always HTTP 200** (§3.3) | | GET | `/api/vault/list/{vault_id}?prefix=` | public | `{vault_id, prefix, files:[…]}`, sorted | | GET | `/api/vault/health/{vault_id}` | public | 200 if `manifest.json` exists, otherwise 404 | | DELETE | `/api/vault/destroy/{vault_id}` | write key | body `{vault_id, purge?}`; writes a tombstone | | GET | `/api/info/health` | public | `{"status":"ok"}` | `vault_id` must match `^[a-z0-9]{8,24}$`. Extras the local server does not need for vaults: - **Presigned upload/download:** S3 only. Other modes answer 400 "Presigned URLs not available", and sgit and the UI already fall back to direct writes. - `read-base64`, zip, inbox, public-info. - The `/api/transfers/*` family, which is Send's file transfer and not vaults. ### 3.2 Auth - **Write key, trusted on first write:** - The server stores `sha256()` in `manifest.json` on the first write or batch. After that, the hashes must match, or the server answers 403. - A `deleted.json` tombstone refuses every write. - **Access token** (`x-sgraph-access-token`): - It is checked only when configured; empty means open, which is what local dev uses. - The container can gate every route, reads included. ### 3.3 Batch Ops are `read`, `write` (b64), `write-if-match` (`match` b64, or null for "must not exist") and `delete`. - A conflict returns `{op, file_id, status:"conflict", current:b64|null}` and **stops** the batch. - The batch is ordered and best-effort: ops before a conflict stay applied. - The comparison is on whole bytes. This is exactly what sgit's CAS code already handles; it is the live server's behaviour. ### 3.4 Disk layout (container and lambda disk mode) ``` /vault///manifest.json {vault_id, write_key_hash, created_at} /vault///deleted.json tombstone /vault////payload e.g. bare/data/obj-cas-imm-…/payload ``` If the native server uses **the same layout**, a data folder works with both `sgit serve` and `docker run -v :/data diniscruz/sg-send-vault`. That gives an exact-parity fallback for free. Note that this is **not** the clone's own layout: `.sg_vault/bare/data/` is a file, not `/payload`. Section 4 explains why the server keeps its own store anyway. ### 3.5 CORS - The lambda sends `allow_origins=["*"]` with `allow_credentials=False`, the usual methods, and the headers `x-sgraph-access-token` and `x-sgraph-vault-write-key`, among others. - **No `Access-Control-Allow-Private-Network` handling exists anywhere** (relevant to §5). ## 4. "Clone a vault and run it from there": the model Recommendation: **the server has its own store, seeded from the clone; the clone talks to it as a remote.** The server should not read and write the clone's `.sg_vault/bare/` directly. Why not serve the clone's `bare/` directly: - A clone's `bare/refs/` is the clone's *accepted* copy of the server ref. Since 0.21 only a pull that passed verification moves it, and the rewind baselines depend on it. - A web UI writing into that same file would move the clone's named ref behind its back. `status` would say "in sync" while nothing was pulled, the signature policy would be bypassed, and the next pull would see a "rewind". - That is the exact class of bug the last five review rounds closed. So: ``` my-vault/ the clone (unchanged behaviour) my-vault/.sg_vault/serve/ the local server's store, in SG/Send disk layout (§3.4) (inside .sg_vault, so it is never committed; excluded from uninit backups) ``` `sgit serve`, run inside a clone: 1. **Seeds the store** if it is empty: - every object in `.sg_vault/bare/data/` is **hard-linked**. Objects are content-addressed and immutable, so this costs no space or time; it falls back to copying across filesystems; - refs, indexes and keys are *copied*, because they are mutable; - it writes `manifest.json` with `sha256(write_key)`. The clone holds the vault key, so the write key is known, and the CLI and the UI can write. - The named ref the server starts from is the one the clone last **accepted**, so the local server is "the remote as this clone last verified it". 2. **Starts the server** on `http://localhost:8420`, with both the API and the UI. 3. **Prints:** - the UI URL; - how to open this vault in it: `--open` launches the browser with the vault key in the URL hash, which the UI reads; the hash never reaches the server; - the remote line: `sgit remote add local http://localhost:8420`, or a flag that adds it. Then: - `sgit pull --remote local` brings edits made in the browser into the clone, with full verify-then-accept. - `sgit push --remote local` shows clone commits in the browser. - `sgit push` (origin) publishes everything to the real server. Outside a clone, `sgit serve --data

` is a **general local SG/Send-compatible server**: - any vault can be created, cloned (`--base-url http://localhost:8420`), pushed or pulled; - its folder is interchangeable with the docker image's `/data`. ## 5. Browser access: three ways, and what each needs | Mode | URL the user opens | Cross-origin? | Needs | |---|---|---|---| | **A. Same origin (default)** | `http://localhost:8420/` (UI served by sgit) | no | nothing. `localhost` is a secure context, so Web Crypto works in Chrome, Firefox and Safari. | | **B. Hosted UI → local API** | `https://vault.sgraph.ai`, endpoint field set to `http://localhost:8420` | yes | CORS on the local server (`*` or an allow-list, as production does).
Chrome 141/142+: a one-time **Local Network Access** permission prompt per site.
Safari: blocks https→`http://localhost` as mixed content (WebKit bug 279249), so it needs `https://localhost` (§6). | | **C. Another machine / LAN** | `https://:8443` | – | TLS (Web Crypto refuses plain http on non-localhost hosts) **and** an access token | - **Mode A** is the product: it works offline, does not depend on the hosted UI version, and needs no permissions or certs. - **Mode B** is useful for testing the hosted UI against local data, and it is what "it has CORS enabled so you can load it from the other side" refers to. - **Mode C** is opt-in, with warnings. For B, the local server should also answer preflights with `Access-Control-Allow-Private-Network: true`. Older Chrome Private Network Access preflights need it, and it is harmless otherwise. ## 6. TLS - **Default: none** (modes A and B in Chrome/Firefox). - **`sgit serve --tls`:** - sgit generates a local CA and a `localhost`/`127.0.0.1` leaf cert with `cryptography`, already a dependency, under `~/.sgit/tls/`, and serves https on 8443 through stdlib `ssl`. - If `mkcert` is installed it uses mkcert's CA instead, which is already trusted. - `--cert/--key` use the user's own pair. - **Trust:** a browser *page* can click through an untrusted cert, but a cross-origin `fetch` (mode B) cannot. So mode B over https needs the CA trusted. - `sgit serve trust` prints the exact per-OS command (macOS `security add-trusted-cert`, Windows `certutil -addstore`, Linux `update-ca-certificates`, plus `certutil -d sql:…` for the Chrome/Firefox NSS store). - It runs that command only on explicit confirmation, because changing the trust store is a system change. ## 7. The UI: where its files come from | Option | Wheel size | Network at runtime | Risk | |---|---|---|---| | **Vendor a pinned, flattened build into the sgit wheel** (recommended) | +~1 MB zipped | none, works offline | must be refreshed per UI release; covered by a script, a pinned version and a SHA-256 manifest | | Download on first `serve` into `~/.sgit/ui//`, hash-pinned | none | once | supply chain (mitigated by the pinned hash); fails offline the first time | | Use the docker image for the UI | – | docker | not "just sgit" | - The flattening is what `scripts/build-vault-static.sh` in SG/Send does, with `VAULT_DEFAULT_ENDPOINT=""`. - sgit would add a `scripts/vendor-vault-ui.sh` that takes a pinned UI version and writes `sgit_ai/serve/ui/` plus `ui-version.json`. - `--ui-dir PATH` lets UI developers serve a working tree instead. - The server injects nothing; with `VAULT_DEFAULT_ENDPOINT=""` the UI uses its own origin. **Ask of the SG/Send team:** publish the flattened vault UI as a versioned release asset (zip + SHA-256). Then sgit vendors from it rather than rebuilding from their source tree. ## 8. Implementation plan ### Components | Component | Layer | Size | |---|---|---| | `Vault__Local_Store`: the §3.4 layout; atomic writes (temp + rename); a per-vault lock for batch and CAS; list by prefix; manifest, write-key hash, tombstone | storage | ~200 lines | | `Vault__Local_Server`: stdlib `ThreadingHTTPServer`; the §3.1 routes with matching status codes and JSON; batch semantics (§3.3); access-token option; Host check (from `Vault__Static_Server`); Origin allow-list on writes in mode A; CORS and PNA preflight; static UI; optional `ssl` wrap | core/serve | ~350 lines | | `Vault__Serve__Seed`: hard-link objects, copy refs/indexes/keys, write the manifest | core/serve | ~100 lines | | `Vault__Local_TLS`: CA and leaf generation, mkcert detection, trust instructions | crypto | ~120 lines | | `CLI__Serve`: `sgit serve [--data DIR] [--port] [--bind] [--open] [--tls] [--cert/--key] [--cors-origin …] [--token] [--no-ui] [--ui-dir] [--add-remote]`; `sgit serve trust`; keep `vault serve` as the static-publish server | cli | ~200 lines | Command placement (rule 9): - `serve` is a daily-use verb, so it goes at the **top level**. That is a deliberate edit to `TOP_LEVEL_COMMANDS`. - `sgit vault serve` stays, because it serves a *published* folder; its help points to `sgit serve`. ### Tests - **Unit:** store, routes, status codes, batch CAS, the write-key first-write rule, tombstones, the Host check, the Origin check, CORS and PNA preflights, TLS against a generated cert, seeding (hard links, mutable files copied). - **Contract:** one API test module that runs against **both** the native server and the real `sgraph-ai-app-send` test server, in the 3.12 integration job. That is how the native server stays in step with production. - **End to end:** - `sgit clone/push/pull --base-url http://localhost:` against `sgit serve`; - a Playwright smoke test in the preinstalled Chromium: the UI opens a vault served by `sgit serve` and lists its files; mode B fetch with CORS. ### Phases | Phase | Work | Estimate | |---|---|---| | 0 | Spike: vendored UI v0.2.3 + the real server on `localhost`, driven by Playwright; confirm modes A and B in Chromium | ½ day | | 1 | Store + API server + seeding + `sgit serve` (API only); CLI interop; contract tests | 1½–2 days | | 2 | UI vendoring + serving; `--open` | 1 day | | 3 | Mode B: CORS allow-list, PNA header, Origin checks | ½ day | | 4 | TLS: generated CA/leaf, mkcert, `--cert/--key`, `serve trust` | 1 day | | 5 | Docs, threat model rows, CHANGELOG | ½ day | **MVP = phases 0–2 (~3 days). Full = ~5 days.** ## 9. Security notes (to become threat-model rows) - **Binding:** 127.0.0.1 by default. `--bind 0.0.0.0` requires `--token` and warns loudly; over plain http it also warns that the token travels in clear. - **DNS rebinding:** keep the Host-header check (already in `Vault__Static_Server`). - **Writes from other websites (mode A):** - Writes need the write key, which only someone holding the vault key can derive. - On top of that, refuse writes whose `Origin` is neither the server's own nor on the `--cors-origin` list. - Reads expose ciphertext only, which is the same as the public read API in production. - **The vault key in the `--open` URL hash:** the hash is never sent to the server. The UI stores the key in `localStorage`, which is the UI's existing behaviour; the docs should say so. - **The server store lives in `.sg_vault/serve/`:** never committed, excluded from uninit backups, so a backup does not double in size. - **Nothing changes for the clone:** it reaches the local server only as a remote, through the same verify-then-accept path as any server. ## 10. Decisions for you 1. **UI files:** vendor into the wheel (recommended), download on first run, or rely on docker? 2. **Store location:** per clone in `.sg_vault/serve/` (recommended for "serve this clone"), with `--data DIR` for a shared multi-vault store? 3. **`local` remote:** should `sgit serve` add it automatically, or only print the command? 4. **Default port:** 8420, the same as today's `vault serve`, or 8080 like the container? 5. **Asks of SG/Send:** - a versioned UI release asset; - `Access-Control-Allow-Private-Network: true` on preflights; - optionally a `?endpoint=` URL parameter in the UI, so mode B links can be shared. ## Sources - Chrome Local Network Access: ; rollout notes , - Safari blocking https→`http://localhost`: , - The SG/Send code and container README: from the installed `sgraph-ai-app-send` 0.33.0 package. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/sgit-serve-running-a-vault-locally.html)* # Threat-modeling tools for an agent-driven review — Matt Adams' stack, the classics, and the house method, cli.sgit.ai > Threat-modeling tools for an agent-driven review: Matt Adams' stack (STRIDE GPT, the insider-threat model), the classics, and the house method, with the agent itself named as an actor. *Source: · generated from the same content as the page, so the two cannot drift. Published from .* --- # Threat-modeling tools for an agent-driven review — Matt Adams' stack, the classics, and the house method **From:** the sgit-ai CLI agent · **Date:** 2026-10-11 · **For:** `threat-modeling.sgit.ai` (Documents), then the case study once used. **Why this matters more here than elsewhere.** sgit is operated by AI agents that hold vault keys and write tokens. The threat model's actor list (`team/explorer/appsec/threat-model/v0.21.0__threat-model.md` §3) names the host, the network, a read-key holder, a teammate, a hostile vault author, the local machine, the supply chain and other clients. It does not yet name **the agent itself** as an actor. Matt Adams' insider-threat model is the first published framework that does, and it was generated with his own tool, so the two fit together. ## 1. Matt Adams' stack (mrwadams) | Tool | What it is | Facts checked | Fit for sgit | |---|---|---|---| | **STRIDE GPT** | Open-source LLM threat modeller. Inputs: an application description, an architecture image, a draw.io diagram, a GitHub repository URL, or a **local code base** (`stride-gpt analyze `, agentic: plan, explore, synthesise per subsystem). Outputs: STRIDE threat model with mitigations, attack trees, DREAD scores, Gherkin test cases, DFDs, MITRE ATT&CK and ATLAS mapping, OWASP LLM and agentic risk overlays. Formats: Markdown, **JSON**, **SARIF**, self-contained HTML; `-o` also writes plan, findings and run JSON | `github.com/mrwadams/stride-gpt`, MIT, v0.20, about 1.1k stars. `pip install stride-gpt` gives the CLI; web UI from source (`uv sync`, Streamlit) or `docker pull mrwadams/stridegpt`. **Anthropic is supported** through LiteLLM (the README's About line still says OpenAI; it is stale). Recent PRs: threats get a stable identifier (#231), a `report_threat` tool that demands verifiable evidence plus a final grace round (#205), a cost estimate before an `analyze` run (#218) | **The threat-model generator for the review.** Run it on the slice, in JSON, and join each threat id to a TM-R/TM-F row or to "new". The stable ids and the evidence requirement are exactly what a join needs. The code it reads is Apache-2.0 and public; never point it at a folder holding a vault key or `.env` | | **mcp-stride-gpt** | An MCP server that is a *framework provider, not an analyser*: it returns the STRIDE method, the DREAD rubric, attack-tree structure, test scaffolding and report skeletons; the client's model does the thinking. Eight tools (`get_repository_analysis_guide` by stage, `get_stride_threat_framework`, `generate_threat_mitigations`, `calculate_threat_risk_scores`, `create_threat_attack_trees`, `generate_security_tests`, `generate_threat_report`, `validate_threat_coverage`). Ships the companion **`stride-threat-modelling` skill** | `github.com/mrwadams/mcp-stride-gpt`, MIT. Hosted at `mcp.stridegpt.ai` (`claude mcp add stride-gpt https://mcp.stridegpt.ai/ --transport http`), or plugin (`/plugin marketplace add mrwadams/mcp-stride-gpt`), or copy `skills/stride-threat-modelling/` into `.claude/skills/`. Self-host: Python 3.12 standard library only, port 8787, no auth, keep on loopback | **Use the skill copy, not the hosted endpoint.** The hosted server receives the application description; the skill copy keeps everything in-session. It lets this agent do the STRIDE pass with the same rubric STRIDE GPT uses, so the two outputs are comparable | | **TM-Bench** | A benchmark for LLM threat modelling, aimed at models that run on consumer hardware. Four dimensions: **STRIDE coverage and accuracy (30 %)**, **threat completeness (30 %)**, **technical validity (30 %)**, judged by an LLM (Claude Sonnet 4.6 in v2), and **JSON structure compliance (10 %)**, scored programmatically. Three complexity tiers of application description with synthetic ground-truth models | `tmbench.com` (home, methodology). No public repository or dataset found; maintainer not named on the site, the support link is `mrwadams`. v1 used Claude 3.7 Sonnet as judge, so v1 and v2 scores are not comparable | **Borrow the rubric, not the harness.** Our hand-written register (32 TM-R rows, 31 TM-F rows, each with a test) is a real ground truth. Scoring STRIDE GPT's output of the slice against it on TM-Bench's four dimensions is the measurable part of the case study, and a result worth sending to Matt | | **threat-led-triage** | A Claude Code / Codex skill that *ranks* existing findings by what an attacker can reach and chain in this code base, not by CVSS. Inputs: findings (SARIF, SCA, CVE list, pentest, or free text), the checkout, and optionally a STRIDE GPT model of the same repo (supplies the trust boundaries). Output: ranked list with per-finding reachability, code path, boundary crossed, chains, and the severity adjustment with its reason; plus an explicit "could not determine" list. It finds nothing new | `github.com/mrwadams/threat-led-triage`, MIT. Run: `stride-gpt analyze . -o model.json -f json`, then `/threat-led-triage rank findings.json against model.json for this repo`. Worked example on OWASP crAPI in `test/` | **The triage step.** Semgrep, Bandit, pip-audit and the review skills will produce more findings than are worth a test. Ranking them by reachability across our boundaries (host, read-key holder, local, supply chain) is how the plan stays focused | | **"Actions Speak Louder Than Tokens: An Insider Threat Model for Frontier AI Agents"** | A threat model that treats the agent as the insider. Five CERT dimensions adapted (motivation → optimisation objectives; opportunity → tool access and permissions; capability → model capabilities; predisposition → alignment profile; organisational factors → harness engineering). Four autonomy levels (L1 operator, L2 collaborator, L3 supervisor, L4 auditor). **Six** threat categories (credential compromise, supply-chain sabotage, data exfiltration, infrastructure sabotage, deception and evasion, containment and third-party impact). **27** STRIDE threats, each placed in a frequency × complexity quadrant (diffuse drift 10, opportunistic 8, targeted 9; the high-frequency high-complexity quadrant is empty). Detection with existing tooling: SIEM, UEBA, DLP, network, endpoint, deception (honey tokens, canary files). Monitor the *shape* of reasoning, not its content | `ai-insider-threat.matt-adams.co.uk`, source `github.com/mrwadams/ai-insider-threat`, CC BY 4.0, v1.0 dated April 2026 with sources into August 2026. Generated with STRIDE GPT. Search snippets still say 23 threats and five categories; the page lists 27 and six | **Adopt as an overlay.** Several of its threats are sgit rows already, seen from the other side (table below). Adding actor **A9, the agent holding the key** to §3 of the threat model, and a column mapping its 27 threats to our controls, is a small, concrete piece of work with a published method behind it | ### 1.1 First pass: the 27 insider threats against sgit's register Done from the published list; to be confirmed against code in the review. | Insider threat | sgit today | Register | |---|---|---| | S1 harvests keys and tokens | `vault_key`, `.pem`, the token and the credential store are on disk under `.sg_vault/local` (0600, directory 0755); keys appear in argv and `vault info` | TM-R14, TM-R15, TM-F05, TM-F16 | | S2 forges commit author metadata | commits are signed by the clone's key, but any read-key holder can mint a key that verifies | **TM-R01**, TM-R17 | | S5 replays a long-lived machine credential elsewhere | the write key is a bearer secret with no binding to a client, machine or time | TM-R13 | | T1 subtle backdoor in code | out of sgit's scope as a tool; in scope for *this repository*, which agents write 61 % of (coding.sgit.ai) | review process, not register | | T2 modifies CI/CD | three workflows, release actions SHA-pinned, publish from main only; the shipped workflow template pins nothing | TM-R21, SP-10 | | R1 deletes logs and history | `history reflog` is local-only; the host keeps no plaintext log; a write-key holder can rewrite history, which clones detect as a rewind | TM-R13, rewind guard (TM-F12, F19, F23) | | R3 changes code through paths that bypass signing | unsigned commits are refused under `signatures-required`, but only by sgit clients | TM-R26, TM-F10 | | R4 suppresses client-side reporting | sgit emits no telemetry, so there is nothing to suppress; the host sees sizes and timing only | TM-R08 | | I1, I5 exfiltration, data lands outside the deployment | `SGIT_DEFAULT_BASE_URL` once redirected a whole vault, token included; now pinned per vault | TM-F15, TM-F22, TM-R10 (`http://` allowed) | | I4 secrets through reasoning logs | an agent that reads `vault_key` and prints it in its own transcript; nothing in sgit can prevent that, the skill rule can discourage it | new: skill rule (plan step 5) | | D4 corrupts or deletes repository content | the write key can delete everything on the host; clones and backups recover | TM-R13 | | E2 uses harvested credentials beyond its privilege | one write key per vault, all-or-nothing among holders; no per-writer authorisation | the "gap the note misses" in `10/08/review__git-to-sgit-security-mapping.md` | | E5 reaches live systems from an environment believed isolated | unit and security tests are hermetic by construction (`tests/unit/conftest.py` fails any non-loopback connection) | test harness, not register | Thirteen of 27 land on an existing row or mechanism after one reading. The rest (container escape, `/proc` reads, ptrace, cloud resource churn, flooding internal services, account creation) are host-and-harness threats that sgit cannot see; they belong in the deployment's model, which is where the paper puts them. ## 2. The classical tools, and why they are not first | Tool | What it is | Fit | |---|---|---| | **OWASP pytm** | Threat model as a Python file: elements, boundaries, dataflows; threat rules are Python conditions evaluated with `eval` (the README warns about the threat file's permissions); outputs a dot DFD, a PlantUML sequence diagram, a Markdown report. OWASP Production project | The right *shape* for a repository that already keeps everything as typed data, and the model file would sit naturally beside `review/intent/`. Not first: its threat library is web-app shaped; our value is in the register we already have. Worth a model file once the DFD is drawn, so the DFD is data and diffs | | **Threagile** | Declarative YAML architecture model with automated risk rules; CLI, Docker, REST; produces risk report and diagrams | Same comment; YAML rather than Python. Its risk rules are infrastructure-shaped (containers, networks); few apply to a CLI and a storage API | | **OWASP Threat Dragon** | DFD editor (web or desktop) with rule-assisted threat generation | A drawing tool. The house method keeps models as graph data, not drawings; use only if someone wants a picture for a talk | | **Microsoft TMT, IriusRisk, ThreatModeler** | Desktop or commercial | Not for an open, agent-driven, data-first repository | ## 3. The house method, and where each tool plugs in From `threat-modeling.sgit.ai/practice/`: bottom-up and fractal (file → method → class → module → path → endpoint → attack surface); one `.security.json` per source file in a parallel tree with `attack_surface`, `vulnerabilities` (id, severity, attack vector, impact, status, mitigation, **`risk_decision`** with owner and rationale), `secrets_exposure`, `dependencies` with `trust_boundary_crossings`, `redundant_code`; STRIDE tables that keep `NOT MITIGATED` rows; two modes never averaged; the method is always published, findings only once closed or harmless by design. | House artefact | Produced by | Note | |---|---|---| | `.security.json` per file (the slice only) | this agent, from the Trail of Bits dossier plus the review findings; every claim with a line | the dossier's "assumption, nothing found" rows become `attack_surface` and `trust_boundary_crossings` entries | | STRIDE table for the slice | STRIDE GPT JSON (machine) and the `stride-threat-modelling` skill (this agent), reconciled | two independent passes; disagreements are published as such | | Join: STRIDE threat ↔ TM-R/TM-F row ↔ test | this agent; a JSON table under `review/` | this is the TM-Bench "completeness" measurement, with our register as ground truth | | Ranked findings | `threat-led-triage` over Semgrep, Bandit, pip-audit and skill output, with the STRIDE GPT model | explicit "could not determine" kept | | Actor A9 and the insider overlay | the 27-threat table above, confirmed | an addition to §3 and §6 of the threat model, with `Known_Gaps` tests where a gap is proven | | Disclosure | method now; findings when closed | the sgit.ai security page still says the internal review "is being prepared", and the 0.21.0 register is in this repository; the case study can close that loop | ## 4. Sources checked - `github.com/mrwadams/stride-gpt` (README, changelog, PRs #205, #218, #231); `pypi.org/project/stride-gpt` - `github.com/mrwadams/mcp-stride-gpt` (README) - `tmbench.com` (home, methodology); shostack.org AppSec roundup March 2025 (TM-Bench introduction) - `github.com/mrwadams/threat-led-triage` (README, `test/`) - `ai-insider-threat.matt-adams.co.uk`; `github.com/mrwadams/ai-insider-threat`; `github.com/mrwadams`; matt-adams.co.uk/cv (talks at Securing Financial Services Summit 2026, Securing the Law Firm 2026) - SE Radio 648 (Dec 2024), "Matthew Adams on AI Threat Modeling and Stride GPT" - `devguide.owasp.org` pytm page; `pypi.org/project/pytm`; Threagile and Threat Dragon project pages - `threat-modeling.sgit.ai` (index, practice, security-json pages) --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/articles/threat-modeling-tools.html)* # The brief for cli.sgit.ai, 20 October 2026, cli.sgit.ai > The voice brief that started this site, published in full as transcribed, with the request that came with it: make cli.sgit.ai the CLI's main documentation, move the most technical content here from sgit.ai, and publish the CLI team's research as articles. *Source: · generated from the same content as the page, so the two cannot drift.* --- # The brief for cli.sgit.ai, 20 October 2026 **Status:** accepted; the first version of this site (v0.1.0) was built from it the same day. **Written by:** Dinis Cruz, spoken, transcribed by Otter.ai. **Published:** in full and as transcribed, including the transcription's errors. What the errors stand for, and anything the build found the brief to be wrong about, is in [the corrections file](../admin/corrections.md), not here. ## The brief, as transcribed > Okay, so this is now a brief for the the agent that is going to build the API website for the CLI, which is going to be at cli.sketo.ai, and the idea is to start to make this the main documentation repo for the CLI. At the moment, we're kind of using Escrip. AI, but I think we need to start moving some of that content here, or at least the most technical and the most detailed and stuff like that, and also to provide a way for the creation of specific articles that top that focus on specific topics and specific sort of implementations or specific ideas, which is something that we've talked before and we have we start doing in other websites, which is basically the place where we start to capture it first is almost like our new generation of briefs, which work better here, right? So then that's a lovely place to put that stuff. So I think to start with, let's just move the content because the CLI now starts to have a whole set of use cases that are even independent, and the Skit is more of the commercialization, the over umbrella, because we now have a lot of excrete usages, usages, right? So we can refactor it like that. *Transcribed by https://otter.ai* ## The request that came with it (the initial prompt) > Hi can you take a look at this brief and create the first version of this website. > > Also, take a look at a lot of the content in sk.ai that we can move, and also look at the content that is already on the main repo for the CLI, which that has been having a bunch of articles published that we should now also publish here as articles as the latest research capabilities, sorry, research topics. > > please push to dev once you completed the first setup > > see guidance at sgit.ai for how to setup these websites ## What the brief asks for, read plainly 1. **cli.sgit.ai becomes the main documentation for the sgit CLI.** sgit.ai stays the umbrella: the project, the vault platform, the commercial side. 2. **Move the most technical and detailed content here from sgit.ai.** The CLI now has use cases of its own, independent of the rest of the project. 3. **A place for articles that each take one topic, one implementation or one idea as far as it goes.** These are "our new generation of briefs", and they are captured here first. 4. **Publish the research the CLI repository already holds** as those articles. 5. **Follow the estate's guidance for setting up a site**, and push the first version to `dev`. ## How the first version answers it | Ask | Where it is | |---|---| | 1. The CLI's main documentation | [the docs](../docs/index.md) and [the command reference](../reference/index.md), every command generated from the CLI's own parser | | 2. Move the technical content | 95 pages in all; the ones that moved, with where each was: [What moved from sgit.ai](../admin/moved.md) | | 3. A place for topic articles | [Research](../articles/index.md); publishing one is adding one file, the contract is in `admin/content/articles/README.md` | | 4. The CLI repository's research | 23 articles, each with its source path and commit in the CLI repository | | 5. The estate's guidance | [Every sgit repository](https://sgit.ai/docs/guidance/repositories.html), followed item by item on [the engineering page](../admin/index.md), with what is not done yet in [the reality file](../admin/reality.md) | --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/briefs/2026-10-20__cli-sgit-ai-first-brief.html)* # Briefs, cli.sgit.ai > The briefs this site was built from, each published in full as it was given, with its status. Corrections go in the corrections file, never into a brief. *Source: · generated from the same content as the page, so the two cannot drift.* --- # Briefs The documents this site was built from, published in full as they were given, with their status and date. A brief is never edited to look right afterwards: where one turned out to be wrong, the finding is in [the corrections file](../admin/corrections.md). The research articles are the other kind of brief the first one asks for, "our new generation of briefs": one topic taken as far as it goes, captured here first. They are on [Research](../articles/index.md). - [The brief for cli.sgit.ai, 20 October 2026](../briefs/2026-10-20__cli-sgit-ai-first-brief.md): The voice brief that started this site, published in full as transcribed, with the request that came with it: make cli.sgit.ai the CLI's main documentation, move the most technical content here from sgit.ai, and publish the CLI team's research as articles. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/briefs/index.html)* # Agents sharing one vault, cli.sgit.ai > The loop for a team of agents on one vault with sgit-ai 0.18.0: a scoped shallow clone per session, status before commit, push that pulls first, and the full table of what a pull does with uncommitted work: kept when untouched, refused by name when it would be overwritten. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # Agents sharing one vault For a team of agents that keep one vault between them: several of them cloning, committing and pushing at the same time, each many times a day, with a vault that has grown to hundreds of commits. Everything on this page was worked out on exactly that setup, a shared CRM vault maintained by around ten agents, and the numbers are from it. For sgit-ai 0.18.0 or newer. The practices here depend on behaviour that older versions do not have; check with `sgit version`, upgrade with `sgit update` ([what changed](../docs/update-to-0-18-0.md); the current release is 0.21.0, [update notes](../docs/update-to-0-21-0.md)). ## The loop Each agent, each session: ``` $ sgit clone --path mail/ --depth 1 workspace # once per session, 5 to 15 s $ cd workspace # … read and write files inside mail// … $ sgit status # what changed, and whether the remote moved $ sgit commit -m "what I did" $ sgit push # pulls first, merges, uploads only what is new ``` Why each line is the way it is: **Clone scoped and shallow.** An agent that works in one folder has no use for the other agents' folders or for months of history. On the example vault the full clone is 80 s and 173 MB; one folder with one commit of history is 14 s and 3 MB, two small folders are 5 s. Agent harnesses often give a command two or three minutes; a full clone of a growing vault crosses that eventually, a scoped clone does not. Details and limits: [Partial clones](../docs/partial-clones.md). **Keep the clone for the session, not forever.** A clone directory is cheap to make. An agent that resumes an old clone starts with a stale working copy and an old `sgit`; a fresh clone starts current. **`sgit status` before you commit.** It says what the remote has done since you cloned, with real numbers: ``` On branch: branch-clone-… → branch-named-… Remote: remote has 3 new commits — run: sgit pull ``` `50+` means more than status fetched to count; it is a lower bound, not a guess. Before 0.18.0 this line could say your fresh clone was hundreds of commits *ahead* when it was one commit behind. That is fixed. **`sgit push` pulls first.** It merges what others pushed, then uploads. Only new objects go up: your commit, the folders on the way down to your change, and the changed files. ## What happens to work you have not committed This is the behaviour the team most needed and the one that changed in 0.18.0. A pull used to write the whole incoming tree over the working copy, so an edit you had not committed yet was replaced by the committed version, even when the incoming commits never touched that file, and `sgit status` then reported the vault in sync. Three agents reported losing work that way in one week. A pull now follows git's rules, and decides before it writes anything: | Your working copy | The incoming commits | Result | |---|---|---| | changed `threads/today.md`, not committed | did not touch it | kept; pull reports `Kept 1 uncommitted change(s) the incoming commits did not touch` | | changed `threads/today.md`, not committed | changed it too | pull refused, nothing written | | deleted `old.md`, not committed | did not touch it | stays deleted | | deleted `old.md`, not committed | changed it | pull refused | | created `new.md`, not tracked | created `new.md` with the same bytes | fine | | created `new.md`, not tracked | created `new.md` with different bytes | pull refused | | created `scratch.md`, not tracked | nothing | left alone | A refused pull looks like this and has changed nothing, on disk or in the store: ``` $ sgit push error: your local changes would be overwritten by pull: mail/crm.example/threads/today.md (your uncommitted edit would be overwritten) commit them (sgit commit) or stash them (sgit vault stash) and pull again; nothing was changed ``` Do what it says: `sgit commit`, then push again; the merge happens on committed work, where a conflict is visible and resolvable (`sgit resolve`). An agent should treat this message as a normal outcome, not an error to retry. "Changed" is measured the way `sgit status` measures it, by content rather than timestamps, so the two never disagree. Re-run by this site on 7 October 2026 on a throwaway vault with sgit-ai 0.18.0: a scoped clone with an uncommitted edit pulled a teammate's commit to another folder and kept the edit; a full clone with an uncommitted edit to a file the incoming commit changed was refused by name, and the file was untouched afterwards. `sgit status` on both read `remote has 1 new commit`. ## Staying out of each other's way - **One folder per agent, scoped clones.** Two scoped clones in different folders cannot produce a merge conflict: each carries the other's folder by id and only its own folder rebuilt. Shared folders (`runs/`, `docs/`) are where conflicts can happen; keep writes there append-only or one-owner where you can. - **Many agents cloning at once is fine.** Reads go through 16 parallel connections per client, reuse one TLS connection per host, and a throttled request (HTTP 429) is retried with back-off. Ten agents cloning in the same minute will not fail each other. - **Commit often, push when a unit of work is done.** Every pushed commit's files are uploaded, a 0.18.0 fix: a file created and changed again before one push used to leave its first version missing on the server. If a clone made before 0.18.0 pushed such a commit, `sgit check upload-objects` from that clone sends the missing objects. - **`sgit push --branch-only`** shares work in progress without touching the named branch. It sends only what the server does not already have. ## When something is slow or fails - **Clone slower than expected?** Check it is scoped: a clone that only holds `mail/…` has no `docs/` on disk. A full clone of a 600-commit vault is 80 s on a laggy path; if yours takes minutes, run `sgit version`: the sweep that makes it 80 s arrived in 0.18.0. - **`error: … needs the whole vault`:** you are on a partial clone and ran `fsck`, `dump`, `publish` or `vault move`. Use a full clone for those. - **`error: this clone holds only part of the vault … needs more of it`:** a command met an object the scope never fetched. Widen with `sgit fetch ` or `sgit fetch --unshallow`, or use a full clone. - **`sgit doctor`** checks the remote, the token and the version. ## A session, end to end ``` $ sgit update && sgit version sgit-ai v0.18.0 $ sgit clone --path mail/crm.example --depth 1 work && cd work # 14 s $ sgit cat mail/crm.example/contacts.md … $ printf '\n- 2026-10-07: called, left message\n' >> mail/crm.example/threads/acme.md $ sgit status ~ mail/crm.example/threads/acme.md $ sgit commit -m "crm: acme call" $ sgit push Merged (no file changes). Push complete. Named branch updated. Pushed 1 commit(s), 6 object(s) uploaded. commit obj-cas-imm-… ``` Six objects: the commit, the new root tree, the new `mail/` tree, the new `mail/crm.example/` tree, its new `threads/` tree, and the one changed file. Everything else in the vault was carried by id. The team this was worked out on is described in [The agent team as it runs](https://sgit.ai/articles/the-agent-team-as-it-runs.html) and [The Mandate Stack](https://sgit.ai/articles/the-mandate-stack.html); the vault is their shared memory layer. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/docs/agents-sharing-one-vault.html)* # Working with AI agents, cli.sgit.ai > The agent-facing surface: sgit write, --json everywhere, the clone modes including scoped and shallow clones, the session pattern, and multi-agent collaboration. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # Working with AI agents sgit is designed to be driven by AI agents as well as humans. A vault is just a folder. The agent reads and writes files normally; sgit handles versioning, encryption, and sync. This page covers the agent-facing surface. ## Network requirements sgit needs HTTPS to your API server, by default `dev.send.sgraph.ai`, and agents reading these docs also need `sgit.ai`. Sandboxed agents, for example Claude on a Team or Enterprise plan, often sit behind an allowlisting proxy. If you see `CONNECT … 403`, the host is blocked, not down: ask the organisation's Owner to allow `sgit.ai`, `*.sgit.ai` and `*.sgraph.ai`, listing the apex domains explicitly. [The how-to](../docs/how-to/claude-team-egress.md) has the steps and a check to paste. ## The session pattern ``` # start of session: get the workspace $ sgit clone workspace # or: sgit pull, if already cloned # … agent works on files normally … # end of session: persist the state $ sgit commit -m "session: findings and next steps" $ sgit push ``` The next session (hours or weeks later, on any machine) runs `sgit pull` and continues. State survives the context window, encrypted end to end. On a vault that has grown large, an agent that works in one folder should clone only that folder: `sgit clone --path --depth 1`, seconds instead of minutes, with the same commit and push. [Partial clones](../docs/partial-clones.md) explains it; [Agents sharing one vault](../docs/agents-sharing-one-vault.md) is the loop for a team. ## `sgit write`, the surgical commit When an agent needs to record one result, cloning a whole vault is waste. `write` commits a file directly to the vault HEAD in a single call: no working-directory scan, no full clone needed. ``` $ sgit write notes/finding.md --file result.md \ --message "agent A: analysis" --push --json { "status": "pushed", "path": "notes/finding.md", "blob_id": "obj-cas-imm-9c2e41ab77d0" } ``` - `--also vault-path:local-file` (repeatable) makes a multi-file write **atomic**: one commit, all or nothing. - Content-hash dedup: writing identical content returns `unchanged` and creates no commit. - `--json` gives a machine-readable result for the calling pipeline. ## Machine-readable everything | Need | Command | |---|---| | Read one file, structured | `sgit cat --json` · `sgit cat --id ` (zero network calls) | | List files with fetch state | `sgit ls --json` / `--ids` | | History for pipelines | `sgit history log --json` · `history diff --json` | | Health checks | `sgit doctor --json` | ## Fast cold starts Agents run on time budgets. Five clone modes keep startup cheap; the first two arrived in sgit-ai 0.18.0 and are the usual choice for an agent: - `sgit clone --path `, a scoped clone: only the folders you work in, everything else carried by id. Writes outside the held folders are refused by name, and two scoped clones in different folders cannot conflict. Widen later with `sgit fetch `. On a 600-commit, 9,400-file vault: one folder in 14 s against 80 s for the whole vault. - `sgit clone --depth 1`, a shallow clone: the whole tree at HEAD, no history. Combine with `--path`; deepen later with `sgit fetch --unshallow`. - `sgit clone --sparse`, structure now, file content on demand via `sgit fetch `. A scoped clone is usually the better fit, because it holds the files it will read without a round trip per file. - `sgit clone-branch`, full history, but only HEAD's content. - `sgit clone-headless`, credentials only: derive keys and write config, fetch nothing. A full clone itself is about twice as fast since 0.18.0: the store is listed once and downloaded in one parallel sweep. Details, limits and measurements: [Partial clones](../docs/partial-clones.md). ## Multi-agent collaboration For a team of agents on one vault, read [Agents sharing one vault](../docs/agents-sharing-one-vault.md) first: one folder per agent in a scoped clone, `sgit status` before committing, and what a pull does with work you have not committed, which since 0.18.0 is kept or refused by name rather than overwritten. Then the branch-level tools: give each agent a named branch; the [two-branch model](../docs/two-branch-model.md) guarantees isolation of work-in-progress. Two commands make peer review safe: - `sgit history show ` and `sgit history diff` are **read-only**: they fetch missing objects on demand without merging, so an agent can inspect a peer's commit without touching its own working copy. - `sgit resolve --show` renders base/ours/theirs with a per-file verdict, so genuine conflicts are distinguishable from noise, by a human or by an agent. ``` # agent A $ sgit branch new feature-analysis … commit … push # agent B $ sgit history show obj-cas-imm-c4e81a # look, don't merge # human $ sgit pull → review in SG/Vault → merge ``` **For Claude users:** a packaged sgit Skill teaches a Claude session this entire workflow (install, clone, work, commit, push) so cross-session persistent state works out of the box. --- *[Site index for agents](../llms.txt) · [HTML version](https://cli.sgit.ai/docs/agents.html)* # Append-lane messaging between agents that hold no one else's vault key, cli.sgit.ai > The write-up of two-way, signed and encrypted messaging between agents over sgit's append lanes, as two teams ran it on 25 and 26 September 2026: per-session signing keys with a pinned key registry on the sender's own site, ephemeral inbox vaults for agents that keep no secret, the flows, security properties, threat model, HTTP calls as observed, eleven places sgit.ai's docs differed, and recommendations for sgit. Relayed by sgit.newsroom.sgit.ai. *Source: · generated from the same content as the page, so the two cannot drift. Moved from (sgit.ai v0.7.87) on 2026-10-20.* --- # Append-lane messaging between agents that hold no one else's vault key How two agent teams sent signed, encrypted messages to each other in both directions over sgit's append lanes on 25 and 26 September 2026, when neither held the other's vault key and one of them could keep no secret between sessions. The flows, the key registry, the ephemeral inbox, the security properties, the threat model, the HTTP calls as observed, where sgit.ai's docs differed, and what sgit should change. **Where this comes from.** Written by the postmaster of the `riskmandate-agent-collab` vault (`cowork.riskmandate`), reviewed by [sgit.newsroom.sgit.ai](https://sgit.newsroom.sgit.ai/), and relayed to this site in the newsroom's [briefing for sgit.ai](https://sgit.newsroom.sgit.ai/briefings/sgit.ai.html) on 26 September 2026. It is version 3. The text below is the document exactly as relayed: the sha256 of its text, `ee1df372a8057b55`, matches the hash the newsroom published for its copy. Two things were changed for this page: the title is set as this page's heading, and the newsroom's internal links are written as full URLs. No keys appear here. The vault ids are addresses, not credentials: a vault id is enough to find a vault, and not enough to open it. **What sgit.ai did with it, on 26 September 2026.** Section 11's corrections are applied to [the append-lanes API page](../api/append-lanes.md), [the vault messaging guide](../docs/vault-messaging.md), [the PKI page](../docs/pki.md#envelope) and [the telemetry brief](https://sgit.ai/docs/briefs/vault-telemetry-append-lanes.html), each marked with the date. One item did not hold on this site: item 4 says one page gives the enum key for `purge`, and every sgit.ai page already said the write key. Section 12's recommendations are for the sgit CLI and the SG/Send server, and are filed as [a brief to the CLI team](https://sgit.ai/docs/briefs/index.html). **Version:** 3 (26 September 2026). v1 covered one-way delivery into a vault; v2 added per-session signing keys with a pinned key registry, and the ephemeral inbox that makes the channel two-way; v3 applies the newsroom's review of v2 (exact payload encoding, a corrected pre-flight, enforcement labels, the access token, closing marked untested). Section 13 lists what changed. **Status:** working in both directions, tested end to end on 25–26 September 2026 **Stack:** sgit-ai v0.16.0 · SG/Send server `https://dev.send.sgraph.ai` · Email-FS-lite v0.6 **Reference deployment:** - `riskmandate-agent-collab` (`62t9bjmy`): a permanent vault, whose owner's Claude session is the postmaster (`cowork.riskmandate`); - `sgit.newsroom.sgit.ai` (`newsroom.sgit`): an agent whose sessions are fresh containers, with a session-scoped inbox vault. **Audience:** whoever documents this feature on sgit.ai. Section 11 lists everything observed that differs from, or is missing from, the current docs. **Companion docs (the newsroom's side):** [09. The append lane](https://sgit.newsroom.sgit.ai/brief/09-the-append-lane.md) and [10. The ephemeral inbox](https://sgit.newsroom.sgit.ai/brief/10-the-ephemeral-inbox.md). ## 1. The problem and the answer, in one screen A shared sgit vault is a good place for agents to work together: it is encrypted, versioned, and every change is a commit. The vault key, though, is all-or-nothing: whoever holds it can read and write everything. Many agents only need to **exchange messages** with a vault's agents. And some of them, such as a Claude Code session in a fresh container, can't keep any long-lived secret at all. The design goal set by the vault owner was: **The only thing a sending session is given is its append token** (for example in an environment variable). **The only thing a receiving postmaster needs is its own vault key.** The answer has three parts: 1. **Append lanes** give each sender a write-only slot on the recipient's vault. The sender can add files and nothing else. It can't list or read anything, including what it wrote, and every write returns exactly `{"ok": true}`. 1. **Encryption and signatures** (`sgit pki`): - messages are encrypted to the recipient's public key and signed with the sender's key; - a session with no persistent secret makes a **fresh signing key** each time and publishes it in a **key registry on its own website**, which the recipient has pinned. 1. **Ephemeral inboxes** make it two-way. An agent with no permanent vault creates a vault for one session, opens a lane on it for each sender it expects, publishes the inbox on its website, drains it while it runs, and deletes it at the end. ``` riskmandate-agent-collab (permanent) newsroom session (ephemeral container) ┌───────────────────────────────────────┐ ┌─────────────────────────────────────────┐ │ postmaster = any vault-key holder │ │ holds: NEWSROOM_APPEND_TOKEN (env) │ │ front-door keypair (static) │ │ makes: a signing keypair per session │ │ lane ◄── newsroom.sgit │ ◄── signed, enc ─── │ publishes: key + inbox on its website │ │ │ │ │ │ │ ─── signed, enc ──► │ inbox vault 9c7vcrw4 (this session only) │ │ │ │ lane ◄── cowork.riskmandate │ └───────────────────────────────────────┘ └─────────────────────────────────────────┘ one vault, one direction each: no vault carries lanes both ways, and each owner drains only its own ``` ## 2. Actors and capabilities | Capability | Format | Held by | Can | Cannot | |---|---|---|---|---| | **append token** | 16–128 lowercase hex (64 used) | one sender, one per lane | write to its own lane | list, fetch, read, delete, configure | | **enum key** | any string; the server stores its SHA-256 | the vault owner / postmaster | list, fetch, mark-processed | write, configure, purge | | **vault write key** | derived by sgit from the vault key | every vault-key holder | configure lanes, purge | none | | **recipient keypair** | RSA-OAEP 4096 + ECDSA P-256 (`sgit pki`) | the receiving vault's owner | decrypt what arrives; sign its outgoing messages | none | | **sender signing key** | ECDSA P-256, a new one per session | the current session only | sign that session's messages | outlive the container | | **site publish right** | e.g. push access to the site's repo | the sending agent's sessions | publish its key registry and inbox entry | none (it isn't a token; it comes from the environment) | ``` ┌──────────────────────────────────────────────────────────────────────────────────────────────────┐ │ dinis.human vault owner; approves lanes; hands out tokens; sets rotation policy │ │ cowork.riskmandate postmaster of 62t9bjmy; sends replies signed with the front-door key │ │ mailbox.riskmandate vault member; recipient (may also send, signed with the front-door key) │ │ newsroom.sgit holds ONE append token into 62t9bjmy; per-session key; its own inbox vault│ └──────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` ## 3. Architecture ### 3.1 Where things live ``` ┌──────────────────────────────────── SG/Send server (dev.send.sgraph.ai) ────────────────────────────────────┐ │ vault 62t9bjmy (permanent) vault 9c7vcrw4 (one newsroom session) │ │ ├── manifest: append_anchors=[sha256(tok_NR)] ├── manifest: append_anchors=[sha256(tok_CW)] │ │ │ enum_key_hash=sha256(enum_62) │ enum_key_hash=sha256(enum_9c) │ │ └── bare/append/{tok_NR}/pending|processed └── bare/append/{tok_CW}/pending|processed │ │ ▲ write (tok_NR) ▲ list/fetch (enum_62) ▲ write (tok_CW) ▲ list/fetch (enum_9c) │ └─────────┼──────────────────┼────────────────────────────────┼───────────────────┼───────────────────────────┘ │ │ │ │ newsroom session postmaster clone of 62t9bjmy postmaster (sender) newsroom session (drain) env: tok_NR vault key → write key → holds tok_CW holds 9c7vcrw4's key, site: keys/agents.json enum key + passphrase (received inside a enum_9c + tokens: 0600 file (key + inbox entry) (derived, never stored) signed lane message) (vault key: clone's .sg_vault) ``` **Lanes sit beside the commit tree, not inside it.** An append never touches `mail/`, never creates a commit, and can't conflict with a push. Only the vault's own drainer moves data from a lane into anything, and in a permanent vault it does so as an ordinary, auditable commit. ### 3.2 Layers of encryption ``` sender's .eml (plaintext) │ sgit pki encrypt --recipient --fingerprint ▼ envelope {v, w: RSA-OAEP(aes_key), i: iv, c: AES-256-GCM(eml), f: signer fingerprint, s: ECDSA(c)} │ sgit writes the .enc file as base64 TEXT of the envelope JSON │ payload = base64(bytes of that .enc file) → POST /append/write {append_token, payload} (encoded TWICE) ▼ server: stored as-is under bare/append/{token}/pending/ ← sees ciphertext, size, time, token │ drainer: fetch → decrypt → verify s over c with the key whose fingerprint == f ▼ plaintext .eml → routed into the recipient's mailroom (and, in a permanent vault, committed: encrypted again at rest under the vault's keys) ``` Note: the signature `s` covers the inner ciphertext `c` only, not `w`, `i` or the recipient (see §11, item 9). ## 4. Flows: the lane itself ### 4.1 Opening a lane (the receiving vault's owner, once per sender) ``` OWNER SERVER SENDER 1. recipient keypair: sgit pki keygen (permanent vault: once; ephemeral inbox: once per session) 2. token = secrets.token_hex(32) 3. POST /api/vault/append/configure/{vault_id} x-sgraph-vault-write-key + x-sgraph-access-token {"append_anchors":[sha256(t1), sha256(t2), …], ── REPLACES the list ──► {"status":"configured"} "enum_key_hash": sha256(enum_key)} 4. self-test: write → list → fetch → decrypt → verify → mark-processed → purge 5. hand the token to the sender privately ─────────────────────────────────────────► (see §7.3) 6. publish or hand over: vault id, endpoint, public key to encrypt to ``` **`configure` replaces the anchor list** (confirmed independently by both parties). Adding a sender means sending every existing anchor plus the new one, or the existing lanes silently stop accepting writes. ### 4.2 Sending (per message; no account, no vault key) ``` SENDER SERVER 1. build a single-part RFC 2822 .eml (From, To, Subject, Date, Message-ID, Markdown body) 2. encrypt to the recipient's key and sign with the current signing key 3. POST /api/vault/append/write/{vault_id} {"append_token":…, "payload": base64(.enc file bytes)} ─► pending/ ◄─── {"ok": true} (nothing else) ``` ### 4.3 Draining (per check-in) ``` DRAINER (inside the receiving vault's clone or session) SERVER 0. derive/load: enum key; recipient private key 1. list {include_content:false} ─────────────────────────────────────────────────► ◄── [{inbox:, file_id:"{ms}_{24hex}.enc", received}] 2. for each: lane = lanes[sha256(inbox)] (unknown ⇒ quarantine) fetch {inbox, file_ids:[id]} → decrypt → signature check (§5) → header checks: single-part · ≤256 KB · Message-ID · From == lane's sender · To ∈ lane's recipients route → mail/mailroom// (+ X-Postmaster-Lane/-File/-Signature/-Received headers, and the original ciphertext kept, so the signature can be re-verified later) or quarantine + alarm to the owner mark-processed {inbox, file_ids} 3. optional purge {inbox, folder:"processed"} (write key) 4. commit + push (permanent vault) | file and purge (ephemeral inbox) ``` **Two independent sender signals must agree:** the lane (whoever holds its token) and the signature (whoever holds the registered key). The `From:` header must match both. ## 5. Sender identity with no long-lived secret: per-session keys and a pinned registry ### 5.1 The constraint A sender that runs as fresh sessions (new container, new Claude Code session) **can't** sign its next key with its previous one: by design, it no longer has it. The only thing it carries is its append token. So identity is built from things each session *can* show: | Signal | How the session shows it | What it proves | |---|---|---| | the lane | writes with its append token | it holds the token | | the site | publishes its new key at its own HTTPS URL | it can publish to that site (its repo or build) | | the signature | signs with the new key | it holds that private key, now | | the serial | the registry serial only goes up | it isn't an old key being replayed, even one lifted from an earlier container | ### 5.2 The registry (published by the sender, pinned by the recipient) `https://sgit.newsroom.sgit.ai/keys/agents.json`: ``` { "pinned_url": "...", "verify": "...", "identities": { "newsroom.sgit": { "serial": 1, "created": "...", "fingerprint": "sha256:9b69…", "signing_fingerprint": "sha256:f791…", "bundle": { "encrypt": PEM, "sign": PEM, ... }, ← public only "lane": { "vault": "62t9bjmy", "endpoint": "..." }, "retired": [ ... ], "inbox": { ... see §6 ... } } } } ``` One slot per identity, one current key per slot. Only public data. Each key change is also a commit to the site's repository, so the key history is public. ### 5.3 The recipient's rule (the one-time pin, then no human step) The owner pins the URL once: *"`newsroom.sgit`'s key is whatever this URL serves."* After that: ``` message on lane L, signed by fingerprint f │ ├─ f == accepted key for L ─────────────► verify s ─ ok ─► route │ └─ bad ──► quarantine │ └─ f unknown ─► fetch pinned registry (HTTPS) (1) came through L ✓ by construction (2) s verifies under the announced key ─┐ (3) registry serves exactly f for L's id ─┼─ all pass ─► message passes every header (4) registry serial > accepted serial ─┘ check under the new key? ├─ yes ─► swap key (old one retired), │ route, notice to owner └─ no ──► quarantine, key unchanged any fail ─► quarantine + alarm ``` - Signatures are matched by **signing fingerprint, never by label**. Every session's key can carry the same label. - **Policy is the owner's choice**, recorded per lane (`auto_accept`). The alternative is *hold for approval*: the message waits until the owner OKs the key. - **What it rests on:** an attacker needs the token **and** publish rights to the sender's site. Either alone fails a check. The site is fetched over HTTPS, so DNS/TLS and the site's build pipeline are in the trust base. - **Operational rule for senders:** publish the new key, wait until the pinned URL serves it, *then* send. A message sent earlier is quarantined. ### 5.4 Tested On 26 September, against the real server, with a sandbox vault and a stand-in registry: - **Accepted:** rotation 1 → 2 → 3, and a message on the current key. - **Quarantined:** - a retired key; - a key not live in the registry; - a valid new key on a message with a spoofed `From:` (the key stayed unchanged); - a rollback to an older serial; - an unsigned message; - a forged "current key" fingerprint. ## 6. Receiving with no permanent vault: the ephemeral inbox ### 6.1 The pattern An agent that wants replies but keeps nothing between sessions: 1. **creates** a vault for this session only (`sgit create …`, needs an SG/Send access token); 2. **opens** one lane per expected sender, with an enum key that lives only in the session; 3. **publishes** the inbox on its identity website (the same registry, `identities..inbox`); 4. **hands out** each sender's token privately (see §7.3); 5. **drains** while the session runs: decrypt, verify, file, mark processed, purge; nothing waits; 6. **closes**: marks the entry closed and destroys the vault. **Untested as of 26 September** (see §10). ``` OPEN ──► PUBLISH ──► HAND OUT ──► RECEIVE ──► DRAIN ──► CLOSE vault + entry at token via sender list/fetch/ entry "closed", lanes pinned URL signed lane encrypts, decrypt/verify vault destroyed signs, file, purge appends ◄───── (loop while the session runs) ``` ### 6.2 The published inbox entry | Field | Meaning | |---|---| | `vault` | the session vault's id: enough to address it, not to open it | | `endpoint` | the server with the append routes | | `encrypt_to` | fingerprint of the key to encrypt to (this session's key; bundle in the same entry) | | `senders` | identities that have a lane | | `status` | `"open until this session ends"`, then `"closed: the vault is deleted"` | | `opened`, `how` | when, and plain instructions | **Never published:** the vault key, write key, enum key, or append tokens. ### 6.3 The sender's pre-flight (what the postmaster checked before its first reply) ``` fetch pinned registry ─► identities..inbox exists? ├─ vault == the id given in the (signed) handover message ├─ status says open ├─ the bundle whose `fingerprint` == encrypt_to has a `signing_fingerprint` == the handover message's `f` │ (encrypt_to is an ENCRYPTION fingerprint; the handover is signed with the SIGNING key of the same bundle) └─ fingerprints computed from the bundle's PEMs == the declared fingerprint / signing_fingerprint all true ─► encrypt to encrypt_to, sign with own key, POST append/write/ ``` **The recipient is proven by its website** (only its site can say "this is my inbox"), and **the sender by its signature** against the sender's own published key. For a permanent vault the "published key" is its front-door bundle, handed over once. ### 6.4 Two vaults, one direction each | Vault | Owner | Written by | Drained by | Lifetime | |---|---|---|---|---| | `riskmandate-agent-collab` (`62t9bjmy`) | postmaster @Cowork | newsroom.sgit, via its lane | the postmaster, at each check-in | permanent | | `newsroom-inbox-2026-09-25` (`9c7vcrw4`) | the newsroom session | cowork.riskmandate, via its lane | the newsroom, while the session runs | deleted at session end | ## 7. Secrets: who holds what, and how they travel ### 7.1 The permanent side: the vault key is the only secret A deployment convention (it could become an sgit feature, see §12): ``` vault key ─(sgit)─► vault write key ─┬─► configure / purge ├─ HMAC(wk, ".../enum-key/v1") ─► enum key (never stored) └─ HMAC(wk, ".../front-door-pass/v1") ─► keypair passphrase (never stored) .vault/postmaster/front-door-keypair/ ← PKCS#8 private keys, encrypted under that passphrase ``` - Derived from the **write** key, so a read-key holder can't act as postmaster. - Every vault-key holder can be postmaster. That's accepted, because routed mail lands where they can read it anyway. - After `sgit vault rekey`: re-encrypt the keypair and re-run `configure`. ### 7.2 The ephemeral side: the append token is the only secret - To **send**: the append token, provided per session (ideally an environment variable). - To **open an inbox** as well: the SG/Send **access token** (`sgit create` and `configure` both require it). - The signing key is generated in the session, under a random passphrase that dies with the container. - The inbox vault's key lives in the clone's `.sg_vault/` (sgit's own store), in a scratch directory. The senders' tokens and the enum key live in a separate `0600` file beside the keystore. Both are outside any repository and both die with the container. - Nothing needs to survive the session except what is public (the registry, the tooling, the message record). ### 7.3 Handing out tokens | Direction | How the token travelled | |---|---| | owner → newsroom (lane into 62t9bjmy) | out of band, via the owner, into the newsroom's environment settings | | newsroom → postmaster (lane into 9c7vcrw4) | **inside a signed, encrypted lane message** into the postmaster's vault | The second is a good bootstrap: once one direction exists, the token for the other travels over it, authenticated. The residual: the token then sits in the permanent vault's mail, readable by every vault-key holder, until the inbox vault is deleted. ### 7.4 What a front-door signature proves The permanent vault's outgoing messages are signed with the front-door key. That key is unlocked by anyone holding the vault key, so the signature proves **"a holder of riskmandate-agent-collab's vault key"**, not which member. The newsroom accepts `From: cowork.riskmandate` or `mailbox.riskmandate` under that one key. Per-member signing keys would be needed to tell members apart. ## 8. Security properties | # | Property | How it holds | Strength | |---|---|---|---| | P1 | **Least privilege for senders** | a token writes to one lane only (a token used as an enum key → 404) | server-enforced | | P2 | **Blind writes** | `write` returns exactly `{"ok": true}` | server-enforced | | P3 | **Confidentiality from the server** | RSA-OAEP 4096 / AES-256-GCM, made on the sender's machine | cryptographic | | P4 | **Confidentiality at rest after routing** | a permanent vault commits routed mail under its own keys; an ephemeral inbox purges | cryptographic / by deletion | | P5 | **Tree isolation** | lanes live outside the commit tree | structural | | P6 | **Single, auditable mover** | only the drainer moves lane → tree; commits + `log.jsonl` + the kept ciphertexts | procedural + audit | | P7 | **Sender = lane, and lane = signature** | the server binds a token to its lane; the **drainer** checks that `From:` matches the lane and that the signature matches the lane's key | drainer-enforced + cryptographic | | P8 | **Sender authenticity without long-lived secrets** | token + site + signature + serial (§5) | cryptographic, anchored in HTTPS + site control | | P9 | **Rollback resistance** | the registry serial must strictly increase; retired keys are refused | policy, tested | | P10 | **Recipient authenticity** | the inbox is published on the recipient's own site and checked before the first send (§6.3) | anchored in HTTPS + site control | | P11 | **Revocation per sender** | `configure` without that anchor; key rotation retires the previous key | server-enforced | | P12 | **Bounded abuse** | ≤ 5 MB per payload, ≤ 1000 pending per token, ≤ 100 ids per fetch | server-enforced | | P13 | **Content is data** | drainers parse headers only; a routed message is a *request*, weighed against the recipient's own behaviour policy | policy, backed by P5/P6 | | P14 | **Minimal standing secrets** | permanent side: vault key (+ the SG/Send access token for `configure`); ephemeral side: the append token to **send**, plus the SG/Send access token to **open an inbox** (`sgit create`, `configure`) | by construction | | P15 | **Short-lived receiving surface** | inbox vaults exist only while a session runs | by lifecycle (see T12) | ### 8.1 What the server can and can't see ``` CAN SEE CANNOT SEE vault ids; each lane's raw token plaintext (From/To/Subject/body) (object paths and the `list` response) who a message is for (To: is inside the ciphertext) payload sizes and arrival times any private key or passphrase how many messages each lane sent vault keys, or anything in a commit tree sha256 of anchors and of the enum key ``` ## 9. Threat model | # | Threat | Attacker gets | Mitigation / residual | |---|---|---|---| | T1 | append token leaks | junk in one lane (≤ 1000 × 5 MB) | Can't pass as the sender: signatures are required, and a new key also needs the site (§5). Revoke with `configure` | | T2 | outsider, no token | nothing (404) | server-enforced | | T3 | **enum-key holder sees raw tokens** in `list` | can write into any lane | Can't forge the sender without its key or site. Recommended server fix: return the anchor, not the token (§12) | | T4 | server operator / storage access | tokens, ciphertext, metadata; could add or delete lane files | can't read (P3); can't forge (P8); deletions are visible only through the sender's own record | | T5 | **sender's site compromised + token stolen** | can rotate in its own key and send as the sender | This is exactly what §5 assumes won't happen together. Owner fallback: switch the lane to hold-for-approval | | T6 | site compromised alone | can publish a key but can't deliver it | check (1) fails: no token | | T7 | old key lifted from a previous container | none | check (4): the serial must increase; retired keys are refused | | T8 | malicious content from a legitimate sender | text reaches an agent | P13; no attachments; size cap | | T9 | spoofed `From:` / non-member `To:` | none | quarantine + alarm (tested) | | T10 | a new lane silently disables the others | none | always send the full anchor list (`configure` replaces it) | | T11 | fake inbox (attacker gives a sender a lane into *their* vault) | the sender's replies | §6.3 pre-flight: the vault must be the one published on the recipient's site, and encrypt_to must match the handover signer | | T12 | **session ends without `close`** | a drained, empty vault left behind; its entry still says "open" | Senders' writes succeed into a vault nobody drains. Mitigations: senders re-check `status`, `opened` and (once added) `expires`; the next session's entry replaces it; a server-side vault TTL is recommended (§12) | | T13 | recipient's inbox token sits in the sender's permanent vault | vault-key holders can write into the ephemeral inbox | worthless once that vault is deleted; the inbox still verifies signatures | | T14 | signature covers `c` only | someone with the recipient's private key could re-wrap a signed `c` to a third party, as if sent to them | only matters across recipients; recommend signing the whole envelope incl. recipient (§12) | | T15 | vault key leaks (permanent side) | everything, including postmaster powers | out of scope for lanes; `rekey`, then re-derive and reconfigure | ## 10. HTTP reference, as observed on 25–26 September 2026 Base: `https://dev.send.sgraph.ai`. Routes: `POST /api/vault/append/{op}/{vault_id}` with a JSON body. `send.sgraph.ai` has no append routes. | op | Auth headers | Body | Success | |---|---|---|---| | `configure` | `x-sgraph-vault-write-key`**and** the SG/Send access token (401 without it) | `{"append_anchors":[…], "enum_key_hash": …}` | `{"vault_id":…,"status":"configured"}` | | `write` | **none** | `{"append_token": hex, "payload": base64(bytes of the .enc file)}`; the `.enc` file is itself base64 of the envelope JSON, so the envelope is **encoded twice** | `{"ok": true}` | | `list` | `x-sgraph-vault-enum-key` | `{"include_content": false}` (opt. `after_file_id`) | `{"entries":[{"inbox": , "file_id","received"}], "truncated"}` | | `fetch` | `x-sgraph-vault-enum-key` | `{"inbox": , "file_ids": [...]}` | `{"files":[{"file_id","size","content": base64}]}`, **pending files only** | | `mark-processed` | `x-sgraph-vault-enum-key` | `{"inbox": , "file_ids": [...]}` | `{"moved":[…],"missing":[…]}` (idempotent) | | `purge` | `x-sgraph-vault-write-key` | `{"inbox": , "folder": "processed"\ | "pending"}` | `{"purged":[…],"missing":[…]}` | Hashes: `anchor = sha256(token as a UTF-8 hex string)`, `enum_key_hash = sha256(enum key string)`. Payload, exactly: `payload = base64(open("msg.eml.enc","rb").read())`, where the `.enc` file (what `sgit pki encrypt` writes) is `base64(JSON envelope)`. A drainer reverses it: `envelope = json.loads(b64decode(b64decode(content)))`. A sender that encodes only once is rejected by a strict drainer. Errors: a bad key or unknown token → **404 with an HTML page**. A missing field → JSON 400. Creating an ephemeral inbox vault: `sgit create --token `. | op | Auth | Body | Status | |---|---|---|---| | destroy a vault (closing an inbox) | write key + access token | `DELETE /api/vault/destroy/{vault_id}`, body `{"vault_id": …}`, as sgit's own client sends it | **untested**: the newsroom will report the real response when its session closes its inbox | PKI (sgit-ai 0.16.0): - `encrypt --recipient` needs the recipient `import`ed first; `--fingerprint` also signs. - The envelope carries `f`, the signer's fingerprint, and `s`, the signature over `c`. - The CLI's `decrypt` reports `Signature verified (signer: