Partial clones — a folder scope and a history depth (6 Oct 2026)

Branch: claude/charming-babbage-5hzcxq · Vault measured: y8pwtjlw (626 commits, 8,013+ trees, 9,404 blobs, 18,720 objects at HEAD time) Ask: agents that know which folders they read and write should be able to clone just those, read and write; plus a shallow (no-history) mode; and none of it may change the full clone.

1. The idea, in one paragraph

The vault is a Merkle tree: a commit names a root tree, a tree names its children by id. To hold mail/crm.riskmandate/ a clone needs the root tree, the mail/ tree (the spine) and the folder's own subtree; everything else is an entry it keeps by id ("opaque"). To commit a change inside the folder, rebuild the folder's subtree from disk, then each tree on the spine with one entry replaced, and keep the siblings' ids — exactly what the whole-vault builder would produce, byte for byte, because tree encryption is deterministic (a test proves the rebuilt root id equals the full builder's). Push uploads the new commit, the spine trees, the changed subtree and the new blobs; the server needs nothing new. Two scoped clones in different folders cannot conflict: out-of-scope entries are always the remote's, by id.

2. What was built

3. Numbers on the real vault (this session, through its egress gateway)

On the scoped clone: status in sync; history log shows HEAD and stops; check fsck refuses naming what the clone holds and how to widen.

4. "No side effects on the full clone" — how that was held

• Every scope branch is behind scope.is_scoped() / is_partial(); a whole-vault clone takes the exact calls it took before (flatten, build, build_from_flat, _fetch_missing_objects without the new kwarg, checkout). The existing pull-step tests use a fake sync client with no crypto and no new kwargs, and they pass unchanged — which is the proof the full path is not touched.
• Vault__Sub_Tree.build* with no opaque entries runs the old code; with them, the entry order is re-established to files-then-folders by name so ids match the whole-vault builder.
• Real vault, before and after: a full clone gives the same 18,720 objects, empty scope and boundary in its config, and fsck reports the same 41 missing objects.
• pytest tests/unit/ -n auto: 3,932 passed (21 new partial-clone tests on two real clones in the in-memory API: read, write, delete, refuse-outside-scope, write_file, fast-forward and three-way pulls with changes outside scope, two scoped agents never conflicting, dirty edit survives a pull, depth-1 round trips, unshallow, scoped+shallow, widen, guards, branch switch, and a full-clone-unchanged check; 8 storage tests on the scoped tree).

5. Caveats and follow-ups

• A folder renamed by another agent silently leaves a clone's scope (names are decrypted along the spine). Pull should notice a held path vanishing and say so. Not done.
• Scoped clones cannot see blobs missing elsewhere; the push fix from earlier today should ship with this (same branch).
• history log on a shallow clone ends with "history stops here: shallow clone … sgit fetch --unshallow" instead of an [object not found locally] line.
• A shared, immutable object cache across clones on one machine would make every agent's clone after the first a local copy — the browser-style cache idea; separate change.
• clone-branch (thin) still walks all commits serially; with --depth 1 on sgit clone there is little reason left to use it.

6. Deep review before the PR to main (same day, evening)

Two passes over origin/main...origin/dev after the merge: a code review (bugs, side effects) and a security review (new exploitable issues only, >80 % confidence). CI on dev was green first (run 37544894784 on ddee8a4).

Security review — no new vulnerability; three hardening gaps closed

No high-confidence exploitable issue was introduced. Three pre-existing "trust the host / trust the committer" gaps that the new fetch paths exercise more were closed anyway:

Code review — 15 findings, what changed

What the new tests then found (would have shipped otherwise)

• Typed boundary ids broke unshallow and the status boundary stop. Safe_Str hashes differently from the equal str ('x' in {Safe_Str('x')} is False) and os.path.join sanitises it (/ → _), so once the config fields became Safe types the boundary commit was "not in the local store" and --unshallow on a scoped shallow clone fetched nothing. Vault__Scope.boundary_ids() / folders() hand consumers plain strings; a test pins both behaviours. Worth remembering for every Safe_Str that reaches a path or a set.
• The first status rewrite assumed "a local commit has local ancestors", which a truncated fetch (limit hit) violates; the walk now stops only at known-complete heads.
• Branch-only push's ref CAS bug above (pre-existing, invisible to the in-memory API).

pytest tests/unit/ -n auto: 3,953 passed (21 new review tests under tests/unit/review/).

Follow-up (7 Oct): the merge of main into dev and a CI flake that was a real leak

Merging main into dev for PR #7 conflicted only on the release bot's version bumps (dev v0.17.3, main v0.17.0; dev kept). CI on that merge then failed on test_Dev__Tree__Graph__Happy::test_analyse_temp_dir_cleaned_up, which diffed /tmp before and after analyse() and so could see any vault another xdist worker was creating at that moment. Rewriting it to watch the mkdtemp calls made inside the call exposed something real: every sgit clone (all five entry points) creates /tmp/sgit-clone-* as the workflow workspace's root, the runner removes the workspace inside it on success, and the root stays — one empty dir per clone, 3,881 of them in this sandbox. _run_clone_workflow now removes the root in its finally, success or failure; a review test clones three ways and checks none of the roots survive.

This page as markdown · its source, admin/content/articles/partial-clones-scope-and-depth.md