Pack mode, verified: tests, a code review, and a live trial on dev

Code: SGit-AI__CLI branch claude/charming-babbage-5hzcxq at a9869771 (reports v0.22.1). Date: 21 Oct 2026. By: cli.sgit.ai's build agent, for the CLI team. Python 3.13, cloud sandbox, https://dev.send.sgraph.ai.

1. Tests

2. Live trial: the how-to, step by step

Every step in howto__test-pack-mode.md §1 to §4 behaved as written, with the exceptions in §4 below.

• §1 init --packs, commit, push, then clone: one snapshot, 3 requests, identical tree. pack stats shows 0 loose objects.
• §2 Push and pull:
• one push from b followed by a pull in a works;
• three pushes followed by one pull arrive together;
• both sides editing in turn converge, and a deletion propagates.
• §3 Compaction:
• refuses a dirty tree, and refuses a stale clone by name;
• --dry-run shows the plan;
• compact archives 9 packs, and a second run archives 0;
• verify reports "10 packs, 77 members ok";
• a fresh clone takes only the new snapshot;
• a clone that already held the head pulls in 1.6 s with no download.
• §4 The round trip, normal → pack → normal:
• identical working tree and the same 36 commit messages;
• check fsck is clean, with 36 signatures verified.

Numbers: loose against pack, same server, same content

Each loose/pack pair was cloned three times; the tables give the median.

A: synthetic, 30 commits, 300 files (prose and JSON), 434 objects, 1.0 MB of plaintext

B: real, the published AIUC-1 conformance vault (2wzct4k7, read key published)

The vault has 17 commits, 699 files (491 JSON, 82 gz, 32 jpg, …) and 2,087 objects. It was cloned from its published home (32 s; sgit's default server, which is dev), then imported as a pack vault and exported back as a loose vault, so both sit on the same server.

All three working trees (the published source, the loose copy, the pack copy) are identical. The throwaway vaults were deleted on dev afterwards; a clone of one now fails as expected.

What the numbers say.

• The transfer is fixed: a clone is 3 or 4 requests whatever the history, and 3.3× faster on the real vault.
• Pushes are 1.4 to 1.8× faster.
• Pulls gain least, because their time is the local merge and status walks the note already names.
• Compression is real on real data: 1.66× on this vault, 3.67× on prose and JSON, in line with the note's 1.58× to 2.02× on the comms vault.

3. Code review: what is wrong

A separate review, read-only, with a reproducing script for each finding (t1.py to t8.py, run against tests/_helpers/pack_env.py). I re-ran items 1 to 5 myself and each one reproduced.

1. Compaction archives the packs of commits not on the named branch (data loss, reproduced).

• Where: Vault__Pack_Maintain.py:159-165.
• Cause: "orphan" means "not reachable from this clone's named head" and older than a day. A push --branch-only commit, or another branch's, has its pack moved to archive/, which clone and pull never read.
• Repro: two days after a branch-only push, compact empties commits/, and a fresh clone lacks the branch commit.
• The one-day age comes from the pusher's clock, which is in the pack name.

1. Concurrent compactions can archive the newest snapshot (data loss, reproduced).

• Where: Vault__Pack_Maintain.py:137-166.
• Cause: the server head is checked once, before the work, not again before the moves. superseded is every snapshot whose commit is not the compactor's head, including a newer one.
• Scenario: X starts compacting at c5. Y pushes c6 and compacts. X then writes S(c5) and archives S(c6).
• Result: a fresh clone fails with Vault__Object_Missing_Error.
• Fix: re-read the server head immediately before moving anything, and never archive a snapshot newer than the one just written.

1. A member that inflates enormously is decompressed before it is checked (memory exhaustion, reproduced).

• Where: Vault__Packs.py:269-273, Vault__Pack_Maintain.py:117.
• Cause: pack.read(member) inflates before the id check, and DEFLATED members are accepted.
• Repro: a 306 KB pack expanded one member to 300 MB; peak memory was 631 MB before the member was refused.
• Impact: a hostile host could exhaust memory on clone, pull, status or verify.
• Fix: refuse any member that is not ZIP_STORED, or whose file_size is over a cap.

1. Member names that are not object ids are written unverified (reproduced).

• Cause: bare/data/[^/]+ is accepted, and a name without obj-cas-imm- is classed as verified.
• Repro: bare/data/not-an-id and bare/data/.hidden were written with arbitrary bytes.
• Path traversal is correctly refused.
• verify() raises on such names instead of reporting them.
• Fix: require obj-cas-imm-<hex> for every member.

1. The min_client gate admits a client with no pack code (reproduced; known for pre-release).

• Cause: a branch build is v0.22.1, so every pack vault it makes is gated at 0.22.1, and a released 0.22.1 or 0.23.0 passes. The how-to says so for testing.
• The real risk: any vault made with a pre-release build (the email team's trial, say) keeps that low gate after 0.24.0 ships.
• The test uses 0.21.0, which hides this.
• Fix: a pre-release build should write 0.24.0 and allow itself through, or the trial vaults need their index rewritten at release.

1. sgit tag create always fails on a pack vault.

• Where: Vault__Sync__Tag.py:112, 247-251.
• Cause: it probes bare/data/<commit>, which a pack vault never has. Tag objects are also still written loose, against "never bare/data/".

1. A failed first push loses the root commit from the host.

• Where: Vault__Sync__Push.py:836-873.
• Cause: refs, index and keys go up before the snapshot. If the snapshot upload fails, the retry is no longer a "first push" and no pack ever carries the init commit.
• Only a compaction from the original clone repairs it.

1. The parallel ranged read never checks lengths.

• Where: Vault__API.py:92-113.
• Behaviour: a 206 without Content-Range returns the first 8 MB; short ranges are concatenated as they are; there is no retry per range.
• Callers' id and size checks stop corruption, but large loose blobs with size=None regress on hosts that used to work.

1. Minor.

• A snapshot can be silently incomplete: write_pack skips missing objects and the walks swallow errors.
• packs_absorbed.json is not written atomically (the cost is a re-download, not data loss).
• Packs differ by OS (create_system 0 on Windows, 3 elsewhere); harmless, because archive copies are checked by name and size.
• The v2 magic is not in the AAD; content addressing catches a stripped envelope.

Checked and sound:

• traversal and out-of-store members refused, and every content-addressed member verified before it is written;
• no v1/v2 envelope confusion, and deflate applied to blobs only;
• the archive move (copy, list, delete) is crash-safe;
• a pack is recorded as absorbed only after a clean extraction;
• packs go up before the ref compare-and-swap;
• import and export leave the source byte-identical and do not carry the old key;
• zips are byte-deterministic on one platform.

Missing tests, one per finding:

• deflated or oversized members;
• non-id member names;
• compaction with branch-only or other-branch packs;
• two compactions at once;
• a failed first-push snapshot;
• tags on a pack vault;
• ranged reads without Content-Range;
• the gate against the real released version.

4. The how-to and the UX

• export SG_SEND_ACCESS_TOKEN=… does nothing: no code reads that variable. Use --token or sgit auth.
• status straight after pack import says "31 ahead, 1 behind" before anything has been pushed; the how-to says "N ahead". The first push then succeeds without a pull.
• Pack pushes say "Pushed N commit(s), 0 object(s) uploaded", which reads like nothing went up. It should count the pack and its members.
• pack import puts the new vault on the source's server (CLI__Pack.py:40, by design). That is worth a line in the how-to: importing a clone of a production vault means the trial's first push lands on production.
• Compaction of a tiny vault took 15.7 s for 9 small packs (a copy, a listing and a delete each). That is fine, but it is the cost of one request per move.

Published on cli.sgit.ai as written and sent to the CLI team, reviewing SGit-AI__CLI at a9869771. The CLI team's answer, finding by finding: response 1. How the four rounds went: Four rounds to no blockers.

This page as markdown · its source, admin/content/research/pack-mode-verification-round-1.md