Skip to content

feat(pm): make mcpp.lock authoritative for git deps (2026.8.1.2) - #330

Merged
Sunrisepeak merged 4 commits into
mcpp-community:mainfrom
ZheFeng7110:feat/git-branch-lock-offline-anchor
Aug 1, 2026
Merged

feat(pm): make mcpp.lock authoritative for git deps (2026.8.1.2)#330
Sunrisepeak merged 4 commits into
mcpp-community:mainfrom
ZheFeng7110:feat/git-branch-lock-offline-anchor

Conversation

@ZheFeng7110

@ZheFeng7110 ZheFeng7110 commented Aug 1, 2026

Copy link
Copy Markdown
Member

Summary

Makes mcpp.lock a resolution input for git dependencies — and an authoritative
one. Before this change the lock was write-only: a branch-based git dependency
(branch = "develop") re-ran git ls-remote every time prepare_build ran, and
the commit written into mcpp.lock was never read back by anything.

After this change the lock is read first. A branch that already has a commit
recorded resolves from it with no network round-trip, and that commit is what
gets built whether or not the local clone survived
— evicting ~/.mcpp/git, or
cloning the project on another machine, can no longer put the build on a newer
branch tip. mcpp update <dep> drops the entry and remains the one way a branch
advances.

Closes #329.

Why authoritative, and not "an anchor the cache confirms"

A lock that only counts while the clone happens to exist hands the "which commit
do we build" decision to the survival of a cache directory. mcpp new does not
gitignore mcpp.lock — it is meant to be committed — so the failure is ordinary:
a teammate clones the project, has no ~/.mcpp/git, and silently builds a
different commit while rewriting the lock.

This is also the dominant convention: cargo build on a git branch dependency
builds the locked revision and cargo update re-resolves it — the same split
mcpp already follows for profile defaults.

Measured, with the branch moved to v2 and the cache removed (this is the new
assertion in tests/e2e/24_git_dependency.sh):

build output
lock as cache hint branch dep v2 — silently advanced, lock rewritten
lock authoritative branch dep v1 — the commit the lock recorded

Shape

The block collapses into two independent questions, each with at most one network
operation and therefore exactly one --offline gate:

  1. Which commit? tag/rev name one outright. A branch is floating:
    mcpp.lock answers it, else git ls-remote.
  2. Is it on disk? The commit selects the cache directory; a miss is a clone.

The separate tag/rev leg is gone with it — along with its per-build from cache
noise line and its "is locked but its local cache is missing" message, which
was emitted for dependencies that were never in the lock.

Also fixed here

  • --offline no longer refuses a local git remote. docs/05-mcpp-toml.md
    defines --offline as "never touch the network … anything already installed
    still builds", and the dependency-download gate in prepare.cppm draws the
    same line in a comment. A git = naming a local directory (or a file://
    URL) is served by filesystem reads, so refusing it buys no isolation. Remotes
    are classified by shape, keeping a Windows drive letter (C:\repo — colon,
    no @) on the local side.
  • A clone killed between git clone and git checkout no longer serves the
    wrong commit forever.
    The cache directory is named after the commit but was
    only checked for existence. Branch deps now compare git rev-parse HEAD and
    re-clone on mismatch (tag/rev keep their ref name as the identity, so there is
    nothing to compare).
  • Windows: the clone no longer depends on cd changing drive. git clone … && cd <dir> && git checkout does not change drive in cmd.exe without /d,
    and the cache root routinely lives on a different one than the project. Every
    other cross-drive site in the repo (process.cppm, msvc.cppm) writes
    cd /d; this one did not. Now git -C <dir>, which needs no shell at all.
  • An unreadable mcpp.lock is reported as degraded, not as a plain warning.
    Per src/diag.cppm that channel is for "the engine did less than asked" and
    requires an impact — here every git branch dep silently falls back to the
    network and may move past the recorded commit. As a warning it was invisible
    to --strict.
  • parse_git_source no longer splits a tag/rev whose name contains @.
    Only branch entries carry @<commit> (that is what the writer emits), so a
    tag legitimately named v1.0@rc1 was being read as ref v1.0 + commit rc1.
    Branch entries still split on the last @, so feat@v2 round-trips.

Note on the original framing

The first revision described this as removing an ls-remote from "every
mcpp build". That overstates it: a no-op rebuild takes the fast path in
cmd_build.cppm, which returns before prepare_build runs, so no ls-remote
happened there either. The cost this removes is on builds that do re-prepare —
sources changed, mcpp.toml touched, a --profile/--target/--features flag
passed, or after mcpp clean. That is also why the e2e has to mcpp clean
first: without it the assertion passes on main too.

Changes

  • src/pm/lock_io.cppm: LockedGitSource + parse_git_source().
  • src/build/prepare.cppm: load mcpp.lock up front; two-phase git resolution;
    local-remote classification; cache integrity check; git -C.
  • src/pm/commands.cppm: correct the cmd_update comment — dropping a lock
    entry is now the thing that lets a branch advance, not a no-op.
  • docs/05-mcpp-toml.md, docs/zh/05-mcpp-toml.md: git deps + mcpp.lock, and
    the --offline row.
  • CHANGELOG.md, mcpp.toml, src/toolchain/fingerprint.cppm: 2026.8.1.2.
  • tests/e2e/24_git_dependency.sh: offline rebuild from the lock; the branch
    does not advance when the cache is evicted.
  • tests/unit/test_pm_lock_io.cpp: @ in branch names, @ in tag/rev names,
    empty commit, scp-like URLs.

Test plan

  • mcpp build
  • mcpp test (46 passed, 0 failed)
  • tests/e2e/24_git_dependency.sh
  • A/B: the new e2e fails on the previous head (builds v2) and passes here
  • .github/tools/check_version_pins.sh

- Add LockedGitSource + parse_git_source to lock_io.cppm.
- Load mcpp.lock in prepare_build and use recorded branch commit to skip
  git ls-remote when the local cache still matches.
- For tag/rev, reuse cached clone and fail early in --offline when missing.
- Update e2e test 24 to assert branch deps do not re-ls-remote on rebuild.
- Add unit tests for git source parsing.

Closes mcpp-community#329
@speak-agent

Copy link
Copy Markdown
Member

Code review

The main path is correct: computeGitRoot() matches the real cache-key formula field for field, so a lock hit provably lands on the existing clone and skips it; readCacheHead compares against a detached HEAD left by git clone --depth 1 --branch X && git checkout <commit>, so both sides are full 40-char shas; and the skip path still repopulates root_git_lock_identities, so the lock stays idempotent. mcpp update <dep> erases by p.name, which is the same key the anchor lookup uses, so the refresh path in the e2e still holds.

Found 4 issues:

  1. The new e2e assertion is vacuous — it passes identically on main without this change. ls-remote appears in build output only inside the failure message "git ls-remote of '{}' failed"; on the success path the command is captured with 2>&1 into r.output and never printed, so the grep can never match whether or not the network call happened. And the only case that would match is a failed build, which set -e (line 5) already aborts on at the build2=$(...) assignment. The Cloning guard doesn't discriminate either: on main the second build resolves to the same commit, hence the same gitRoot hash, hence no re-clone. Net effect: the headline behaviour of this PR has no test coverage, and the e2e box in the description is unchecked. The positive signal already exists — this PR adds ui::info("Resolved", "... from lock"), so grep -q 'from lock' would actually assert it.

# Second build with the lock in place must not hit the network for ls-remote.
build2=$("$MCPP" build 2>&1)
echo "$build2" | grep -q 'ls-remote' && { echo "FAIL: branch dep re-ls-remoted on rebuild"; exit 1; } || true
echo "$build2" | grep -q 'Cloning' && { echo "FAIL: branch dep re-cloned on rebuild"; exit 1; } || true

  1. The cache-key formula is now derived in two places. computeGitRoot and the block that actually creates the directory are byte-identical today, so this is not a live bug — but any future axis added to one side (profile, triple) makes the anchor miss forever, silently degrading back to per-build ls-remote with no error and, given issue 1, no failing test. The second site should just call computeGitRoot(resolvedGitRev).

mcpp/src/build/prepare.cppm

Lines 3027 to 3035 in c3e7095

auto computeGitRoot = [&](const std::string& rev) {
std::hash<std::string> H;
auto urlHash = std::format("{:016x}",
H(spec.git + "|" + spec.gitRefKind + "|" + spec.gitRev
+ "|" + rev));
return mcppHome / "git" / urlHash;
};

mcpp/src/build/prepare.cppm

Lines 3109 to 3117 in c3e7095

// Cache key: hash(url + refkind + declared ref + resolved commit).
// For fixed rev/tag deps the declared ref is also the resolved ref.
std::hash<std::string> H;
auto urlHash = std::format("{:016x}",
H(spec.git + "|" + spec.gitRefKind + "|" + spec.gitRev
+ "|" + resolvedGitRev));
auto gitRoot = mcppHome / "git" / urlHash;
std::error_code ec;

  1. LockedGitSource::url is parsed but never compared — the anchor matches on refKind + ref only. Online this self-heals (a changed URL changes the gitRoot hash, the cache misses, and it falls through to ls-remote), but under --offline a repo URL change is reported as locked to commit <old-repo-commit> but its local cache is missing or stale, pointing the user at a stale cache when the actual cause is a different repository. Adding anchor.url == spec.git to the match would fix both the diagnostic and the fallback.

mcpp/src/build/prepare.cppm

Lines 3018 to 3027 in c3e7095

// Look up an offline anchor for this git dep.
std::optional<std::string> lockedCommit;
if (auto it = gitLockAnchors.find(name); it != gitLockAnchors.end()) {
auto const& anchor = it->second;
if (anchor.refKind == spec.gitRefKind && anchor.ref == spec.gitRev) {
lockedCommit = anchor.resolvedCommit;
}
}

  1. Both offline diagnostics recommend mcpp update <dep>, which cannot help offline. For git deps that command only drops the lock entry (src/pm/commands.cppm:449-458) — it never resolves or clones. In the "lock has no commit, cannot resolve offline" case, following that advice deletes the only remaining anchor. run without --offline is the only real remedy and should be the sole suggestion here.

mcpp/src/build/prepare.cppm

Lines 3061 to 3074 in c3e7095

if (!skipLsRemote) {
if (mcpp::platform::env::offline_mode()) {
if (lockedCommit) {
return std::unexpected(std::format(
"git dep '{}' locked to commit {} but its local cache is missing or stale; "
"run without --offline to refresh, or `mcpp update {}` to re-resolve",
name, *lockedCommit, name));
} else {
return std::unexpected(std::format(
"git dep '{}' uses branch '{}' and mcpp.lock has no commit; "
"cannot resolve offline. Run `mcpp update {}` or build without --offline.",
name, spec.gitRev, name));
}
}

Two smaller notes, same PR:

  • The comment in cmd_update explaining why that command used to be a no-op rests on "the build path never reads mcpp.lock (prepare writes it and nothing on that path loads it)". This PR makes that false, and it is load-bearing — the next person changing mcpp update will reason from it. Worth updating in the same change.

    mcpp/src/pm/commands.cppm

    Lines 413 to 419 in 7f1489d

    // Refresh the index FIRST (#315/D6).
    //
    // This command used to only drop lock entries and tell the user to run
    // `mcpp build` — but the build path never reads mcpp.lock (prepare writes
    // it and nothing on that path loads it), so the whole command was a no-op:
    // it changed no behaviour whatsoever. It is also the only command whose
    // stated purpose is "get me newer dependencies", which since #315 is

  • parse_git_source splits on the first @, so a branch name containing @ (feat@v2) parses wrong. The consequence is only a safe degradation (anchor misses, falls back to ls-remote), but rfind matches the write format better. Relatedly, branch=develop@ yields optional(""), which passes the if (lockedCommit) test in the offline branch and prints locked to commit with an empty sha — the wrong one of the two messages.

    mcpp/src/pm/lock_io.cppm

    Lines 198 to 206 in c3e7095

    auto refPart = fragment.substr(eqPos + 1);
    auto atPos = refPart.find('@');
    if (atPos == std::string_view::npos) {
    out.ref = std::string(refPart);
    } else {
    out.ref = std::string(refPart.substr(0, atPos));
    out.resolvedCommit = std::string(refPart.substr(atPos + 1));
    }

Behaviour note, not a defect: ui::info("Resolved", ...) now prints on every build for both branch and tag/rev deps, including no-op incremental builds that used to be silent. The tag/rev one is purely informational — it gates on exists() and changes no decision — so it may belong behind --verbose.

@Sunrisepeak
Sunrisepeak self-requested a review August 1, 2026 13:12
ZheFeng7110 and others added 3 commits August 1, 2026 22:18
- tests: assert 'from lock' on rebuild instead of vacuous 'ls-remote'
- prepare: dedupe cache-key formula via computeGitRoot(resolvedGitRev)
- prepare: add anchor.url == spec.git to offline anchor match
- prepare: drop 'mcpp update' from offline diagnostics (offline no-op)
- commands: refresh stale comment in cmd_update about lockfile semantics
- lock_io: split fragment on last '@' (branch names with '@') and
  reset empty resolvedCommit so the offline branch prints the right hint
…heck

A bare second `mcpp build` takes the try_fast_build path when
build.ninja is fresh (no source change), so prepare_build is skipped
and neither the lock-anchor code nor `git ls-remote` runs — leaving
the `from lock` assertion vacuous (it cannot match; same failure
mode as the original ls-remote assertion the reviewer flagged).

Run `mcpp clean` first so the fast-path cannot fire; mcpp.lock and
the git cache (in MCPP_HOME/git) both survive `mcpp clean`, so the
next build re-prepares, hits the anchor, prints 'from lock' and skips
both ls-remote and the clone.
The lock was read for the first time in the previous commits of this PR, but
only as a hint the local clone had to confirm: a recorded commit counted while
`~/.mcpp/git/<hash>` existed, and otherwise the branch was re-resolved over the
network and the lock rewritten. That hands "which commit do we build" to the
survival of a cache directory. `mcpp new` does not gitignore `mcpp.lock` — it is
meant to be committed — so the failure is ordinary: clone the project on a second
machine, get a different commit, and see the lock quietly change under you.
Measured on the new e2e assertion, with the branch moved and the cache evicted:
the previous head built v2, this builds the v1 the lock recorded. `cargo build` /
`cargo update` split it the same way.

Making the lock authoritative also collapses the block into two independent
questions, each with at most one network operation and therefore exactly one
--offline gate: which commit (tag/rev name one; a branch is answered by the lock,
else by ls-remote), and is it on disk (the commit picks the cache directory; a
miss is a clone). The separate tag/rev leg goes with it, taking its per-build
"from cache" noise line and its "is locked but its local cache is missing"
message — which was emitted for deps that were never in the lock.

Four defects fixed along the way:

- --offline refused git remotes that are local directories. docs/05-mcpp-toml.md
  defines --offline as "never touch the network ... anything already installed
  still builds", and prepare.cppm's dependency-download gate draws the same line
  in a comment; ls-remote/clone against a local path are filesystem reads, so
  refusing them buys no isolation. Remotes are now classified by shape, which
  keeps a Windows drive letter (C:\repo — colon, no @) on the local side.

- A clone killed between `git clone` and `git checkout` served the wrong commit
  forever: the directory is named after the commit but was only checked for
  existence. Branch deps now compare `git rev-parse HEAD` and re-clone on
  mismatch (tag/rev keep the ref name as identity, so there is nothing to
  compare).

- The clone depended on `cd` changing drive on Windows. cmd.exe needs `cd /d`,
  and MCPP_HOME routinely sits on a different drive than the project; every
  other cross-drive site in the repo (process.cppm, msvc.cppm) writes /d, this
  one did not. `git -C <dir>` needs no shell at all.

- An unreadable mcpp.lock was a plain warning, invisible to --strict. Per
  src/diag.cppm that is the degraded channel: the engine silently does less than
  asked — every git branch dep falls back to the network.

Also: parse_git_source no longer splits a tag/rev whose name contains '@' (only
branch entries carry @<commit>, which is what the writer emits), the cmd_update
comment now records the right causality, and docs/CHANGELOG cover the new
semantics in both languages.

Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
@Sunrisepeak Sunrisepeak changed the title feat: use mcpp.lock commit as offline anchor for git deps feat(pm): make mcpp.lock authoritative for git deps (2026.8.1.2) Aug 1, 2026
@Sunrisepeak
Sunrisepeak merged commit 3338209 into mcpp-community:main Aug 1, 2026
15 checks passed
@ZheFeng7110
ZheFeng7110 deleted the feat/git-branch-lock-offline-anchor branch August 1, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: use mcpp.lock commit as offline anchor for git branch dependencies

3 participants