From dd208c9302737f6d61b4f327de87b197479f8d0d Mon Sep 17 00:00:00 2001 From: Zhibo Lin <147509942+LE0-Lin@users.noreply.github.com> Date: Wed, 4 Nov 2026 21:35:45 +0800 Subject: [PATCH 1/3] feat: compare Cursor attachment changes against a baseline --- CHANGELOG.md | 2 + docs/README.md | 1 + docs/cursor-scope-impact.md | 133 +++++++++ docs/cursor-scope.md | 11 +- docs/limitations.md | 11 +- docs/simplified/limitations.md | 4 +- docs/simplified/user-guide.md | 9 + docs/traditional/limitations.md | 4 +- docs/traditional/user-guide.md | 9 + docs/translations.md | 9 + docs/user-guide.md | 15 + examples/README.md | 3 + examples/cursor-scope-impact/README.md | 56 ++++ .../base/.cursor/rules/policy.mdc | 5 + .../narrowed/.cursor/rules/policy.mdc | 5 + .../renamed/.cursor/rules/approval.mdc | 5 + .../uncertain/.cursor/rules/policy.mdc | 4 + scripts/verify_installed_package.py | 65 ++++ src/agent_config_score/cli.py | 80 ++++- src/agent_config_score/entrypoint.py | 2 + src/agent_config_score/scanner.py | 105 ++++++- tests/test_cursor_scope_impact.py | 281 ++++++++++++++++++ 22 files changed, 803 insertions(+), 16 deletions(-) create mode 100644 docs/cursor-scope-impact.md create mode 100644 examples/cursor-scope-impact/README.md create mode 100644 examples/cursor-scope-impact/base/.cursor/rules/policy.mdc create mode 100644 examples/cursor-scope-impact/narrowed/.cursor/rules/policy.mdc create mode 100644 examples/cursor-scope-impact/renamed/.cursor/rules/approval.mdc create mode 100644 examples/cursor-scope-impact/uncertain/.cursor/rules/policy.mdc create mode 100644 tests/test_cursor_scope_impact.py diff --git a/CHANGELOG.md b/CHANGELOG.md index f855dd0..fb868fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ ### Added +- Unreleased `scope compare` and Git-backed `scope diff` for the same declared Cursor file context: file lifecycle, trigger matches and aggregated exact-directive availability, with original line evidence and captured source hashes. Optional caller-owned flags gate recognized active losses and unknowns separately; active duplicate/renamed sources and remaining multi-file triggers retain availability. Includes four runnable synthetic examples and source-build CI guidance. This is bounded attachment/recognition analysis, not general semantic impact or runtime enforcement; normal scoring, policy schema, published packages and Action refs are unchanged. + - An opt-in, unscored `scope` command predicting root Cursor project-rule activation for explicitly declared file context. A bounded scalar-metadata parser and iterative glob model retain unknowns, source-line evidence and multi-file opposing-directive signals even for disjoint patterns. Includes a runnable synthetic example, installed-wheel acceptance and aligned core guides. It does not observe Cursor, prove semantic conflicts, alter scan/gate scores or ship in the published v0.24.0 artifacts or rolling Action. - An offline runnable design lab with five synthetic before/after scenarios and two passing controls, comparing baseline new-error gating, score-delta-only gating and explicit candidate-threshold caller overrides through the real CLI. Retains observed decisions, commands and source/scenario hashes without claiming independent accuracy, AI superiority or user adoption. Runtime rules, scores, package artifacts and Action refs are unchanged. diff --git a/docs/README.md b/docs/README.md index 39a1955..968a6e6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,6 +14,7 @@ read the result, and troubleshoot a failing gate. - [Instruction templates](../examples/README.md) - [Opt-in deletion safeguard review and completion](guardrail-suggestions.md) (v0.24.0+) - [Cursor rule activation for declared file context](cursor-scope.md) (unreleased, unscored) +- [Cursor attachment changes and opt-in review gate](cursor-scope-impact.md) (unreleased) - [Repository doctor](doctor.md) - [Optional PR comments](pr-comments.md) - [Score-history workflow](score-history.md) and [local history command](score-history-cli.md) diff --git a/docs/cursor-scope-impact.md b/docs/cursor-scope-impact.md new file mode 100644 index 0000000..91b4704 --- /dev/null +++ b/docs/cursor-scope-impact.md @@ -0,0 +1,133 @@ +# Comparing Cursor attachment changes + +**Unreleased.** Source builds after v0.24.0 extend the [Cursor file-context +profile](cursor-scope.md) with two entrypoints: + +```bash +agent-config-score scope compare ../repo-base . --target src/api/service.py --json +agent-config-score scope diff origin/main --target src/api/service.py --json +``` + +`compare` reads two supplied repository trees. `diff` compares a local Git ref +with the repository's working tree, including uncommitted files. It retains the +resolved baseline commit and removes its temporary detached worktree on success +or failure. Omit the ref for conservative local default-branch detection; use +`--path REPOSITORY` outside the target repository. No ref is fetched. + +Both sides use the **same** normalized `--target` set and activation profile. +Each selected source is captured once per side for both metadata and directive +evidence. Both trees must complete source preflight/reads. A changed tool-source +identity during comparison aborts instead of mixing implementations. + +## Three distinct kinds of evidence + +1. **Rule lifecycle:** added, removed or modified files, with before/after modes, + activation reasons, metadata lines, original body lines and source hashes. +2. **Trigger changes:** which supplied paths stop/start matching. A rule may lose + an API-file trigger but remain attached because another declared file matches. +3. **Recognized directive availability:** aggregate sources across the rules in + each context. Renaming a rule or retaining the same directive in another + active rule need not lose its recognized active availability. + +Source hashes cover UTF-8 decoded, newline-normalized text of read `.mdc` +sources. Plain `.md` and unmodelled extension casing are not read and have null +hashes. Changed bodies/metadata can be visible even when their effect on the +supplied targets is unchanged. This is not an arbitrary-glob equivalence or +global narrowing proof; changes affecting undeclared files are outside the run. + +An exact directive is identified by its existing normalized English body hash +and polarity. Original evidence keeps source paths and one-based line numbers. +Raw directive and metadata values are not exported. An identical hash is not +proof of semantic equivalence, safe behavior or authentic evidence. + +| Availability | Meaning for this recognized directive and declared context | +|---|---| +| `active` | At least one recognized source is predicted automatically attached | +| `unknown` | No active source is established, and a matching source has unknown activation or a non-inactive rule body is uninterpreted | +| `inactive` | Recognized sources exist, all are predicted inactive, and no potentially active opaque body obscures the result | +| `not_recognized` | No source of this exact directive was recognized and no potentially active opaque body obscures the result | + +Uninterpreted, potentially active bodies conservatively leave missing identities +unknown: the checker cannot establish whether they contain replacements. A +manual/model-selected rule with an interpretable body contributes uncertainty +to identities recognized in that body. Unsupported English paraphrases do not +establish equivalence. Rephrasing a directive can therefore require review even +if a human considers its meaning preserved. + +`rule_changes` records file/activation/trigger changes. `directive_changes` +records changed availability or source evidence, including harmless line shifts +and moves whose availability stays `active_to_active`. Summary counts keep +baseline-active identities, retained identities, recognized losses/gains, unknown +rules and uninterpreted non-inactive bodies separate. Gains include newly +established active availability from unknown; they do not prove improvement. +Both full side reports and their tool identities are retained. + +## Optional review gate + +```bash +agent-config-score scope diff origin/main --target src/api/service.py --fail-on-lost-directives --fail-on-unknown --json +``` + +- Without flags, completed comparisons return **0**, including changes/unknowns. +- `--fail-on-lost-directives` returns **1** when a baseline-active identity becomes + `inactive` or `not_recognized`. It does not equate unknown with definite loss. +- `--fail-on-unknown` returns **1** when **either** side has unknown rule activation + or an uninterpreted non-inactive body. This also covers an uninspectable + baseline, not just a newly uncertain candidate. +- Input, source, encoding, Git, pair-budget or identity-consistency failures use + **2**, with no successful/partial JSON report. + +Use both flags when uncertainty requires review. `review_gate` records the +chosen flags, failure reasons and decision. The flags belong to the caller; +existing repository score-policy JSON and suppressions do not configure this +profile. Review workflow/tool changes normally. This does not extend the +baseline-governed policy schema or alter normal scan/diff/Action defaults. + +An empty recognized baseline protects zero directive identities. Exit 0 does +not imply that required safeguards exist. Neither flag blocks new opposing +directives automatically; side reports retain those separate review signals. +Full semantic conflicts, harmful unrecognized text and runtime enforcement +remain outside this gate. + +## Source-build GitHub Actions example + +```yaml +name: Cursor attachment review +on: [pull_request] +permissions: + contents: read +env: + ACS_TOOL_REVISION: SCOPE_IMPACT_TOOL_COMMIT +jobs: + scope: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 + persist-credentials: false + - uses: actions/setup-python@v7 + with: + python-version: "3.12" + - name: Install the reviewed tool source + run: python -m pip install "git+https://github.com/LE0-Lin/AgentConfigScore.git@$ACS_TOOL_REVISION" + - name: Review declared API-task context + env: + BASE_COMMIT: ${{ github.event.pull_request.base.sha }} + run: >- + agent-config-score scope diff "$BASE_COMMIT" + --target src/api/service.py + --fail-on-lost-directives --fail-on-unknown +``` + +The recipe pins tool source containing these unreleased commands separately +from the repository being checked. Installing the published v0.24.0 package or +using the rolling `v0` Action does not supply them. Choose task files relevant to +your repository and review each declared context; one context is not +whole-repository coverage. The workflow uses ordinary read-only PR permissions. + +The profile inherits its bounded metadata/glob/prose grammar, whole-repository +supported-source preflight, ignore rules and non-atomic filesystem limits. +Targets are logical names whose existence is not checked. No Cursor process, +model API, agent execution, maintainer contact or upload is involved. Try the +[four synthetic examples](../examples/cursor-scope-impact/README.md). diff --git a/docs/cursor-scope.md b/docs/cursor-scope.md index 4029f5e..c51b023 100644 --- a/docs/cursor-scope.md +++ b/docs/cursor-scope.md @@ -120,6 +120,8 @@ explicit. Each rule records mode, activation/reason, pattern count, metadata line numbers, body start, matched targets, directive-analysis status and the number of recognized directives considered for pairing. Inactive rules have zero considered directives even when their body contains supported prose. +Read `.mdc` sources additionally carry a newline-normalized `source_sha256`; +unread formats have a null hash. This binds captured text without exporting it. Reports retain source paths and supplied target paths. Raw descriptions, glob values and directive bodies are not exported; hashes are not anonymization or @@ -135,13 +137,16 @@ formats are counted but not interpreted. Unknown extension casing remains unknown. Existing containment checks and their non-atomic filesystem limits apply; this is not an isolated selected-file traversal. -Exit `0` means the advisory analysis completed, including signals and unknowns. +For standalone `scope PATH`, exit `0` means the advisory analysis completed, including signals and unknowns. Invalid inputs, unreadable/invalid UTF-8 sources or resource-limit failures use exit `2` without a successful JSON report. There is no exit `1` conflict gate, score, policy threshold, suppression handling or pass/fail safety result. Normal scan, compare, diff and Action scoring are unchanged. In particular, the old general contradiction check is not suppressed by this new command. -This is the first activation-model phase, not effective-policy inheritance, -configuration-change impact analysis, or execution-invariant verification. +The [comparison commands](cursor-scope-impact.md) additionally report bounded +before/after attachment and exact recognized-directive availability, with +explicit opt-in review gates. General effective-policy inheritance, semantic +constraint-impact analysis and execution-invariant verification remain outside +the implementation. Try the [runnable synthetic example](../examples/cursor-scope/README.md). diff --git a/docs/limitations.md b/docs/limitations.md index 6dcc593..4a7ccb2 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -96,8 +96,15 @@ not semantic-conflict proofs. Uninterpreted bodies and unknown activation stay visible; no signals is not a clean bill of health. Completed advisory analysis returns `0` even with signals; input/read/limit errors use `2`. Whole-repository supported-source preflight still applies. See the [grammar, report and limits](cursor-scope.md). -This first activation-model phase is not in published v0.24.0 artifacts and -does not implement constraint-impact comparison or execution verification. +Unreleased [comparison commands](cursor-scope-impact.md) additionally compare +the same declared context across trees/a local Git baseline. File lifecycle and +trigger losses are distinct from loss of aggregated exact recognized directives; +another active copy or remaining trigger may retain availability. Opt-in +caller-owned flags gate known recognition losses and unknowns separately with +exit `1`; default comparisons remain advisory. Neither semantic paraphrase +equivalence, arbitrary scope narrowing nor runtime compliance is established. +An unrecognized/empty baseline protects zero identities. These additions are not +in published v0.24.0 artifacts or the existing baseline score-policy schema. ## Command interpretation diff --git a/docs/simplified/limitations.md b/docs/simplified/limitations.md index 815820d..6d31ca2 100644 --- a/docs/simplified/limitations.md +++ b/docs/simplified/limitations.md @@ -46,7 +46,9 @@ v0.24.0+ 提供[工具源码身份（英文）](../report-identity.md)，用于 可选 `scope` 根据声明的文件路径，预测根目录 `.cursor/rules/` 规则的自动加载。它不计分，不观察 Cursor、不检查目标存在性、不解析其他格式／优先级，也不改变正常扫描和回归发现。inactive 仅表示这些路径未触发规则；手动引用、模型选择、不支持的元数据与运行设置仍未被观察。多个文件可触发不同 glob 的规则共同加载。 -有限英文语法识别的相反指令对只提供带来源行号的审核信号，不是语义冲突证明。未知加载条件和未解释的正文会保留；没有信号不代表没有问题。分析完成返回 `0`，包括存在信号的情况；输入／读取／限制错误返回 `2`。仍会执行全仓库支持来源预检查。详见[语法、报告与限制（英文）](../cursor-scope.md)。这是加载条件模型的第一阶段，尚未包含在已发布 v0.24.0 中，也未实现约束变更影响比较或执行验证。 +有限英文语法识别的相反指令对只提供带来源行号的审核信号，不是语义冲突证明。未知加载条件和未解释的正文会保留；没有信号不代表没有问题。独立上下文分析完成返回 `0`，包括存在信号的情况；输入／读取／限制错误返回 `2`。仍会执行全仓库支持来源预检查。详见[语法、报告与限制（英文）](../cursor-scope.md)。这些能力尚未包含在已发布 v0.24.0 中，也未实现通用语义约束影响分析或执行验证。 + +尚未发布的 [scope 比较命令（英文）](../cursor-scope-impact.md)还可比较两个目录或本地 Git 基线下相同的声明上下文，区分文件／触发路径变化与精确识别的指令失去全部活动来源。相同指令的另一份活动来源或其他仍匹配的任务文件可以保留可用性。可选调用方标志分别对已知识别丢失和未知返回 `1`；默认比较仍只给建议。不证明语义改写等价、任意路径上的范围收缩或运行时遵守。空／未识别基线保护零个指令身份。这些能力尚未进入 v0.24.0 或现有基线评分策略格式。 ## 命令与禁止上下文 diff --git a/docs/simplified/user-guide.md b/docs/simplified/user-guide.md index bda8d6b..db62fca 100644 --- a/docs/simplified/user-guide.md +++ b/docs/simplified/user-guide.md @@ -137,6 +137,15 @@ v0.24.0 之后的源码构建可根据你声明在上下文中的逻辑文件路 这是带来源行号、不计分的审核建议，不是实际 Agent 轨迹或阻断门禁。分析完成返回 `0`，包括存在信号／未知的情况；输入或读取失败返回 `2`。原有扫描和回归评分不变。该命令仍使用全仓库支持来源预检查，无关的失效指令链接也可能导致失败。详见[模型与限制（英文）](../cursor-scope.md)及[可运行的合成示例（英文）](../../examples/cursor-scope/README.md)。已发布的 v0.24.0 包和滚动 Action 不包含此命令。 +比较修改前后同一组声明的上下文： + +```bash +agent-config-score scope compare ../repo-base . --target src/web/app.ts --fail-on-lost-directives --fail-on-unknown +agent-config-score scope diff origin/main --target src/web/app.ts --fail-on-lost-directives --fail-on-unknown --json +``` + +这些尚未发布的命令分别报告文件变化、触发路径减少，以及精确识别的指令失去全部预计活动来源。另一条活动规则中的相同指令，或仍匹配的其他任务文件，可以保留可用性。两个标志显式启用调用方配置的审核门禁：丢失或未知返回 `1`，操作失败返回 `2`；不带标志时，完成的比较仍为建议性结果（`0`）。基线未识别到活动指令，就没有受保护的指令身份。详见[比较约定与 CI 示例（英文）](../cursor-scope-impact.md)及[四个合成对照（英文）](../../examples/cursor-scope-impact/README.md)。 + ## 可选：面向维护者的具体建议（v0.24.0+） ```bash diff --git a/docs/traditional/limitations.md b/docs/traditional/limitations.md index 6567351..eeb7d4e 100644 --- a/docs/traditional/limitations.md +++ b/docs/traditional/limitations.md @@ -46,7 +46,9 @@ v0.24.0+ 提供[工具原始碼身分（英文）](../report-identity.md)，用 選用 `scope` 根據宣告的檔案路徑，預測根目錄 `.cursor/rules/` 規則的自動載入。它不計分，不觀察 Cursor、不檢查目標存在性、不解析其他格式／優先順序，也不改變正常掃描和回歸發現。inactive 僅表示這些路徑未觸發規則；手動引用、模型選擇、不支援的中繼資料與執行設定仍未被觀察。多個檔案可觸發不同 glob 的規則共同載入。 -有限英文語法識別的相反指令對只提供帶來源行號的審閱訊號，不是語意衝突證明。未知載入條件和未解釋的正文會保留；沒有訊號不代表沒有問題。分析完成回傳 `0`，包括存在訊號的情況；輸入／讀取／限制錯誤回傳 `2`。仍會執行全儲存庫支援來源預先檢查。詳見[語法、報告與限制（英文）](../cursor-scope.md)。這是載入條件模型的第一階段，尚未包含在已發布 v0.24.0 中，也未實現約束變更影響比較或執行驗證。 +有限英文語法識別的相反指令對只提供帶來源行號的審閱訊號，不是語意衝突證明。未知載入條件和未解釋的正文會保留；沒有訊號不代表沒有問題。獨立上下文分析完成回傳 `0`，包括存在訊號的情況；輸入／讀取／限制錯誤回傳 `2`。仍會執行全儲存庫支援來源預先檢查。詳見[語法、報告與限制（英文）](../cursor-scope.md)。這些能力尚未包含在已發布 v0.24.0 中，也未實現通用語意約束影響分析或執行驗證。 + +尚未發布的 [scope 比較命令（英文）](../cursor-scope-impact.md)還可比較兩個目錄或本機 Git 基準下相同的宣告上下文，區分檔案／觸發路徑變化與精確識別的指令失去全部活動來源。相同指令的另一份活動來源或其他仍符合的任務檔案可以保留可用性。選用呼叫端旗標分別對已知識別遺失和未知回傳 `1`；預設比較仍只給建議。不證明語意改寫等價、任意路徑上的範圍收縮或執行時遵守。空／未識別基準保護零個指令身分。這些能力尚未進入 v0.24.0 或現有基準評分策略格式。 ## 命令與禁止上下文 diff --git a/docs/traditional/user-guide.md b/docs/traditional/user-guide.md index 27f17cc..7d27439 100644 --- a/docs/traditional/user-guide.md +++ b/docs/traditional/user-guide.md @@ -137,6 +137,15 @@ v0.24.0 之後的原始碼建置可根據你宣告在上下文中的邏輯檔案 這是帶來源行號、不計分的審閱建議，不是實際 Agent 軌跡或阻斷門禁。分析完成回傳 `0`，包括存在訊號／未知的情況；輸入或讀取失敗回傳 `2`。原有掃描和回歸評分不變。此命令仍使用全儲存庫支援來源預先檢查，無關的失效指令連結也可能導致失敗。詳見[模型與限制（英文）](../cursor-scope.md)及[可執行的合成範例（英文）](../../examples/cursor-scope/README.md)。已發布的 v0.24.0 套件和滾動 Action 不包含此命令。 +比較修改前後同一組宣告的上下文： + +```bash +agent-config-score scope compare ../repo-base . --target src/web/app.ts --fail-on-lost-directives --fail-on-unknown +agent-config-score scope diff origin/main --target src/web/app.ts --fail-on-lost-directives --fail-on-unknown --json +``` + +這些尚未發布的命令分別報告檔案變化、觸發路徑減少，以及精確識別的指令失去全部預測活動來源。另一條活動規則中的相同指令，或仍符合的其他任務檔案，可以保留可用性。兩個旗標明確啟用呼叫端設定的審閱門禁：遺失或未知回傳 `1`，操作失敗回傳 `2`；不帶旗標時，完成的比較仍為建議性結果（`0`）。基準未識別到活動指令，就沒有受保護的指令身分。詳見[比較約定與 CI 範例（英文）](../cursor-scope-impact.md)及[四個合成對照（英文）](../../examples/cursor-scope-impact/README.md)。 + ## 選用：面向維護者的具體建議（v0.24.0+） ```bash diff --git a/docs/translations.md b/docs/translations.md index 9391143..5bae9d9 100644 --- a/docs/translations.md +++ b/docs/translations.md @@ -161,6 +161,15 @@ remain identical in all three guides. The detailed grammar and runnable synthetic fixture are linked in English; this is not an observed agent run, constraint-impact engine or published v0.24.0/Action update. +On 2026-11-04, built on `e68506c5fc3e1d5bc2aa1460fe2d5a0d7fdbd36f`, both core +guides/limitations were synchronized for unreleased Cursor attachment-change +comparisons: same declared context, captured evidence, retained copies and +multi-file triggers, exact-recognition loss versus unknown, caller-owned opt-in +gates, empty baselines and Git/operational failure semantics. The new executable +guide examples are identical in all three languages. Detailed comparison/CI +guidance and four synthetic controls remain English references. No application +documents, published package/Action refs or frozen evaluation data were changed. + When an English source page changes, review both variants together and record the new source revision here. Preserve executable examples, rule IDs and JSON keys verbatim. Keep the score limitations and failure/exit semantics aligned; diff --git a/docs/user-guide.md b/docs/user-guide.md index 70cf6a3..5a67228 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -235,6 +235,21 @@ invalid instruction links can stop it. See the [model and limits](cursor-scope.m and [runnable synthetic example](../examples/cursor-scope/README.md). This command is not in the published v0.24.0 package or rolling Action. +Compare the same declared context before and after a change: + +```bash +agent-config-score scope compare ../repo-base . --target src/web/app.ts --fail-on-lost-directives --fail-on-unknown +agent-config-score scope diff origin/main --target src/web/app.ts --fail-on-lost-directives --fail-on-unknown --json +``` + +These unreleased commands separate file changes, lost trigger matches and exact +recognized directives losing all predicted active sources. Another active copy +or a remaining multi-file trigger can preserve availability. Both flags opt into +a caller-configured review gate: loss or unknown uses `1`; operational failures +use `2`. Without flags, completed comparisons remain advisory (`0`). No recognized +baseline means zero protected identities. See the [comparison contract and CI +recipe](cursor-scope-impact.md) and [four synthetic controls](../examples/cursor-scope-impact/README.md). + ## Optional: actionable maintainer results (v0.24.0+) ```bash diff --git a/examples/README.md b/examples/README.md index 0f2e25e..a1afe3c 100644 --- a/examples/README.md +++ b/examples/README.md @@ -8,6 +8,9 @@ it is not independent accuracy or adoption evidence. The separate [Cursor file-context example](cursor-scope/README.md) demonstrates single-file activation, multi-file co-activation and explicitly unknown model selection. It needs an unreleased source build and does not change scoring. +The [activation-change example](cursor-scope-impact/README.md) then compares +glob narrowing, retained multi-file triggers, renamed rule sources and manual +loading uncertainty with an explicitly requested review gate. These templates provide conservative repository workflow, verification, safety, and handoff guidance without assuming a programming language or package manager. diff --git a/examples/cursor-scope-impact/README.md b/examples/cursor-scope-impact/README.md new file mode 100644 index 0000000..c3b3541 --- /dev/null +++ b/examples/cursor-scope-impact/README.md @@ -0,0 +1,56 @@ +# Cursor activation-change example + +Requires an unreleased source installation after v0.24.0. These are synthetic +development fixtures, not real user cases or accuracy measurements. No agent or +instruction-file command is executed. Logical target files need not exist. + +The same instruction, `Never delete files without approval`, appears in all +four trees. Only its attachment metadata or source filename changes. From the +repository root: + +```bash +agent-config-score scope compare examples/cursor-scope-impact/base examples/cursor-scope-impact/narrowed --target src/api/service.py --fail-on-lost-directives --fail-on-unknown +``` + +Expected exit **1**: `src/**` becomes `src/web/**`. The declared API file stops +triggering the rule. One recognized baseline-active directive becomes inactive; +the report points to metadata line 2 and directive line 5. Ordinary standalone +scans of these two trees both score 100; that does not certify deletion safety. + +```bash +agent-config-score scope compare examples/cursor-scope-impact/base examples/cursor-scope-impact/narrowed --target src/api/service.py --target src/web/app.ts --fail-on-lost-directives --fail-on-unknown +``` + +Expected exit **0**: the frontend file still triggers the rule in this multi-file +context. The report retains the lost API trigger match, but the exact directive +still has an active source. Trigger globs do not limit where an attached +instruction can have an effect. + +```bash +agent-config-score scope compare examples/cursor-scope-impact/base examples/cursor-scope-impact/renamed --target src/api/service.py --fail-on-lost-directives --fail-on-unknown +``` + +Expected exit **0**: the old rule file is removed and a new one is added, but the +same normalized directive/polarity remains in an automatically attached rule. +File lifecycle changes stay visible without becoming a lost-directive failure. + +```bash +agent-config-score scope compare examples/cursor-scope-impact/base examples/cursor-scope-impact/uncertain --target src/api/service.py --fail-on-lost-directives --fail-on-unknown --json +``` + +Expected exit **1** for **unknown**, not a definite inactive directive. Removing +globs with `alwaysApply: false` makes loading depend on manual mentions, which +this profile does not observe. `--fail-on-unknown` requests review of that +uncertainty; `--fail-on-lost-directives` alone does not block it. + +For a real local Git change, use: + +```bash +agent-config-score scope diff origin/main --target src/api/service.py --fail-on-lost-directives --fail-on-unknown --json +``` + +The candidate is the working tree, including uncommitted changes. The report +retains the resolved baseline commit. Default comparisons are advisory; the two +flags explicitly opt into a review gate. See the [comparison contract and CI +example](../../docs/cursor-scope-impact.md) for exact-recognition, unknown, +privacy and operational-error limits. diff --git a/examples/cursor-scope-impact/base/.cursor/rules/policy.mdc b/examples/cursor-scope-impact/base/.cursor/rules/policy.mdc new file mode 100644 index 0000000..f5bd561 --- /dev/null +++ b/examples/cursor-scope-impact/base/.cursor/rules/policy.mdc @@ -0,0 +1,5 @@ +--- +globs: "src/**" +alwaysApply: false +--- +Never delete files without approval. diff --git a/examples/cursor-scope-impact/narrowed/.cursor/rules/policy.mdc b/examples/cursor-scope-impact/narrowed/.cursor/rules/policy.mdc new file mode 100644 index 0000000..432eb21 --- /dev/null +++ b/examples/cursor-scope-impact/narrowed/.cursor/rules/policy.mdc @@ -0,0 +1,5 @@ +--- +globs: "src/web/**" +alwaysApply: false +--- +Never delete files without approval. diff --git a/examples/cursor-scope-impact/renamed/.cursor/rules/approval.mdc b/examples/cursor-scope-impact/renamed/.cursor/rules/approval.mdc new file mode 100644 index 0000000..f5bd561 --- /dev/null +++ b/examples/cursor-scope-impact/renamed/.cursor/rules/approval.mdc @@ -0,0 +1,5 @@ +--- +globs: "src/**" +alwaysApply: false +--- +Never delete files without approval. diff --git a/examples/cursor-scope-impact/uncertain/.cursor/rules/policy.mdc b/examples/cursor-scope-impact/uncertain/.cursor/rules/policy.mdc new file mode 100644 index 0000000..f7a0c39 --- /dev/null +++ b/examples/cursor-scope-impact/uncertain/.cursor/rules/policy.mdc @@ -0,0 +1,4 @@ +--- +alwaysApply: false +--- +Never delete files without approval. diff --git a/scripts/verify_installed_package.py b/scripts/verify_installed_package.py index dad398d..6e52664 100644 --- a/scripts/verify_installed_package.py +++ b/scripts/verify_installed_package.py @@ -101,6 +101,69 @@ def verify_cursor_scope(commands: list[list[str]], workspace: Path) -> None: "Declared-context analysis weakened ordinary danger scanning") +def verify_cursor_scope_impact(commands: list[list[str]], workspace: Path) -> None: + base, head = workspace / "scope base", workspace / "scope head" + for root in (base, head): + (root / ".cursor/rules").mkdir(parents=True) + + def text(pattern): + return f'---\nglobs: "{pattern}"\nalwaysApply: false\n---\nNever expose synthetic-private-invariant-value.\n' + + baseline, candidate = base / ".cursor/rules/policy.mdc", head / ".cursor/rules/policy.mdc" + baseline.write_text(text("src/**"), encoding="utf-8") + candidate.write_text(text("src/web/**"), encoding="utf-8") + arguments = ["scope", "compare", str(base), str(head), "--target", "src/api/service.py"] + strict = ["--fail-on-lost-directives", "--fail-on-unknown", "--json"] + original = (baseline.read_bytes(), candidate.read_bytes()) + for command in commands: + report_text = run([*command, *arguments, *strict], workspace, expected=1) + report = json.loads(report_text) + require(report["summary"]["recognized_active_losses"] == 1 + and report["rule_changes"][0]["head"]["metadata_lines"]["globs"] == 2 + and report["directive_changes"][0]["base"]["sources"][0]["line"] == 5, + "Installed comparison lost scope change or original evidence") + require(report["scored"] is False and report["review_gate"]["reasons"] == ["recognized_active_directives_lost"], + "Installed scope gate confused score, uncertainty and recognized loss") + require("synthetic-private" not in report_text and "src/web/**" not in report_text, + "Scope comparison exported raw source values") + advisory = json.loads(run([*command, *arguments, "--json"], workspace)) + require(advisory["review_gate"]["failed"] is False, "Comparison became a default blocking gate") + multi = json.loads(run([*command, *arguments, "--target", "src/web/app.ts", *strict], workspace)) + require(multi["summary"]["recognized_active_losses"] == 0 + and multi["summary"]["retained_active_directives"] == 1, + "Remaining multi-file trigger did not preserve attachment") + require((baseline.read_bytes(), candidate.read_bytes()) == original, "Scope comparison modified input files") + candidate.rename(candidate.with_name("renamed.mdc")) + candidate = candidate.with_name("renamed.mdc") + candidate.write_bytes(baseline.read_bytes()) + renamed = json.loads(run([*commands[0], *arguments, *strict], workspace)) + require(renamed["summary"]["rules_removed"] == 1 and renamed["summary"]["recognized_active_losses"] == 0, + "Rule rename became a false directive loss") + candidate.write_text("---\nalwaysApply: false\n---\nNever expose synthetic-private-invariant-value.\n", encoding="utf-8") + unknown = json.loads(run([*commands[0], *arguments, *strict], workspace, expected=1)) + require(unknown["summary"]["recognized_active_losses"] == 0 + and unknown["review_gate"]["reasons"] == ["unknown_scope_or_body"], "Unknown loading became a definite loss") + hooks = workspace / "scope-empty-hooks" + hooks.mkdir() + for args in (["init", "-q", "-b", "main", "--template="], ["config", "core.hooksPath", str(hooks)], + ["config", "commit.gpgsign", "false"], ["config", "user.name", "Scope Acceptance"], + ["config", "user.email", "scope@example.invalid"], ["add", "."], + ["-c", "core.autocrlf=false", "commit", "-qm", "scope baseline"]): + run(["git", *args], base) + sha = run(["git", "rev-parse", "HEAD"], base).strip() + baseline.write_text(text("src/web/**"), encoding="utf-8") + git_args = ["scope", "diff", "HEAD", "--path", str(base), "--target", "src/api/service.py", *strict] + for command in commands: + diff = json.loads(run([*command, *git_args], workspace, expected=1)) + require(diff["comparison_inputs"]["base_commit"] == sha and diff["summary"]["recognized_active_losses"] == 1, + "Installed Git scope comparison lost baseline identity or change") + baseline.write_bytes(b"private-invalid-utf8-\xff") + require(run([*commands[0], *git_args], workspace, expected=2) == "", "Git scope source error emitted partial JSON") + worktrees = run(["git", "worktree", "list", "--porcelain"], base) + require(sum(line.startswith("worktree ") for line in worktrees.splitlines()) == 1, + "Git scope comparison leaked a baseline worktree") + + def verify(python: Path, expected_version: str) -> list[str]: checks: list[str] = [] with tempfile.TemporaryDirectory(prefix="acs-acceptance-") as directory: @@ -132,6 +195,8 @@ def verify(python: Path, expected_version: str) -> list[str]: verify_cursor_scope(commands, workspace) checks.append("Cursor declared-file activation, multi-file opposition, unknown metadata, line evidence and read-only aliases") + verify_cursor_scope_impact(commands, workspace) + checks.append("Cursor compare/Git change gate across aliases, retained triggers/renames, unknowns, source privacy and cleanup") run([*cli, "init", "--dry-run"], repo) require(list(repo.iterdir()) == [], "Dry-run wrote files") diff --git a/src/agent_config_score/cli.py b/src/agent_config_score/cli.py index 8c7f6db..513b835 100644 --- a/src/agent_config_score/cli.py +++ b/src/agent_config_score/cli.py @@ -8,15 +8,15 @@ from . import __version__ from .config import ConfigError, Policy, load_policy from .feedback import feedback_markdown -from .gitdiff import GitError, baseline_worktree, repository_root +from .gitdiff import GitError, baseline_worktree, detect_base_ref, repository_root from .history import HistoryError, load_history, summarize_history from .guardrails import GuardrailError, TEMPLATES, review_guardrails from .initializer import InitError, WORKFLOW_ACTION_REF, initialize_repository from .regression import compare, markdown_report from .rules import RULES, get_rule, rule_guidance from .sarif import sarif_report -from .scanner import analyze, analyze_cursor_scope, badge_svg, html_report -from .scopes import ScopeInputError +from .scanner import analyze, analyze_cursor_scope, compare_cursor_scope, badge_svg, html_report +from .scopes import ScopeInputError, normalize_targets RESET = "\033[0m" BOLD = "\033[1m" @@ -31,6 +31,8 @@ agent-config-score feedback [PATH] [options] agent-config-score guardrails [PATH] [options] agent-config-score scope [PATH] --target FILE [options] + agent-config-score scope compare BASE HEAD --target FILE [options] + agent-config-score scope diff [BASE_REF] --target FILE [options] agent-config-score diff BASE_REF [options] agent-config-score compare BASE HEAD [options] @@ -458,8 +460,10 @@ def _main_guardrails(argv: list[str]) -> int: def _main_scope(argv: list[str]) -> int: + if argv and argv[0] in {"compare", "diff"}: + return _main_scope_comparison(argv[0], argv[1:]) parser = argparse.ArgumentParser(prog="agent-config-score scope", description=( - "Unscored Cursor project-rule activation prediction for declared file context; not an observed agent run or safety gate.")) + "Unscored Cursor rule activation prediction. Use scope compare or scope diff for change impact; not an observed agent run.")) parser.add_argument("path", nargs="?", default=".", help="Repository directory") parser.add_argument("--target", action="append", required=True, metavar="FILE", help="Repository-relative file assumed in context; repeat for multi-file context; existence is not checked") @@ -489,6 +493,74 @@ def _main_scope(argv: list[str]) -> int: return 0 +def _main_scope_comparison(command: str, argv: list[str]) -> int: + parser = argparse.ArgumentParser(prog=f"agent-config-score scope {command}", description=( + "Compare predicted Cursor attachment and exact recognized directives for the same declared file context.")) + if command == "compare": + parser.add_argument("base", help="Baseline repository directory") + parser.add_argument("head", help="Candidate repository directory") + else: + parser.add_argument("base_ref", nargs="?", help="Local baseline ref; autodetected when omitted (no fetch)") + parser.add_argument("--path", default=".", help="Git repository directory; the candidate is its working tree") + parser.add_argument("--target", action="append", required=True, metavar="FILE", + help="Logical repository-relative file in both contexts; repeat for multi-file context") + parser.add_argument("--json", action="store_true", help="Print structured before/after evidence") + parser.add_argument("--fail-on-lost-directives", action="store_true", + help="Exit 1 when a baseline-active exact directive becomes inactive or unrecognized") + parser.add_argument("--fail-on-unknown", action="store_true", + help="Exit 1 when either tree has unknown rule activation or a non-inactive uninterpreted body") + args = parser.parse_args(argv) + try: + # Reject bad logical paths before Git metadata changes or source reads. + normalize_targets(args.target) + if command == "compare": + report = compare_cursor_scope(Path(args.base), Path(args.head), args.target) + else: + repo = repository_root(Path(args.path)) + ref = args.base_ref if args.base_ref is not None else detect_base_ref(repo) + with baseline_worktree(repo, ref) as (base, sha): + report = compare_cursor_scope(base, repo, args.target) + report["comparison_inputs"] = {"base": "git_ref", "base_ref": ref, "base_commit": sha, + "head": "working_tree"} + except (ScopeInputError, GitError) as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + summary = report["summary"] + reasons = [] + if args.fail_on_lost_directives and summary["recognized_active_losses"]: + reasons.append("recognized_active_directives_lost") + if args.fail_on_unknown and summary["has_unknown"]: + reasons.append("unknown_scope_or_body") + report["review_gate"] = {"fail_on_lost_directives": args.fail_on_lost_directives, + "fail_on_unknown": args.fail_on_unknown, "failed": bool(reasons), "reasons": reasons} + if args.json: + print(json.dumps(report, indent=2, ensure_ascii=False)) + else: + print(f"Cursor scope comparison ({report['comparison_profile']}; unscored)") + print("Same declared targets: " + ", ".join(report["targets"])) + for change in report["rule_changes"]: + print(f"{change['change']:8} {change['file']}: {change['transition']}") + for side in ("base", "head"): + row = change[side] + if row: + evidence = ", ".join(f"{key}:{line}" for key, line in row["metadata_lines"].items()) + print(f" {side}: {row['reason']}; metadata lines [{evidence}]; matched {', '.join(row['matched_targets']) or '(none)'}") + for change in report["directive_changes"]: + locations = [] + for side in ("base", "head"): + origins = ", ".join(f"{s['file']}:{s['line']}" for s in change[side]["sources"]) or "(no recognized source)" + locations.append(f"{side} {origins}") + print(f"Directive {change['polarity']} {change['directive_sha256'][:12]}: {change['transition']}; " + "; ".join(locations)) + print(f"Recognized active directives: {summary['baseline_active_directives']} baseline, " + f"{summary['retained_active_directives']} retained, {summary['recognized_active_losses']} lost. " + f"Unknown rules: {summary['unknown_rules']}; uninterpreted non-inactive bodies: {summary['uninterpreted_noninactive_bodies']}.") + if not summary["baseline_active_directives"]: + print("Baseline contains no recognized active directives to preserve.") + print("Review gate: " + ("FAILED (" + ", ".join(reasons) + ")" if reasons else "not triggered")) + print("Exact recognition and predicted attachment only; no semantic-equivalence, agent-behavior or safety verdict.") + return 1 if reasons else 0 + + def _main_compare(argv: list[str]) -> int: args = build_compare_parser().parse_args(argv) base = _existing_dir(args.base) diff --git a/src/agent_config_score/entrypoint.py b/src/agent_config_score/entrypoint.py index 52ea2d4..8b886c3 100644 --- a/src/agent_config_score/entrypoint.py +++ b/src/agent_config_score/entrypoint.py @@ -20,6 +20,8 @@ agent-config-score feedback [PATH] [options] agent-config-score guardrails [PATH] [options] agent-config-score scope [PATH] --target FILE [options] + agent-config-score scope compare BASE HEAD --target FILE [options] + agent-config-score scope diff [BASE_REF] --target FILE [options] agent-config-score diff [BASE_REF] [options] agent-config-score compare BASE HEAD [options] diff --git a/src/agent_config_score/scanner.py b/src/agent_config_score/scanner.py index 2ff9448..7a50c83 100644 --- a/src/agent_config_score/scanner.py +++ b/src/agent_config_score/scanner.py @@ -1325,6 +1325,12 @@ def analyze_cursor_scope(root: Path, targets: list[str]) -> dict: Default scan/regression scoring is intentionally unchanged. """ target_paths = normalize_targets(targets) # Validate before repository reads. + report, _ = _cursor_scope_snapshot(root, target_paths) + return report + + +def _cursor_scope_snapshot(root: Path, target_paths: tuple[str, ...]) -> tuple[dict, dict]: + """Capture each source once for both activation and directive evidence.""" root = root.resolve(strict=True) if not root.is_dir(): raise OSError(errno.ENOTDIR, "Scope repository must be a directory") @@ -1333,6 +1339,7 @@ def analyze_cursor_scope(root: Path, targets: list[str]) -> dict: sources = [path for path in discovered if path.relative_to(root).as_posix().startswith(".cursor/rules/")] rules = [] directives = defaultdict(lambda: {"pos": [], "neg": []}) + directive_index = defaultdict(list) for path in sources: file = path.relative_to(root).as_posix() text = "" @@ -1346,6 +1353,8 @@ def analyze_cursor_scope(root: Path, targets: list[str]) -> dict: raise OSError(exc.errno, "Unable to read Cursor scope source") from exc model = parse_cursor_rule(file, text) activation = model.activate(target_paths) + activation["source_sha256"] = (hashlib.sha256(text.encode("utf-8")).hexdigest() + if path.suffix == ".mdc" else None) if model.body_line is None: candidates, interpretable = [], False else: @@ -1355,14 +1364,16 @@ def analyze_cursor_scope(root: Path, targets: list[str]) -> dict: activation["directive_analysis"] = "bounded_prose" if interpretable else "not_interpreted" activation["recognized_directives"] = 0 rules.append(activation) - if activation["activation"] == "inactive" or not interpretable: + if not interpretable: continue for number, line in candidates: for polarity, body, _ in _directives(line): - activation["recognized_directives"] += 1 key = hashlib.sha256(body.encode("utf-8")).hexdigest() - directives[key][polarity].append({"file": file, "line": model.body_line + number - 1, - "polarity": polarity}) + source = {"file": file, "line": model.body_line + number - 1, "polarity": polarity} + directive_index[(polarity, key)].append(source) + if activation["activation"] != "inactive": + activation["recognized_directives"] += 1 + directives[key][polarity].append(source) by_file = {rule["file"]: rule for rule in rules} pairs = sum(len(rows["pos"]) * len(rows["neg"]) for rows in directives.values()) if pairs > 10000: @@ -1379,7 +1390,7 @@ def analyze_cursor_scope(root: Path, targets: list[str]) -> dict: "shared_target_matches": sorted(set(left["matched_targets"]) & set(right["matched_targets"]))}) signals.sort(key=lambda row: (row["sources"][0]["file"], row["sources"][0]["line"], row["sources"][1]["file"], row["sources"][1]["line"], row["directive_sha256"])) - return {"schema_version": 1, "analysis": "cursor_file_context", "profile": CURSOR_SCOPE_PROFILE, + report = {"schema_version": 1, "analysis": "cursor_file_context", "profile": CURSOR_SCOPE_PROFILE, "tool": identity.to_dict(), "status": "completed", "scored": False, "targets": list(target_paths), "targets_existence_checked": False, "observed_agent_context": False, "activation_inputs": "declared_file_paths_only", @@ -1391,6 +1402,90 @@ def analyze_cursor_scope(root: Path, targets: list[str]) -> dict: "uninterpreted_bodies": sum(rule["directive_analysis"] == "not_interpreted" for rule in rules), "active_signals": sum(row["activation"] == "active" for row in signals), "uncertain_signals": sum(row["activation"] == "unknown" for row in signals)}} + return report, dict(directive_index) + + +def _cursor_directive_states(report: dict, index: dict, keys: list[tuple[str, str]]) -> dict: + by_file = {row["file"]: row for row in report["rules"]} + opaque = [row["file"] for row in report["rules"] + if row["activation"] != "inactive" and row["directive_analysis"] == "not_interpreted"] + states = {} + for key in keys: + sources = [{**source, "activation": by_file[source["file"]]["activation"]} + for source in index.get(key, ())] + if any(row["activation"] == "active" for row in sources): + availability = "active" + elif opaque or any(row["activation"] == "unknown" for row in sources): + availability = "unknown" + else: + availability = "inactive" if sources else "not_recognized" + states[key] = {"availability": availability, "sources": sources, "uninterpreted_sources": opaque} + return states + + +def compare_cursor_scope(base_root: Path, head_root: Path, targets: list[str]) -> dict: + """Compare predicted attachment and exact recognized directives, not intent. + + One identical declared file context is used for both captured trees. Source + failures abort; unsupported bodies cannot be mistaken for proven absence. + """ + target_paths = normalize_targets(targets) + base, base_index = _cursor_scope_snapshot(base_root, target_paths) + head, head_index = _cursor_scope_snapshot(head_root, target_paths) + if base["tool"] != head["tool"]: + raise OSError(errno.EAGAIN, "Tool identity changed during Cursor scope comparison") + base_rules = {row["file"]: row for row in base["rules"]} + head_rules = {row["file"]: row for row in head["rules"]} + rule_changes = [] + for file in sorted(base_rules.keys() | head_rules.keys()): + before, after = base_rules.get(file), head_rules.get(file) + if before == after: + continue + left = before["activation"] if before else "missing" + right = after["activation"] if after else "missing" + lost = set(before["matched_targets"] if before else ()) - set(after["matched_targets"] if after else ()) + gained = set(after["matched_targets"] if after else ()) - set(before["matched_targets"] if before else ()) + rule_changes.append({"file": file, "change": "added" if before is None else "removed" if after is None else "modified", + "transition": f"{left}_to_{right}", "base": before, "head": after, + "trigger_targets_lost": sorted(lost), "trigger_targets_gained": sorted(gained), + "automatic_attachment_lost": left == "active" and right in {"inactive", "missing"}}) + directive_changes = [] + base_active = retained_active = losses = gains = uncertain = 0 + keys = sorted(base_index.keys() | head_index.keys()) + base_states = _cursor_directive_states(base, base_index, keys) + head_states = _cursor_directive_states(head, head_index, keys) + for key in keys: + before, after = base_states[key], head_states[key] + left, right = before["availability"], after["availability"] + lost = left == "active" and right in {"inactive", "not_recognized"} + gained = right == "active" and left != "active" + base_active += left == "active" + retained_active += left == right == "active" + losses += lost + gains += gained + uncertain += "unknown" in {left, right} + if before != after: + directive_changes.append({"polarity": key[0], "directive_sha256": key[1], + "transition": f"{left}_to_{right}", "base": before, "head": after, + "recognized_active_lost": lost, "recognized_active_gained": gained}) + unknown_rules = sum(row["activation"] == "unknown" for report in (base, head) for row in report["rules"]) + opaque_bodies = sum(row["activation"] != "inactive" and row["directive_analysis"] == "not_interpreted" + for report in (base, head) for row in report["rules"]) + return {"schema_version": 1, "analysis": "cursor_file_context_comparison", "profile": CURSOR_SCOPE_PROFILE, + "comparison_profile": "cursor-activation-delta-v1", "tool": base["tool"], + "status": "completed", "scored": False, "targets": list(target_paths), + "observed_agent_context": False, "targets_existence_checked": False, + "comparison_inputs": {"base": "supplied_tree", "head": "supplied_tree"}, + "base": base, "head": head, "rule_changes": rule_changes, "directive_changes": directive_changes, + "summary": {"rules_added": sum(row["change"] == "added" for row in rule_changes), + "rules_removed": sum(row["change"] == "removed" for row in rule_changes), + "rules_modified": sum(row["change"] == "modified" for row in rule_changes), + "automatic_attachment_losses": sum(row["automatic_attachment_lost"] for row in rule_changes), + "baseline_active_directives": base_active, "retained_active_directives": retained_active, + "recognized_active_losses": losses, "recognized_active_gains": gains, + "uncertain_directives": uncertain, "unknown_rules": unknown_rules, + "uninterpreted_noninactive_bodies": opaque_bodies, + "has_unknown": bool(unknown_rules or opaque_bodies)}} def _matching_suppression(finding: Finding, suppressions: tuple[Suppression, ...]) -> Suppression | None: diff --git a/tests/test_cursor_scope_impact.py b/tests/test_cursor_scope_impact.py new file mode 100644 index 0000000..dc1058d --- /dev/null +++ b/tests/test_cursor_scope_impact.py @@ -0,0 +1,281 @@ +import contextlib +import errno +import io +import json +from pathlib import Path +import subprocess +import tempfile +import unittest +from unittest.mock import patch + +from agent_config_score.entrypoint import main +from agent_config_score.provenance import ToolIdentity +from agent_config_score.regression import compare +from agent_config_score.scanner import analyze, compare_cursor_scope + + +def mdc(pattern="src/**", body="Never use tabs.\n"): + return f'---\nglobs: "{pattern}"\nalwaysApply: false\n---\n{body}' + + +class CursorScopeImpactTests(unittest.TestCase): + def setUp(self): + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + self.root = Path(temporary.name).resolve() + self.base, self.head = self.root / "baseline", self.root / "candidate" + self.base.mkdir() + self.head.mkdir() + + def write(self, root, name="policy.mdc", text=None): + path = root / ".cursor" / "rules" / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(mdc() if text is None else text, encoding="utf-8") + return path + + def invoke(self, args): + stdout, stderr = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr): + code = main(args) + return code, stdout.getvalue(), stderr.getvalue() + + def command(self, *flags): + return ["scope", "compare", str(self.base), str(self.head), "--target", "src/api/a.py", *flags] + + def test_glob_narrowing_loses_activation_while_old_scores_and_gate_are_equal(self): + self.write(self.base) + self.write(self.head, text=mdc("src/web/**")) + result = compare_cursor_scope(self.base, self.head, ["src/api/a.py"]) + self.assertEqual([analyze(root).score for root in (self.base, self.head)], [100, 100]) + self.assertEqual(compare(self.base, self.head).new_errors, []) + self.assertEqual(result["summary"]["recognized_active_losses"], 1) + change = result["rule_changes"][0] + self.assertEqual(change["transition"], "active_to_inactive") + self.assertEqual(change["trigger_targets_lost"], ["src/api/a.py"]) + self.assertEqual(change["head"]["metadata_lines"]["globs"], 2) + directive = result["directive_changes"][0] + self.assertEqual(directive["transition"], "active_to_inactive") + self.assertEqual(directive["base"]["sources"][0]["line"], 5) + self.assertEqual(directive["head"]["sources"][0]["activation"], "inactive") + self.assertFalse(result["scored"]) + self.assertFalse(result["observed_agent_context"]) + + def test_multi_file_context_retains_instruction_despite_lost_trigger_match(self): + self.write(self.base) + self.write(self.head, text=mdc("src/web/**")) + result = compare_cursor_scope(self.base, self.head, ["src/api/a.py", "src/web/app.ts"]) + self.assertEqual(result["rule_changes"][0]["trigger_targets_lost"], ["src/api/a.py"]) + self.assertEqual(result["summary"]["automatic_attachment_losses"], 0) + self.assertEqual(result["summary"]["recognized_active_losses"], 0) + self.assertEqual(result["summary"]["retained_active_directives"], 1) + + def test_rename_preserves_directive_while_file_removal_stays_visible(self): + self.write(self.base, "old.mdc") + self.write(self.head, "renamed.mdc") + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(result["summary"]["rules_removed"], 1) + self.assertEqual(result["summary"]["rules_added"], 1) + self.assertEqual(result["summary"]["automatic_attachment_losses"], 1) + self.assertEqual(result["summary"]["recognized_active_losses"], 0) + self.assertEqual(result["directive_changes"][0]["transition"], "active_to_active") + self.assertEqual(self.invoke(self.command("--fail-on-lost-directives", "--fail-on-unknown"))[0], 0) + + def test_another_active_copy_retains_directive_but_inactive_copy_does_not(self): + self.write(self.base, "old.mdc") + path = self.write(self.head, "copy.mdc", mdc("docs/**")) + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(result["summary"]["recognized_active_losses"], 1) + path.write_text(mdc(), encoding="utf-8") + self.assertEqual(compare_cursor_scope(self.base, self.head, ["src/a"])["summary"]["recognized_active_losses"], 0) + + def test_manual_model_selected_and_unsupported_rules_are_uncertain_not_definite_loss(self): + self.write(self.base) + for header in ("alwaysApply: false", "description: Review code\nalwaysApply: false", + 'globs: "{src,lib}/**"\nalwaysApply: false'): + with self.subTest(header=header): + self.write(self.head, text="---\n" + header + "\n---\nNever use tabs.\n") + result = compare_cursor_scope(self.base, self.head, ["src/api/a.py"]) + self.assertEqual(result["summary"]["recognized_active_losses"], 0) + self.assertEqual(result["directive_changes"][0]["transition"], "active_to_unknown") + code, stdout, stderr = self.invoke(self.command("--fail-on-lost-directives", "--fail-on-unknown", "--json")) + self.assertEqual((code, stderr), (1, "")) + self.assertEqual(json.loads(stdout)["review_gate"]["reasons"], ["unknown_scope_or_body"]) + + def test_uninterpreted_candidate_can_hide_a_replacement_and_remains_unknown(self): + self.write(self.base) + for text in ("---\nalwaysApply: true\nNever use tabs.\n", + "---\nalwaysApply: true\n---\nNever use tabs.\n", + "---\nalwaysApply: true\n---\n## Examples\nNever use tabs.\n"): + with self.subTest(text=text): + self.write(self.head, text=text) + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(result["summary"]["recognized_active_losses"], 0) + self.assertEqual(result["summary"]["uninterpreted_noninactive_bodies"], 1) + self.assertEqual(result["directive_changes"][0]["head"]["availability"], "unknown") + + def test_unknown_baseline_is_not_invented_as_a_protected_active_directive(self): + self.write(self.base, text="---\nalwaysApply: false\n---\nNever use tabs.\n") + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(result["summary"]["baseline_active_directives"], 0) + self.assertEqual(result["summary"]["recognized_active_losses"], 0) + self.assertEqual(self.invoke(self.command("--fail-on-unknown"))[0], 1) + + def test_ignored_md_and_closed_examples_do_not_preserve_active_directive(self): + self.write(self.base) + self.write(self.head, "example.md", "Never use tabs.\n") + self.write(self.head, text=mdc(body="## Examples\nNever use tabs.\n## Policy\nReview the change.\n")) + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(result["summary"]["recognized_active_losses"], 1) + self.assertFalse(result["summary"]["has_unknown"]) + + def test_polarity_reversal_loses_one_exact_directive_and_adds_the_other(self): + self.write(self.base) + self.write(self.head, text=mdc(body="Always use tabs.\n")) + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(result["summary"]["recognized_active_losses"], 1) + self.assertEqual(result["summary"]["recognized_active_gains"], 1) + self.assertEqual(len({r["directive_sha256"] for r in result["directive_changes"]}), 1) + + def test_paraphrase_requires_review_instead_of_assumed_semantic_equivalence(self): + self.write(self.base) + self.write(self.head, text=mdc(body="Always indent with spaces.\n")) + self.assertEqual(self.invoke(self.command("--fail-on-lost-directives"))[0], 1) + self.assertEqual(compare_cursor_scope(self.base, self.head, ["src/a"])["summary"]["recognized_active_losses"], 1) + + def test_addition_and_empty_profile_do_not_invent_a_baseline_guarantee(self): + code, stdout, _ = self.invoke(self.command("--fail-on-lost-directives", "--fail-on-unknown")) + self.assertEqual(code, 0) + self.assertIn("no recognized active directives", stdout) + self.write(self.head) + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(result["summary"]["baseline_active_directives"], 0) + self.assertEqual(result["summary"]["recognized_active_gains"], 1) + + def test_line_shifts_and_format_changes_preserve_recognition_with_original_evidence(self): + self.write(self.base) + self.write(self.head, text=mdc(body="\n# Policy\nNever use tabs.\n")) + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + change = result["directive_changes"][0] + self.assertEqual((change["base"]["sources"][0]["line"], change["head"]["sources"][0]["line"]), (5, 7)) + self.assertEqual(change["transition"], "active_to_active") + self.assertEqual(result["summary"]["recognized_active_losses"], 0) + + def test_each_source_is_captured_once_for_both_attachment_and_directive_evidence(self): + sources = {self.write(self.base), self.write(self.head, text=mdc("docs/**"))} + reads = {path: 0 for path in sources} + original = Path.read_text + + def once(path, *args, **kwargs): + if path in sources: + reads[path] += 1 + self.assertEqual(reads[path], 1, "Source reopened after capture") + return original(path, *args, **kwargs) + + with patch.object(Path, "read_text", once): + result = compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(set(reads.values()), {1}) + self.assertEqual(result["summary"]["recognized_active_losses"], 1) + + def test_reports_omit_raw_metadata_and_directives_and_are_deterministic(self): + for root, pattern in ((self.base, "src/**"), (self.head, "private-pattern/**")): + self.write(root, "z.mdc", mdc(pattern, "Never expose private-instruction-value.\n")) + self.write(root, "a.mdc", mdc(pattern, "Always use private-format-value.\n")) + first = compare_cursor_scope(self.base, self.head, ["src/b", "./src/a", "src/a"]) + second = compare_cursor_scope(self.base, self.head, ["src/a", "src/b"]) + self.assertEqual(first, second) + encoded = json.dumps(first) + for literal in ("private-pattern", "private-instruction-value", "private-format-value", str(self.root)): + self.assertNotIn(literal, encoded) + self.assertEqual([row["file"] for row in first["rule_changes"]], [".cursor/rules/a.mdc", ".cursor/rules/z.mdc"]) + + def test_cli_is_advisory_by_default_and_strict_only_when_requested(self): + self.write(self.base) + self.write(self.head, text=mdc("docs/**")) + code, stdout, stderr = self.invoke(self.command("--json")) + self.assertEqual((code, stderr), (0, "")) + self.assertFalse(json.loads(stdout)["review_gate"]["failed"]) + code, stdout, stderr = self.invoke(self.command("--fail-on-lost-directives", "--json")) + self.assertEqual((code, stderr), (1, "")) + self.assertTrue(json.loads(stdout)["review_gate"]["failed"]) + code, stdout, _ = self.invoke(self.command("--fail-on-lost-directives")) + self.assertEqual(code, 1) + self.assertIn("metadata lines [globs:2", stdout) + self.assertIn("policy.mdc:5", stdout) + + def test_candidate_policy_does_not_change_explicit_scope_gate_and_ignore_removal_is_visible(self): + self.write(self.base) + self.write(self.head) + (self.head / ".agentconfigscoreignore").write_text(".cursor/rules/policy.mdc\n", encoding="utf-8") + (self.head / ".agentconfigscore.json").write_text('{"policy":{"fail_on_new_errors":false,"fail_under":0}}', encoding="utf-8") + code, stdout, stderr = self.invoke(self.command("--fail-on-lost-directives", "--json")) + self.assertEqual((code, stderr), (1, "")) + result = json.loads(stdout) + self.assertEqual(result["summary"]["recognized_active_losses"], 1) + self.assertEqual(result["directive_changes"][0]["transition"], "active_to_not_recognized") + + def test_invalid_paths_fail_before_any_snapshot_or_git_worktree(self): + with patch("agent_config_score.scanner._cursor_scope_snapshot", side_effect=AssertionError("snapshot read")): + with self.assertRaises(ValueError): + compare_cursor_scope(self.base, self.head, ["../outside"]) + with patch("agent_config_score.cli.repository_root", side_effect=AssertionError("Git lookup")): + code, stdout, _ = self.invoke(["scope", "diff", "HEAD", "--target", "../outside", "--json"]) + self.assertEqual((code, stdout), (2, "")) + + def test_bad_sources_in_either_tree_do_not_emit_partial_success(self): + for root in (self.base, self.head): + with self.subTest(side=root.name): + self.write(self.base) + self.write(self.head) + source = self.write(root) + source.write_bytes(b"private-source-detail-\xff") + code, stdout, stderr = self.invoke(self.command("--json")) + self.assertEqual((code, stdout), (2, "")) + self.assertNotIn("private-source-detail", stderr) + code, stdout, _ = self.invoke(["scope", "compare", str(self.root / "absent"), str(self.head), "--target", "src/a", "--json"]) + self.assertEqual((code, stdout), (2, "")) + + def test_runtime_identity_change_aborts_instead_of_comparing_different_tools(self): + with patch("agent_config_score.scanner.capture_tool_identity", side_effect=[ToolIdentity("0.24.0", "a" * 64), + ToolIdentity("0.24.0", "b" * 64)]): + with self.assertRaises(OSError) as raised: + compare_cursor_scope(self.base, self.head, ["src/a"]) + self.assertEqual(raised.exception.errno, errno.EAGAIN) + + def test_git_working_tree_comparison_retains_commit_and_cleans_up_on_success_and_error(self): + def git(*args): + process = subprocess.run(["git", "-C", str(self.base), *args], capture_output=True, text=True, encoding="utf-8") + self.assertEqual(process.returncode, 0, process.stderr) + return process.stdout.strip() + + git("init", "-q", "-b", "main", "--template=") + hooks = self.root / "empty-hooks" + hooks.mkdir() + for key, value in (("core.hooksPath", str(hooks)), ("user.name", "Scope Test"), + ("user.email", "scope@example.invalid"), ("commit.gpgsign", "false")): + git("config", key, value) + source = self.write(self.base) + git("add", ".") + git("-c", "core.autocrlf=false", "commit", "-qm", "baseline") + sha = git("rev-parse", "HEAD") + source.write_text(mdc("docs/**"), encoding="utf-8") + before = source.read_bytes() + args = ["scope", "diff", "HEAD", "--path", str(self.base), "--target", "src/api/a.py", "--fail-on-lost-directives", "--json"] + code, stdout, stderr = self.invoke(args) + self.assertEqual((code, stderr), (1, "")) + result = json.loads(stdout) + self.assertEqual(result["comparison_inputs"]["base_commit"], sha) + self.assertEqual(result["comparison_inputs"]["head"], "working_tree") + self.assertEqual(source.read_bytes(), before) + self.assertEqual(sum(line.startswith("worktree ") for line in git("worktree", "list", "--porcelain").splitlines()), 1) + auto_args = [arg for index, arg in enumerate(args) if index != 2] + self.assertEqual(json.loads(self.invoke(auto_args)[1])["comparison_inputs"]["base_ref"], "main") + for candidate in (["scope", "diff", "missing-ref", "--path", str(self.base), "--target", "src/a", "--json"], args): + if candidate is args: + source.write_bytes(b"\xff") + code, stdout, _ = self.invoke(candidate) + self.assertEqual((code, stdout), (2, "")) + self.assertEqual(sum(line.startswith("worktree ") for line in git("worktree", "list", "--porcelain").splitlines()), 1) + + +if __name__ == "__main__": + unittest.main() From a8244bd758b82dfc06d8f55a12543527ce5e477c Mon Sep 17 00:00:00 2001 From: Zhibo Lin <147509942+LE0-Lin@users.noreply.github.com> Date: Wed, 4 Nov 2026 21:36:04 +0800 Subject: [PATCH 2/3] docs: pin scope comparison recipe to tested tool source --- docs/cursor-scope-impact.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cursor-scope-impact.md b/docs/cursor-scope-impact.md index 91b4704..8edb175 100644 --- a/docs/cursor-scope-impact.md +++ b/docs/cursor-scope-impact.md @@ -97,7 +97,7 @@ on: [pull_request] permissions: contents: read env: - ACS_TOOL_REVISION: SCOPE_IMPACT_TOOL_COMMIT + ACS_TOOL_REVISION: dd208c9302737f6d61b4f327de87b197479f8d0d jobs: scope: runs-on: ubuntu-latest From 4109cbf66bcb7756c0283d5eed524f6ff6e33145 Mon Sep 17 00:00:00 2001 From: Zhibo Lin <147509942+LE0-Lin@users.noreply.github.com> Date: Wed, 4 Nov 2026 21:51:02 +0800 Subject: [PATCH 3/3] docs: use a shallow pinned checkout in scope CI recipe --- docs/cursor-scope-impact.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/cursor-scope-impact.md b/docs/cursor-scope-impact.md index 8edb175..371f863 100644 --- a/docs/cursor-scope-impact.md +++ b/docs/cursor-scope-impact.md @@ -109,8 +109,16 @@ jobs: - uses: actions/setup-python@v7 with: python-version: "3.12" + - name: Check out the reviewed tool source + uses: actions/checkout@v7 + with: + repository: LE0-Lin/AgentConfigScore + ref: ${{ env.ACS_TOOL_REVISION }} + path: build/agent-config-score-tool + fetch-depth: 1 + persist-credentials: false - name: Install the reviewed tool source - run: python -m pip install "git+https://github.com/LE0-Lin/AgentConfigScore.git@$ACS_TOOL_REVISION" + run: python -m pip install ./build/agent-config-score-tool - name: Review declared API-task context env: BASE_COMMIT: ${{ github.event.pull_request.base.sha }} @@ -125,6 +133,9 @@ from the repository being checked. Installing the published v0.24.0 package or using the rolling `v0` Action does not supply them. Choose task files relevant to your repository and review each declared context; one context is not whole-repository coverage. The workflow uses ordinary read-only PR permissions. +The separate tool checkout sits under `build/`, which this profile skips during +instruction-source discovery. A depth-one tool checkout avoids cloning its full +history; the repository under review still needs its baseline commit available. The profile inherits its bounded metadata/glob/prose grammar, whole-repository supported-source preflight, ignore rules and non-atomic filesystem limits.