docs: separate stable agent contract from living project context

AGENTS.md

@@ -1,3 +1,137 @@
+# Private fork operating contract
+
+This section defines the durable operating contract for the private `bryanwieg/llama.cpp` fork. It takes precedence over the upstream contributor instructions below **only for work that remains in this private fork**. If a change is intended for submission to `ggml-org/llama.cpp`, follow the upstream rules below in full.
+
+## Mission
+
+Maintain a high-performance, technically conservative llama.cpp fork for local inference and development, with particular attention to AMD/ROCm systems.
+
+Optimize for:
+
+- correctness first
+- measurable throughput and latency improvements
+- preservation of model quality and supported capabilities
+- low blast radius and easy rollback
+- understandable mechanisms that a human maintainer can reason about
+- reuse of current upstream infrastructure before adding fork-specific machinery
+- local-first operation without depending on remote compute
+
+Prefer general mechanisms that benefit a class of workloads over model-specific hacks. A narrow optimization is acceptable when it is explicitly gated, measurable, and does not distort the default upstream path.
+
+## Long-term priorities
+
+These priorities are intentionally stable. Short-term work belongs in the living documents under `docs/agents/`.
+
+1. Stay close enough to upstream that useful llama.cpp changes can continue to flow into the fork without heroic rebases.
+2. Preserve numerical correctness, model behavior, vision/OCR support, MTP/speculative behavior, and long-context capability while optimizing performance.
+3. Improve AMD/ROCm performance through evidence-driven allocator, graph, kernel, memory-layout, and scheduling work.
+4. Prefer reversible, opt-in experiments until repeated measurements justify making behavior broader or default.
+5. Keep the fork maintainable: avoid duplicate subsystems, speculative abstractions, and patches that solve one benchmark by making the architecture harder to understand.
+6. Build reusable infrastructure that can support future dense -> specialist -> sparse-MoE experimentation without coupling the fork to one model checkpoint.
+7. Treat benchmark methodology, provenance, and reproducibility as part of the implementation, not as optional notes.
+
+## Agent skills
+
+Repository-local agent plugins are first-class project tooling. Do not imitate a skill from memory when the actual skill is available; invoke or read the skill and follow its workflow.
+
+The current plugin architecture and update procedures are documented in:
+
+- [Matt Pocock skills integration](docs/agents/mattpocock-skills.md)
+- [Ponytail integration](docs/agents/ponytail.md)
+
+If the local plugins are not installed, use the repository setup scripts rather than creating an ad-hoc installation.
+
+### Issue tracker
+
+Project issues/specs live in the private fork's GitHub Issues. See [docs/agents/issue-tracker.md](docs/agents/issue-tracker.md).
+
+For work intended for upstream `ggml-org/llama.cpp`, the upstream contribution rules later in this file still apply and take precedence over private-fork automation conventions.
+
+### Triage labels
+
+Use the default Matt Pocock triage vocabulary for this private fork: `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, and `wontfix`. See [docs/agents/triage-labels.md](docs/agents/triage-labels.md).
+
+### Domain docs
+
+Use a **single-context** domain-doc layout. Read root `CONTEXT.md` when it exists and relevant ADRs under `docs/adr/`; create or evolve them lazily through the domain-modeling workflow rather than scaffolding empty documents. See [docs/agents/domain.md](docs/agents/domain.md).
+
+### Skill routing
+
+Use the narrowest applicable workflow.
+
+- **Ponytail** is the default implementation discipline. Its normal `full` mode should remain active unless the user changes it. It governs solution size, YAGNI, reuse, and avoiding needless complexity. Do not duplicate its rules here.
+- For non-trivial engineering work, use **`ask-matt`** when the best workflow is unclear.
+- Use **`grill-with-docs`** before a consequential design change when requirements, boundaries, or terminology are unresolved.
+- Use **`diagnosing-bugs`** for regressions, intermittent failures, difficult performance anomalies, or bugs where the root cause is not already demonstrated.
+- Use **`research`** when a decision depends on primary-source technical evidence rather than local code alone.
+- Use **`tdd`** when behavior can be captured by a tight, useful red/green loop. Do not manufacture broad test infrastructure merely to satisfy a ritual.
+- Use **`codebase-design`** or **`domain-modeling`** when the hard part is architectural placement, interfaces, ownership, or vocabulary.
+- Use **`to-spec`** and **`to-tickets`** for work that will span multiple sessions or needs an explicit dependency graph; use **`implement`** to execute that plan.
+- Use **`resolving-merge-conflicts`** for non-trivial merge or rebase conflicts; resolve intent, not just text.
+- Before finalizing a substantive code change, run **`code-review`** for correctness/spec conformance and **`ponytail-review`** as a separate complexity/over-engineering pass.
+
+Skills complement project constraints; they do not override them.
+
+## Engineering method
+
+Before changing code:
+
+1. Read the relevant implementation and call paths.
+2. Read the applicable living agent docs listed below.
+3. Establish the baseline, failure mode, or measurable success criterion.
+4. Check whether current upstream already contains the fix or a superseding design before porting historical fork work.
+
+When implementing:
+
+- Prefer semantic adaptation to current upstream over replaying old commits mechanically.
+- Keep unrelated allocator, graph, kernel, model, and tooling changes separate.
+- Gate risky performance experiments behind explicit opt-in controls until evidence supports broader use.
+- Preserve fallback behavior and backend capability checks where an optimization is backend-specific.
+- Do not trade model quality, correctness, or supported features for tokens/sec unless the user explicitly chooses that tradeoff.
+- Minimize blast radius. The smallest **correct mechanism** wins, not the smallest diff in isolation.
+- Remove temporary diagnostics, tracing, and investigation scaffolding before considering work complete unless they have durable operational value.
+
+## Performance evidence
+
+Performance claims require measurements on representative hardware and workload.
+
+Prefer:
+
+- same binary for ON/OFF comparisons where possible
+- repeated runs rather than one-shot numbers
+- ABBA or similarly drift-resistant ordering
+- separate prompt/prefill latency from decode throughput
+- recording context size, batch/ubatch, quantization, KV precision, MTP/speculation settings, and relevant environment gates
+- checking output/correctness and MTP proposed/accepted behavior in addition to speed
+
+A compile success is not performance validation.
+
+## Repository actions and CI
+
+For this private fork, the user may explicitly authorize agents to create branches, commits, and pull requests. Such authorization applies only to this fork and does not relax the upstream contribution rules below.
+
+Keep CI deliberate and economical:
+
+- no broad automatic workflow matrices
+- no automatic CI merely because a PR was opened or updated
+- no automatic CI on merge/push unless the user changes this policy
+- prefer manual, targeted Windows/Linux x64 builds relevant to the AMD/ROCm workload
+- do not spend CI minutes on unrelated CUDA, ARM, macOS, CANN, SYCL, Android, RISC-V, Snapdragon, packaging, or UI matrices unless a task specifically requires them
+
+## Living project context
+
+Do not turn transient facts into permanent instructions. Load these documents as needed and update them when reality changes:
+
+- [Current state and priorities](docs/agents/current-state.md) - active direction, near-term work, and current assumptions
+- [Technical context](docs/agents/technical-context.md) - hardware, build/runtime environment, model/workload details
+- [Optimization history](docs/agents/optimization-history.md) - ported optimizations, provenance, evidence, and deferred/superseded work
+- [Matt Pocock skills integration](docs/agents/mattpocock-skills.md) - plugin packaging and update procedure
+- [Ponytail integration](docs/agents/ponytail.md) - plugin behavior, hook trust, and update procedure
+
+If a fact is likely to change with a new model, GPU, branch, benchmark, upstream release, or experiment, it belongs in one of those living documents rather than in this contract.
+
+---
+
 # Instructions for llama.cpp
 
 > [!IMPORTANT]

docs/agents/current-state.md

@@ -0,0 +1,48 @@
+# Current state and priorities
+
+This is the short-horizon project brief. It is expected to change often.
+
+## Current direction
+
+The project is centered on the existing dense Qwen3.8/Qwen35-style ~27B workflow and on building reusable optimization/integration infrastructure around it.
+
+The current strategic direction is **not** to pursue the earlier GLM compression path or the Qwen3.8-Flash-Next shrinking effort unless the user explicitly reopens those directions.
+
+The longer-term model-engineering direction remains:
+
+`dense base -> specialists/adapters -> sparse routing/MoE experimentation`
+
+The intent is to improve both capability and throughput while preserving vision/OCR, MTP, and long-context behavior.
+
+## Near-term priorities
+
+1. Keep the private fork rebased conceptually on current llama.cpp rather than accumulating historical ROCmFPX architecture.
+2. Validate already-ported RX 9070 optimizations on the real Windows/HIP machine.
+3. Continue auditing historical donor commits one concept at a time and port only ideas that current upstream has not superseded.
+4. Improve performance without dropping below the IQ4_XS quality floor for production artifacts.
+5. Distinguish TTFT/prefill improvements from decode-throughput improvements.
+6. Keep experimental paths opt-in and easy to remove if measurements do not justify them.
+7. Preserve a clean agent/tooling environment through the repository-local Ponytail and Matt Pocock plugin integrations.
+
+## Current benchmark expectations
+
+Historical quality-focused dense runs have generally been in the mid-30 tokens/sec range, while some ROCm-native/faster model variants have approached roughly 40 tokens/sec with weaker reasoning quality.
+
+The long-term aspiration remains materially higher throughput, including the earlier ~60 tokens/sec target, but treat that as a goal rather than a current baseline or guaranteed outcome.
+
+Always replace these rough expectations with fresh measurements when evaluating a code change.
+
+## Working branches and integration
+
+Use `rx9070-port` as the private integration branch unless the user changes the branch strategy.
+
+Optimization work should normally land through focused feature branches/PRs so individual concepts remain reviewable and reversible.
+
+## Agent plugins
+
+The repository now uses a local Codex marketplace for both:
+
+- Ponytail
+- Matt Pocock engineering skills
+
+See the dedicated integration docs for install/update details. Do not recreate separate ad-hoc copies under `.agents/skills/`.

docs/agents/domain.md

@@ -0,0 +1,52 @@
+# Domain Docs
+
+How engineering skills should consume project knowledge when exploring this fork.
+
+## Layout
+
+This repository uses a **single-context** domain model:
+
+```text
+/
+├── CONTEXT.md              # created/evolved lazily when domain vocabulary becomes useful
+├── docs/
+│   ├── adr/                # architectural decisions, created as real decisions are made
+│   └── agents/             # operational/living agent context
+└── ...
+```
+
+Do not create empty `CONTEXT.md` or ADR files merely to satisfy the layout. The `domain-modeling`, `grill-with-docs`, and architecture workflows should create or update them when terminology or durable decisions actually need to be recorded.
+
+## Before exploring
+
+Read, when present and relevant:
+
+- root `CONTEXT.md`
+- ADRs under `docs/adr/`
+- the applicable living operational docs under `docs/agents/`
+
+The roles are different:
+
+- `CONTEXT.md` defines shared domain language and durable conceptual relationships.
+- `docs/adr/` records durable architectural decisions and their rationale.
+- `docs/agents/` records operational context that can change more frequently: current priorities, hardware/runtime assumptions, optimization history, issue-tracker configuration, and plugin/update procedures.
+- root `AGENTS.md` is the stable project operating contract.
+
+## Use the project's vocabulary
+
+When an output names a domain concept, prefer terminology already defined in `CONTEXT.md`.
+
+If an important concept has no established term, that may be a real domain-modeling gap. Resolve it through `domain-modeling` rather than inventing multiple competing names.
+
+## ADR conflicts
+
+If proposed work conflicts with an existing ADR, surface that explicitly. Do not silently override a durable architectural decision.
+
+## What belongs where
+
+Use this test:
+
+- enduring mission, constraints, engineering behavior -> root `AGENTS.md`
+- stable domain vocabulary -> `CONTEXT.md`
+- durable architectural decision -> `docs/adr/`
+- changing project state/history/tooling context -> `docs/agents/`

docs/agents/issue-tracker.md

@@ -0,0 +1,45 @@
+# Issue tracker: GitHub
+
+Issues and specs for this private fork live as GitHub issues in `bryanwieg/llama.cpp`.
+
+Use the repository's normal GitHub tooling for issue operations. The Matt Pocock engineering skills may create, read, label, comment on, and close issues in this private fork when the user authorizes that workflow.
+
+## Conventions
+
+- **Create an issue**: `gh issue create --title "..." --body "..."`. Use a heredoc for multi-line bodies.
+- **Read an issue**: `gh issue view <number> --comments`, including labels.
+- **List issues**: `gh issue list --state open --json number,title,body,labels,comments` with appropriate filters.
+- **Comment on an issue**: `gh issue comment <number> --body "..."`
+- **Apply/remove labels**: `gh issue edit <number> --add-label "..."` / `--remove-label "..."`
+- **Close**: `gh issue close <number> --comment "..."`
+
+Infer the repo from the local Git remote where practical.
+
+## Pull requests as a triage surface
+
+**PRs as a request surface: no.**
+
+Do not treat pull requests as ordinary triage tickets unless this flag is deliberately changed later.
+
+## When a skill says "publish to the issue tracker"
+
+Create a GitHub issue in the private fork.
+
+## When a skill says "fetch the relevant ticket"
+
+Read the corresponding GitHub issue and its comments/labels.
+
+## Wayfinding operations
+
+Used by `wayfinder`.
+
+- **Map**: one GitHub issue labelled `wayfinder:map`, containing Notes / Decisions-so-far / Fog.
+- **Child ticket**: a GitHub sub-issue where available; otherwise a task-list child that names its parent map.
+- **Blocking**: prefer GitHub native issue dependencies. If unavailable, use a `Blocked by: #n, #n` line.
+- **Frontier**: open, unassigned child tickets whose blockers are closed.
+- **Claim**: assign the selected ticket before beginning work.
+- **Resolve**: record the resolution, close the ticket, and append a concise context pointer to the map's Decisions-so-far.
+
+## Upstream boundary
+
+This document describes the private fork's tracker only. It does not authorize autonomous issue/PR/comment activity against `ggml-org/llama.cpp`. For upstream contribution work, follow the upstream instructions in root `AGENTS.md`.

docs/agents/optimization-history.md

@@ -0,0 +1,111 @@
+# Optimization history
+
+This is a living record of private-fork optimization work and historical donor analysis. It is not a permanent instruction set.
+
+## Source repositories
+
+Primary upstream:
+
+- `ggml-org/llama.cpp`
+
+Private fork:
+
+- `bryanwieg/llama.cpp`
+
+Historical donor/archive:
+
+- `bryanwieg/ROCmFPX-rx9070-archive`
+
+Historical donor commits are evidence and implementation references, not merge targets. Audit current upstream before porting any concept.
+
+## Ported optimization concepts
+
+### Scheduler scratch arena trim
+
+Historical source:
+
+`7b08993509fd7718dd18e8c8ed839ddcc0d8874a`
+
+Private-fork behavior is opt-in through `GGML_ROCM_ARENA_TRIM`.
+
+Historical evidence showed large improvements on selected short/long Qwen workloads, but treat those numbers as historical rather than a current baseline.
+
+A later current-API compile correction removed a stale `src_ctx` gate and fixed a `std::max` type mismatch without changing intended behavior.
+
+### Prompt/prefill reserve
+
+Historical source:
+
+`69a36a358b152e6dbde9e6410ce1b7f2808ab3a7`
+
+Gate:
+
+`GGML_ROCM_PREFILL_RESERVE=1`
+
+Purpose: pre-reserve the prompt graph for the full ubatch when the arena is small enough, primarily reducing TTFT/prefill cost rather than decode cost.
+
+### Recurrent MTP2 state reserve
+
+Historical source:
+
+`6b8a2a0caccdc829c5c094337bd7ace2e65242b8`
+
+Gate:
+
+`GGML_ROCM_HOT_BUFFER_RESERVE=recurrent-mtp2`
+
+Purpose: influence first-touch/allocation placement for the recurrent-state allocation on the target Windows/HIP workload.
+
+### Indexed GDN recurrent-state access
+
+Historical source:
+
+`21edd46f773765db02bbae1d480bc68d1ed97115`
+
+Gate:
+
+`GGML_ROCM_GDN_INDEXED_STATE=single-v1`
+
+Purpose: avoid a state-row gather by using indexed GDN recurrent-state bank access under a narrowly validated geometry/backend configuration.
+
+This path includes capability/fallback handling and should remain narrow until stronger end-to-end evidence supports expansion.
+
+## Deferred experimental concept
+
+### Direct-write GDN fusion
+
+Historical source:
+
+`6f688e7af125bf2b512160c9683b22e81d373038`
+
+This remains reference/WIP material. It changes graph recognition/capture and direct state writeback behavior and has a substantially larger correctness/concurrency blast radius than the indexed-read optimization.
+
+Do not port it merely because it compiles. Treat it as a fresh optimization experiment requiring isolated correctness and performance evidence.
+
+## Superseded or rejected historical work
+
+The following historical concepts should not be replayed without new evidence:
+
+- old MTP verification-row fix: current upstream MTP implementation superseded the historical failure mode
+- old M-RoPE MTP draft-positioning fix: current upstream has newer draft position handling
+- old vision/MTP boundary-crash workaround: historical implementation no longer maps cleanly to the current speculative path
+- diagnostics-only commits
+- gfx1151/RDNA3.5 MMQ geometry work for the RX 9070 target
+- old HIP fast-math removal already addressed upstream
+- speculative replay metrics already upstream/superseded
+- old Qwen4 port superseded by current upstream model support
+
+The historical prompt-cache rollback series is high blast radius and should be reconsidered only if current upstream demonstrates a reproducible problem that maps to the old issue.
+
+## Benchmark discipline for historical ports
+
+When revisiting any donor optimization:
+
+1. Identify the exact historical source commit.
+2. Explain the mechanism, not just the diff.
+3. Check current upstream for equivalent or superseding behavior.
+4. Port semantically to current APIs.
+5. Keep the experiment isolated and opt-in.
+6. Compare ON/OFF on the actual target hardware.
+7. Record TTFT, decode throughput, correctness/output, MTP counts, context, quant, KV settings, and relevant flags.
+8. Remove investigation-only scaffolding before merge.