Flightpath

Explain customer-visible product silence by deterministically replaying a pilot contract against an append-only evidence log.

Same support ticket — “the bot didn’t review our PR” — three different causal classes:

Case	Result	Meaning
silent-draft	EXPECTED_SKIP	Silence is correct (draft PR)
silent-not-enabled	EXPECTED_SKIP	Silence is correct (repo not enabled)
silent-breach	CONTRACT_BREACH	Delivery ok, comment missing — page Eng

SVG version (crisper on some browsers)

pip install -e ".[dev]"
flightpath demo                    # exits 1 (includes a CONTRACT_BREACH case)
flightpath replay cases/silent-draft
flightpath replay cases/silent-breach; echo $?   # 1
flightpath replay cases/silent-breach -o out/explanation.json

Further reading: Engineering philosophy · Interview guide · Changelog

---

Why this exists

Enterprise code-review pilots are event-driven. Webhooks fire. Filters apply. Obligations discharge — or they don’t. Today, “silent PR” tickets stay tribal: Was it a draft? Was the repo enabled? Did GitHub deliver? Did we actually owe a comment?

Forward Deployed Engineers do not lose pilots because a model is slightly worse on a benchmark. They lose them when a customer says “nothing happened” and the room has no shared language for whether that silence was:

1. correct under the pilot contract (expected skip),
2. a real discharge failure (contract breach), or
3. not decidable from the evidence the customer can hold.

Flightpath is an FDE tool for that moment. It does not call private APIs, does not clone a code-review product, and does not guess about internal workers. It folds customer-held evidence through a tiny, explicit pilot contract and emits an explanation. Without a replayable contract, FDEs page the wrong team and customers lose trust.

This is the seed of an in-product “Why didn’t this fire?” surface — not a side-quest startup.

---

Architecture

Five moving parts. No database. No services. No network required.

flowchart LR
  E[evidence.jsonl<br/>append-only log] --> R["replay()"]
  C[contract.yaml<br/>pilot obligations] --> R
  O[observed.yaml<br/>comment present/absent] --> R
  R --> T[terminal / JSON stdout]
  R -.-> X["explanation.json (only with -o)"]

Loading

replay(evidence, contract, observed) -> Explanation

Terminals (exactly three silence classes + OK):

Result	When
EXPECTED_SKIP	Contract says do not owe (draft, repo disabled, …)
CONTRACT_BREACH	Owed ∧ delivery outcome bad or comment missing
INSUFFICIENT_EVIDENCE	Cannot decide without inventing a root cause
OK	Obligation discharged

Exit codes (automation-friendly):

State	Exit
OK / EXPECTED_SKIP	0
CONTRACT_BREACH	1
Tool error (bad path, invalid input)	2
INSUFFICIENT_EVIDENCE	3

Fixtures under cases/ are never mutated. Write an artifact only with -o / --output.

The entire v1 contract fits on one screen — see any cases/*/contract.yaml.

---

Quick start

Requirements: Python ≥ 3.11

git clone https://github.com/adityaaa2306/FlightPath.git
cd FlightPath
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
pytest
flightpath demo

Use -o out/explanation.json when you want an immutable ticket artifact (result, primary reason, trace[], evidence line hashes). Stdout is enough for interactive triage.

---

Case layout

cases/
  silent-draft/
    contract.yaml      # what the pilot promised
    evidence.jsonl     # append-only events (PR + deliveries)
    observed.yaml      # comment: present | absent
  silent-not-enabled/
  silent-breach/

Evidence event types (closed set):

pr.opened · pr.synchronize · pr.ready_for_review · delivery.ok · delivery.fail · repo.meta · clock · note

Redeliveries with the same delivery_id are idempotent (see silent-breach).

---

Design principles

• Local-first — runs on a customer laptop; fixtures need no tokens
• Pure replay — same inputs → same semantic explanation
• Immutable artifacts — optional -o explanation.json is what you attach to the ticket; fixtures stay pristine
• Honest scope — every output is labeled model=pilot_contract_v1, not “production replica”
• No fake distributed systems — webhook at-least-once + obligation discharge; not ECS cosplay

See docs/engineering-philosophy.md for the longer form.

---

Development

pytest -q
python scripts/render_demo.py        # docs/demo.svg
python scripts/render_demo_gif.py    # docs/demo.gif (requires pillow)

Table-driven tests live in tests/test_replay.py: canonical cases, idempotent redelivery, insufficient-evidence paths, golden semantic shapes.

---

What this is not

• Not a code reviewer
• Not a benchmark
• Not a dashboard
• Not a GitHub App
• Not dependent on private Entelligence APIs

It is a deterministic silence classifier an FDE can run in a war room — and a shape a product team can later merge behind “Why Quiet?”

---

License

MIT