diff --git a/docs/query-reference.md b/docs/query-reference.md new file mode 100644 index 00000000..818b9f6c --- /dev/null +++ b/docs/query-reference.md @@ -0,0 +1,195 @@ +# Query reference: filters, sorting, pagination, and field overrides + +This page documents the arguments Flatbread's GraphQL read interface accepts, +plus the two config options that shape the generated schema: per-collection +`overrides` and the global `fieldNameTransform`. Every claim here matches the +current implementation in `packages/core`. + +## Which queries take which arguments + +For a collection named `Post`, the generated schema exposes two top-level +queries: + +| Query | Arguments | +| -------------- | -------------------------------------------- | +| `allPosts` | `filter`, `sortBy`, `order`, `skip`, `limit` | +| `Post(id: ID)` | `id` | + +Only the `all*` list queries accept `filter`. A third resolver — find many by +IDs, with `ids`, `sortBy`, `order`, `skip`, and `limit` but no `filter` — +backs list-valued relation fields such as `Post.authors`; it is not mounted +as its own top-level query. + +## Order of application + +For a list query, Flatbread applies the arguments in this order: + +1. `filter` narrows the collection. +2. `sortBy` sorts the surviving records by one field. +3. `order: DESC` reverses the sorted list (`ASC` is the default). +4. `skip` and `limit` slice the result (see the slice caveat under + [`skip` and `limit`](#skip-and-limit)). + +## `filter` + +`filter` takes a JSON object whose shape mirrors the path to the value you +want to compare. The deepest key that does not hold another object names the +comparison operation; its value is the target to compare against. The syntax +follows a subset of MongoDB's query style. + +```graphql +query HighlyRated { + allPosts(filter: { rating: { gt: 80 } }) { + id + title + rating + } +} +``` + +That filter keeps every post whose `rating` field is greater than 80. Nested +paths work the same way: `{ postMeta: { rating: { gt: 80 } } }` compares +`postMeta.rating` on each record. + +### Combining filters: peer paths AND together + +Peer keys inside one filter object must **all** match. This is a logical AND, +not a union: + +```graphql +query FilteredPosts { + allPosts(filter: { title: { wildcard: "*tion" }, rating: { gt: 80 } }) { + title + } +} +``` + +A post appears in the result only when its title ends in `tion` **and** its +rating exceeds 80. There is no OR combinator across paths; run two queries or +use `in` on one field when you need one. + +### The 14 operations + +| Operation | Meaning | Notes | +| ------------------------ | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| `eq` | `fieldValue === target` | Strict equality. | +| `ne` | `fieldValue !== target` | Strict inequality. | +| `lt`, `lte`, `gt`, `gte` | `<`, `<=`, `>`, `>=` | Both sides must be a string, number, or boolean; anything else compares as `false` (the record is excluded). | +| `in` | `target.includes(fieldValue)` | Target must be an array; throws otherwise. | +| `nin` | `!target.includes(fieldValue)` | Target must be an array; throws otherwise. | +| `includes` | field contains the target | Field must be an array or string; throws otherwise. String matching follows `String.prototype.includes`. | +| `excludes` | field does not contain the target | Same field requirements as `includes`. | +| `exists` | `target ? field != undefined : field == undefined` | Loose check: treats `null` and `undefined` alike. | +| `strictlyExists` | `target ? field !== undefined : field === undefined` | Strict check: `null` counts as existing. | +| `regex` | `target.test(field)` | See the `regex` limits below. | +| `wildcard` | loose string matching | Case-insensitive; accepts one pattern string or an array of patterns; uses the [matcher](https://github.com/sindresorhus/matcher) API. | + +For `regex` and `wildcard`, a string-array field matches when **any** element +matches. + +When the filter path is exactly the top-level `id` field, Flatbread +normalizes both sides for `eq`, `ne`, `in`, and `nin`, so IDs compare +consistently with how records are keyed. + +### `regex` limits + +The `regex` implementation requires an actual JavaScript `RegExp` object as +the target value and throws otherwise. Standard GraphQL JSON variables cannot +carry a `RegExp` instance, so `regex` only works where the caller constructs +the filter in JavaScript and can pass a real `RegExp` — it is not usable from +a plain GraphQL document with JSON variables. Use `wildcard` for loose +matching from GraphQL clients. + +### Dates + +Filters cannot infer date strings and compare them as `Date` values. A `Date` +object passed programmatically may work but is not extensively tested. To fix +this properly, add type checks and comparators in +`packages/core/src/utils/sift.ts` and open a pull request. + +## `sortBy` and `order` + +`sortBy` accepts one root-level field name and sorts ascending by default. +Records whose field values are not sortable against each other (not both +strings, numbers, or booleans) keep their relative order. `order` accepts +`ASC` or `DESC`; `DESC` reverses the list after sorting. + +## `skip` and `limit` + +`skip` drops the first `n` records. `limit` bounds the result. Both accept +integers. + +One implementation caveat matters when you combine them: the result is +computed as `records.slice(skip, limit)`, so **when `skip` is set, `limit` +acts as the slice end index, not a page size**. For example, `skip: 10, limit: 15` returns records 11 through 15 (five records), and `skip: 10, limit: 5` returns nothing. Without `skip`, `limit: 5` returns the first five +records as you would expect. Account for this when paging: to fetch a page of +`n` records after skipping `s`, pass `limit: s + n`. + +## Field overrides + +Overrides define a custom GraphQL type or resolver on top of a field in one +collection — for example to +[optimize images](https://github.com/FlatbreadLabs/flatbread/tree/main/packages/resolver-svimg) +or reshape a value at read time. + +`overrides` is an array on each **content entry**, not a global config key: + +```js +export default defineConfig({ + source: sourceFilesystem(), + transformer: transformerMarkdown(), + content: [ + { + path: 'content/markdown/authors', + collection: 'Author', + overrides: [ + { + // The source field name. + field: 'name', + // The GraphQL type to expose. + type: 'String', + // Transform the value before returning it. + resolve: (name) => String(name).toUpperCase(), + }, + ], + }, + ], +}); +``` + +Each override takes `field`, `type`, `resolve`, and optional `args` and +`description`. The `resolve` function receives the raw field value plus an +object with `source`, `context`, and `args`. + +### Supported `field` path syntax + +| Pattern | Meaning | +| ----------------------- | ------------------------------------------------------ | +| `nested.object` | A field inside a nested object | +| `an.array[]` | Map over every element of an array field | +| `an.array[]with.object` | Map over an array and reach into each element's object | + +The Next.js example config +([`examples/nextjs/flatbread.config.js`](https://github.com/FlatbreadLabs/flatbread/blob/main/examples/nextjs/flatbread.config.js)) +exercises all three patterns on its `OverrideTest` collection and uses +`createSvImgField` from `@flatbread/resolver-svimg` on the `Author` +collection. + +## `fieldNameTransform` + +`fieldNameTransform` is a top-level config function that rewrites every field +name before schema generation. The default removes spaces by capitalizing the +letter that follows each one (`date joined` becomes `dateJoined`); it changes +nothing else. Override it when you need a different global naming rule: + +```js +export default defineConfig({ + // Replace all spaces in field names with an underscore. + fieldNameTransform: (fieldName) => fieldName.replace(/\s/g, '_'), + // ... +}); +``` + +The transform applies to schema fields, override paths, and `refs` lookups +consistently, so a `refs` key declared with the raw field name still resolves +after transformation. diff --git a/packages/flatbread/README.md b/packages/flatbread/README.md index 85514897..912a6a85 100644 --- a/packages/flatbread/README.md +++ b/packages/flatbread/README.md @@ -16,233 +16,168 @@

-Flatbread turns files in Git into a typed relational graph. Files are the -records. `refs` in `flatbread.config.js` link them. You read the graph through -**[GraphQL](https://graphql.org/)**, generated TypeScript, or the `flatbread` -CLI. - -People use it two ways. - -**Durable memory for coding agents.** -[Proof](https://github.com/FlatbreadLabs/flatbread/tree/main/packages/proof) -stores an agent's reasoning as markdown records in the repository: Efforts, -Issues, Findings, Decisions, Constraints, Risks, Citations, and Blobs. An agent -writes them with `flatbread proof write` and reads them back through bounded -queries such as `flatbread proof list` and -`flatbread proof blocking-decisions`. Records live under -`.flatbread-proof/`, so you commit, review, diff, and revert them like any -other file, and the memory outlives the session that produced it. The bundled +Flatbread turns files in Git into a typed relational graph. Each Markdown or +YAML file becomes a record in a named collection, and `refs` in +`flatbread.config.js` link records to each other by ID. Your files stay the +source of truth, with normal Git branches, reviews, and history. GraphQL is +one read interface over that graph, not the whole product: apps can also read +it through generated TypeScript, and coding agents read it through bounded +CLI commands. + +## Choose your path + +People come to Flatbread for two reasons. Pick the one that matches yours. + +| Your goal | Start here | +| ------------------------------------------------------------------------------- | ----------------------------------------------------------------- | +| Give coding agents durable, reviewable project memory | [Path 1: Proof](#path-1-durable-memory-for-coding-agents) | +| Build a site, documentation system, or internal tool from related content files | [Path 2: relational content](#path-2-relational-content-for-apps) | + +Both paths run on the same engine: files become records, configured `refs` +become relations, and Flatbread validates the graph before exposing a read +interface. Every published package requires Node 20.19 or newer. + +## Path 1: durable memory for coding agents + +[Proof](https://github.com/FlatbreadLabs/flatbread/blob/main/packages/proof/README.md) +keeps coding agents and the people they work with aligned. An agent records +durable Issues, Findings, Decisions, Constraints, and Risks under an Effort — +one coherent thread of work. Each record is a Markdown file in your +repository, so the next session and your coworkers read the same reasons, +review them in a pull request, and trace how a choice changed. Nothing lives +in a private chat log or a hosted store. + +The install pins below come from the current +[Proof release manifest](https://github.com/FlatbreadLabs/flatbread/blob/main/packages/proof/skills/proof/release.json). Use the values in that file exactly; do not substitute a +floating branch or guessed version. + +1. Install the Proof skill and the matching `flatbread` package: + + ```bash + npx skills add https://github.com/FlatbreadLabs/flatbread/tree/v1.1.0/packages/proof/skills/proof --skill proof + npm install --save-dev flatbread@1.1.0 + ``` + +2. Add the Proof content model to `flatbread.config.js`, keeping any content + entries you already have: + + ```js + import { + defineConfig, + sourceFilesystem, + transformerMarkdown, + proofContent, + } from 'flatbread'; + + export default defineConfig({ + source: sourceFilesystem(), + transformer: transformerMarkdown(), + content: [ + // Keep existing entries here. + ...proofContent(), + ], + }); + ``` + +3. Keep working state out of Git by adding two lines to `.gitignore`. The + record files themselves stay tracked: + + ```gitignore + **/.flatbread-proof/.journal/ + **/.flatbread/proof/read-cache/ + ``` + +4. Check the setup: + + ```bash + npx flatbread proof bootstrap --verify + ``` + + A complete setup prints one JSON object with `"status":"ready"` and exits + successfully. Bootstrap only inspects the project; it never edits files. + +5. Create and read the first record: + + ```bash + npx flatbread proof write '{"type":"CreateEffort","title":"Choose a search index","body":"Track evidence, constraints, decisions, and open work."}' + npx flatbread proof list --status active + ``` + + The write prints the new record's ID in `artifacts[0].id`. The read prints + a bounded JSON envelope whose `artifact_path` names a Markdown digest. + +The [Proof README](https://github.com/FlatbreadLabs/flatbread/blob/main/packages/proof/README.md) +covers the record model, the write rules, and the session loop an agent +follows. The packaged [Proof skill](https://github.com/FlatbreadLabs/flatbread/blob/main/packages/proof/skills/proof/SKILL.md) -teaches an agent the commands. +teaches an agent those commands and its gate for deciding what deserves +durable memory. -**Relational content for sites, docs, and internal tools.** Markdown and YAML -files become typed collections that link to each other. A post names its -authors by id, and Flatbread resolves them. You get versioned, reviewable -content and joins over files without a CMS database. Start with the -[Quickstart](#quickstart-posts-authors-and-tags). +## Path 2: relational content for apps -Both paths run on the same engine. Plugins control how Flatbread reads files -and turns them into data. +Use this path when Markdown or YAML files need typed links between records. A +post names its authors by ID in frontmatter, and Flatbread resolves those IDs +to `Author` records. You keep normal Git review while gaining validated links +and typed reads — joins over files, without a CMS database. -**Who it is for:** People building coding agents that need memory they can -review in Git, and teams building TypeScript sites, internal tools, and starter -projects that want versioned content with links between entries. +1. Install Flatbread and scaffold a config: -**What Flatbread does not do:** + ```bash + npm install flatbread + npx flatbread init + ``` -- It is not a hosted CMS, dashboard, or writing UI. -- It is not a general-purpose GraphQL platform or database. Transactions, - detailed access control, and many concurrent writers are outside its scope. -- It does not reload its own packages. `flatbread start --watch` picks up valid - content and config changes while you work, but a change to a Flatbread - package needs a rebuild and a restart. See the - [local development loop](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/local-dev-loop.md). - -**GraphQL:** GraphQL is one read interface over the graph. For more detail, see -[Flatbread positioning](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/positioning.md). - -**Glossary:** Definitions for collections, relations, IDs, validation, and the -generated GraphQL types are in the -[glossary](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md). - -**Local development:** Learn what updates automatically and what needs a -restart in the -[local development loop](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/local-dev-loop.md). - -**Export:** The core API can create stable JSON snapshots. See -[JSON export](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/json-export.md). - -**Keeping your data:** Your files, Git history, JSON/CSV exports, GraphQL -introspection, and generated TypeScript remain available when you move away -from Flatbread. See -[data ownership](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/data-ownership.md). + `flatbread init` writes `flatbread.config.js` with `Post` and `Author` + collections and a `refs: { authors: 'Author' }` relation. -Every published package requires Node 20.19 or newer. To work on this monorepo, -use Node 20.19+ with pnpm 10.33.x. - -## Quickstart (posts, authors, and tags) - -Start with the **Next.js example** in `examples/nextjs`. It reads shared -Markdown from `examples/content` through its `content/` symlink. The commands -below use that layout. - -### 1 · What you are modeling - -- **Collections** (`Post`, `Author`) map to folders of files; see the [glossary](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md). -- **Relations:** posts declare `authors:` in frontmatter as a list of **author ids**; Flatbread resolves them through **`refs`** in config (same idea as joins, over files—**not** a remote database). -- **Tags:** in the bundled example, each post exposes **`tags`** as a **YAML string list** in frontmatter. That becomes a **`[String]`** field on **`Post`** in the generated schema. That is **facet-style metadata** repeated per post—not the same machinery as **`refs`** to another collection. If you need normalized tag **records** shared across posts, model a **`Tag`** collection and wire **`refs`** yourself (advanced). - -Illustrative frontmatter: - -```yaml ---- -id: your-post-id -title: Example -authors: - - author-id-one -tags: - - typescript - - content-graph ---- -``` +2. Create `content/markdown/authors/ada.md`: -Markdown **below** the closing `---` is the post body. + ```markdown + --- + id: ada + name: Ada + --- + ``` -### 2 · Content layout (this monorepo) +3. Create `content/markdown/posts/first-post.md`. Markdown below the closing + `---` is the post body: -From the repo root, the markdown that backs the relational story lives here: + ```markdown + --- + id: first-post + title: First post + authors: + - ada + --- -```text -examples/content/markdown/posts/ # Post collection (incl. example-post.md, …) -examples/content/markdown/authors/ # Author collection -``` + Hello from Flatbread. + ``` -The Next example points `flatbread.config.js` at `content/markdown/...` **relative to `examples/nextjs`**, where `content` is the symlink to `../content`. +4. Start the graph server: -**Backing files for posts, authors, and tags (this example):** + ```bash + npx flatbread start --watch + ``` -| What | Where it lives | Glossary terms | -| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Posts** | `examples/content/markdown/posts/*.md` — one **record** per file | [Collection](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#collection), [Record](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#record) | -| **Authors** | `examples/content/markdown/authors/*.md` — one **record** per file | Same; **IDs** in frontmatter wire **relations** | -| **Tags** | The `tags:` YAML list **in each post’s frontmatter** (facet metadata on that **Post**). There is **no** `markdown/tags/` directory here. | [Tag (facet) vs `Tag` collection](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#tag-facet-vs-tag-collection) | + Flatbread prints its GraphQL URL: `http://localhost:5057/graphql`. -### Traceability: same relation model (files, config, query interface) +5. From another terminal, read the relation: -The table below follows one relation from files to config to the generated GraphQL schema. Files and config are the source of truth; GraphQL is one [query interface](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#query-interface) over them. + ```bash + curl http://localhost:5057/graphql \ + -H 'content-type: application/json' \ + --data '{"query":"{ allPosts { id title authors { id name } } }"}' + ``` -| Layer | You see… | Glossary | -| ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Files** | `authors:` ids in a post file match `id:` in author files; `tags:` is a string list on the post | [Relation](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#relation), [ID](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#id), [Tag (facet) vs `Tag` collection](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#tag-facet-vs-tag-collection) | -| **`flatbread.config.js`** | `content` entries with `collection: 'Post' \| 'Author'` and `refs: { authors: 'Author' }` | [Collection](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#collection), [Relation](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#relation) | -| **Generated GraphQL schema + codegen TS** | `allPosts { tags authors { id name } }` — **refs** resolve to **`Author`** objects; **`tags`** stays a scalar list on **`Post`** | [Generated schema and operation types (GraphQL)](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#generated-schema-and-operation-types-graphql), [Cardinality](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md#cardinality) | + The result contains `first-post` with its resolved author + `{ "id": "ada", "name": "Ada" }`. The files and config define that + relation; GraphQL only reads it. -**Illustrative query result** (same **relation model** as [`examples/content/markdown/posts/example-post.md`](https://github.com/FlatbreadLabs/flatbread/blob/main/examples/content/markdown/posts/example-post.md): authors `2a3e` / `40s3`, **tags** from frontmatter). Values are from that file and its resolved **authors**; the shape matches the **`GetPostsAuthorsAndTags`** operation in **§3** after you include **`tags`** and **`authors`** in your **`.graphql`** document (see also `queries/posts.graphql`, which you can extend the same way): +To run Flatbread beside your framework, wrap your dev and build scripts with +`flatbread start`. Everything after `--` passes through to your command. +There is no `flatbread dev` subcommand. ```json -{ - "allPosts": [ - { - "id": "sdfsdf-23423-sdfsd-23444-dfghf", - "title": "The Art of Measuring Cats in Fruit Units", - "tags": ["cats", "measurements", "fruit-science", "important-research"], - "authors": [ - { "id": "2a3e", "name": "Tony" }, - { "id": "40s3", "name": "Eva" } - ] - } - ] -} -``` - -Add **`tags`** (and any other fields) to your **`.graphql`** documents and rerun codegen so operations and `generated/graphql.ts` stay aligned with the files—snippets in docs are **illustrative** until your checked-in queries match. - -### 3 · Run it from the repo root - -Prerequisites: **Node 20.19+** and **pnpm 10.33.x**. See -[CONTRIBUTING.md](https://github.com/FlatbreadLabs/flatbread/blob/main/CONTRIBUTING.md). - -```bash -pnpm install -pnpm build -cd examples/nextjs -pnpm exec flatbread codegen --verbose -``` - -That writes **`generated/graphql.ts`**: TypeScript types and typed document nodes for your **`.graphql`** operations (configure globs under `codegen.documents` in `flatbread.config.js`). - -Add a `.graphql` file (see `queries/posts.graphql` in the example), then rerun **`pnpm exec flatbread codegen --verbose`** so the operation reflects **`tags`**, **`authors`**, etc. Illustrative operation you can paste into `queries/`: - -```graphql -query GetPostsAuthorsAndTags { - allPosts(limit: 5) { - id - title - tags - authors { - id - name - } - } - allAuthors { - id - name - } -} -``` - -After codegen, your app imports types from **`./generated/graphql`**. The result shape of that operation is typed, for example **`GetPostsAuthorsAndTagsQuery`**. Relations resolve to **`Author`** objects; **`tags`** stays a string array on **`Post`**, matching the file metadata. That is the same shape as the [illustrative JSON](#traceability-same-relation-model-files-config-query-interface) under **Traceability**. - -The generated file also exposes a prototype **TypeScript read API** derived from the configured content model. In the Next.js example, [`examples/nextjs/lib/read.ts`](https://github.com/FlatbreadLabs/flatbread/blob/main/examples/nextjs/lib/read.ts) wires **`createFlatbreadReadApi()`** to the existing GraphQL fetcher. It reads **posts**, **authors**, and **tags** with a generated default selection, so there is no hand-written GraphQL document at the call site. - -#### Choosing a read interface - -Files come first. They define records, frontmatter fields, IDs, and `refs`; -`flatbread.config.js` tells Flatbread how to group them into typed collections. -**GraphQL** and the generated TypeScript API are two ways for your app to read -the same data. - -Use **GraphQL operations** when your app needs explicit query documents, custom selections, Apollo or another GraphQL client, persisted operations, or direct access to the GraphQL endpoint. Add `.graphql` documents, include fields like **`tags`** and **`authors`**, and rerun codegen so operation types such as **`GetPostsAuthorsAndTagsQuery`** match the posts/authors/tags graph. - -Use the prototype **generated TypeScript read API** when you want collection-shaped helpers instead of a GraphQL document at each call site. It suits plain reads: posts, authors, tags, and resolved relations. The helpers still run through the GraphQL layer, and their selection-string escape hatch is experimental. Both paths read the same typed graph from the same files; GraphQL is the stable lower-level interface. - -Default filesystem + markdown wiring uses the bundled [`source-filesystem`](https://github.com/FlatbreadLabs/flatbread/tree/main/packages/source-filesystem) and [`transformer-markdown`](https://github.com/FlatbreadLabs/flatbread/tree/main/packages/transformer-markdown) plugins (`flatbread` re-exports them). - -### 4 · Minimal relational config (mental model) - -The example’s production config loads extra collections for tests; **the core onboarding shape** is: - -```js -import { defineConfig, transformerMarkdown, sourceFilesystem } from 'flatbread'; - -export default defineConfig({ - source: sourceFilesystem(), - transformer: transformerMarkdown({ - markdown: { gfm: true, externalLinks: true }, - }), - content: [ - { - path: 'content/markdown/posts', - collection: 'Post', - refs: { authors: 'Author' }, - }, - { - path: 'content/markdown/authors', - collection: 'Author', - refs: { friend: 'Author' }, - }, - ], -}); -``` - -### 5 · Reading the graph: GraphQL (after the model exists) - -Flatbread builds a content graph from files. GraphQL is one read interface over that graph: a generated schema and the resolvers behind it. - -Wire your framework so the CLI wraps dev/build (**`flatbread start`** passes through your command after **`--`**). There is **no** `flatbread dev` subcommand. - -```js -// package.json scripts (adapt the part after `--` to your framework) { "scripts": { "dev": "flatbread start --watch -- next dev --turbopack", @@ -251,237 +186,64 @@ Wire your framework so the CLI wraps dev/build (**`flatbread start`** passes thr } ``` -In the Next.js example, **`pnpm dev`** starts Next and starts -Flatbread in watch mode. The GraphQL endpoint is -**`http://localhost:5057/graphql`** and the Next app is on **`3000`**. -**`pnpm start`** runs production Next without Flatbread. - -```bash -pnpm dev -``` - -When the server starts, Flatbread prints the **`graphql`** URL. Open it to use -Apollo Studio with the generated schema. You can then save queries in -**`.graphql`** files and run **`flatbread codegen`** again. - -With `--watch`, valid content and config changes update the running GraphQL -server. See the -[local development loop](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/local-dev-loop.md) -for the cases that still need a rebuild or restart. - -## Install Flatbread in your own repo - -Outside this monorepo: - -```bash -pnpm add flatbread -``` - -Scaffold **`flatbread.config.js`**: - -```bash -pnpm exec flatbread init -``` - -Point **`content`** entries at **your** `posts/` and **`authors/`** folders, reuse the relational ideas above, and add **`codegen`** in config when you want **`generated/graphql.ts`**. Browse [`packages`](https://github.com/FlatbreadLabs/flatbread/tree/main/packages) for plugins and resolver helpers. - -More detail on the bundled example is in the -[Next.js example README](https://github.com/FlatbreadLabs/flatbread/blob/main/examples/nextjs/README.md). - -For agent memory instead of site content, add `proofContent()` to your -config, then run `flatbread proof bootstrap` to check the setup and -`flatbread proof bootstrap --verify` to fail when something is missing. The -[Proof README](https://github.com/FlatbreadLabs/flatbread/blob/main/packages/proof/README.md) -has the install commands. - -## Query arguments (GraphQL read interface) - -When **GraphQL** is your read interface, list fields use the following arguments in order of application. - -### `filter` - -Every collection in the GraphQL schema takes a `filter` argument that narrows the results. Any leaf field can be used in a filter. - -The syntax for `filter` is based on a subset of [MongoDB's query syntax](https://docs.mongodb.com/manual/reference/operator/query/). - -#### `filter` syntax - -A filter is composed of a nested object with a shape that matches the path to the value you want to compare on every entry in the given collection. The deepest nested level that does not have a JSON object as its value will be used to build the comparison where the `key` is the comparison operation and `value` is the value to compare every entry against. - -#### Example - -```js -filter = { postMeta: { rating: { gt: 80 } } }; - -entries = [ - { id: 1, title: 'My pretzel collection', postMeta: { rating: 97 } }, - { id: 2, title: 'Debugging the simulation', postMeta: { rating: 20 } }, - { - id: 3, - title: 'Liquid Proust is a great tea vendor btw', - postMeta: { rating: 99 }, - }, - { id: 4, title: 'Sitting in a chair', postMeta: { rating: 74 } }, -]; -``` - -The above filter would return entries with a rating greater than 80: - -```js -result = [ - { id: 1, title: 'My pretzel collection', postMeta: { rating: 97 } }, - { - id: 3, - title: 'Liquid Proust is a great tea vendor btw', - postMeta: { rating: 99 }, - }, -]; -``` - -#### Supported `filter` operations - -- `eq` - equal - - This is like `filterValue === resultValue` in JavaScript -- `ne` - not equal - - This is like `filterValue !== resultValue` in JavaScript -- `in` - - This is like `filterValue.includes(resultValue)` in JavaScript - - Can only be passed an array of values which pass strict comparison -- `nin` - - This is like `!filterValue.includes(resultValue)` in JavaScript - - Can only be passed an array of values which pass strict comparison -- `includes` - - This is like `resultValue.includes(filterValue)` in JavaScript - - Can only be passed a single value which passes strict comparison -- `excludes` - - This is like `!resultValue.includes(filterValue)` in JavaScript - - Can only be passed a single value which passes strict comparison -- `lt`, `lte`, `gt`, `gte` - - This is like `<`, `<=`, `>`, `>=` respectively - - Can only be used with numbers, strings, and booleans -- `exists` - - This is like `filterValue ? resultValue != undefined : resultValue == undefined` - - Accepts `true` or `false` as a value to compare against (`filterValue`) - - For checking against a property that could be both `null` or `undefined` -- `strictlyExists` - - This is like `filterValue ? resultValue !== undefined : resultValue === undefined` - - Accepts `true` or `false` as a value to compare against (`filterValue`) - - Checking against a property for `undefined` -- `regex` - - This is like new RegExp(filterValue).test(resultValue) in JavaScript -- `wildcard` - - This is an abstraction on top of `regex` for loose string matching - - Case insensitive - - Uses [matcher](https://github.com/sindresorhus/matcher) and matcher's [API](https://github.com/sindresorhus/matcher#usage) - -Caveats: - -- Currently cannot infer date strings and then compare `Date` types in filters - - should work if you dynamically pass in a `Date` object from your client, though not extensively tested - - to fix this, add argument `typeof` checks and the matching comparator functions in [`packages/core/src/utils/sift.ts`](https://github.com/FlatbreadLabs/flatbread/blob/main/packages/core/src/utils/sift.ts), then open a pull request - -#### Combining multiple filters - -You can union multiple filters together by adding peer objects within your filter object to point to multiple paths. - -#### Example - -Using the `entries` from the previous example, let's combine multiple filters. - -```graphql -query FilteredPosts { - allPosts( - filter: { title: { wildcard: "*tion" }, postMeta: { rating: { gt: 80 } } } - ) { - title - } -} -``` - -Results in: - -```js -result = [{ title: 'My pretzel collection' }]; -``` - -### `sortBy` - -Sorts by the given field. Accepts a root-level field name. Defaults to no sorting. - -### `order` - -The direction of sorting. Accepts `ASC` or `DESC`. Defaults to `ASC`. - -### `skip` - -Skips the specified number of entries. Accepts an integer. - -### `limit` - -Limits the number of returned entries to the specified amount. Accepts an integer. - -## Query from your app - -Follow [Quickstart (posts, authors, and tags)](#quickstart-posts-authors-and-tags) -to model related content, run codegen, and get typed results. For scripts and -framework setup, use the +For a complete app, run the [Next.js example](https://github.com/FlatbreadLabs/flatbread/tree/main/examples/nextjs). +It shows posts linked to authors, GraphQL document code generation with +`flatbread codegen`, and the prototype generated TypeScript read API. The +generated helpers still execute through GraphQL today. + +## How Flatbread works + +1. A source plugin finds files. +2. A transformer turns each file into a record. +3. `content` entries in `flatbread.config.js` group records into named + collections, such as `Post` or `Author`. +4. `refs` connect ID fields in one collection to records in another. +5. Flatbread validates the graph and exposes read interfaces: GraphQL, + generated TypeScript, or Proof's bounded CLI commands. + +One modeling note saves confusion later: a plain string list in frontmatter, +such as `tags: [cats, measurements]`, stays a scalar `[String]` field. It is +not a relation. If tags need their own records shared across posts, model a +`Tag` collection and point a `refs` field at it. The +[glossary](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md) +defines collections, records, IDs, relations, and cardinality. + +When you want typed results in application code, run `flatbread codegen`. It +generates TypeScript types and typed document nodes for your `.graphql` +operations, plus a prototype collection-shaped read API for plain reads +without a query document at each call site. Filters, sorting, pagination, and +field overrides are documented in the +[query reference](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/query-reference.md). + +## What Flatbread is not -## Field overrides - -Field overrides allow you to define custom GraphQL types or resolvers on top of fields in your content. For example, you could [optimize images](https://github.com/FlatbreadLabs/flatbread/tree/main/packages/resolver-svimg/), encapsulate an endpoint, and more! - -### Example - -```js -const config = { - content: { - overrides: [ - { - // The source field name. - field: 'name', - // The GraphQL type to expose. - type: 'String', - // Capitalize the value before returning it. - resolve: (name) => capitalize(name), - }, - ], - }, -}; -``` - -### Supported syntax for field - -- basic nested objects - - `nested.object` - -- a basic array (will map array values) - - `an.array[]` - -- a nested object inside an array (will also map array) - - `an.array[]with.object` - -for more information in Overrides, they adhere to the GraphQLFieldConfig outlined here https://graphql-compose.github.io/docs/basics/what-is-resolver.html - -## Advanced Config - -### `fieldNameTransform` - -Accepts a function which takes in field names and transforms them for the GraphQL schema generation -- this is used internally to remove spaces but can be used for other global transforms as well - -```js -{ - ... - // replace all spaces in field names with an underscore - fieldNameTransform: (fieldName) => fieldName.replace(/\s/g, '_') - ... -} -``` - -# ☀️ Contributing - -See [CONTRIBUTING.md](https://github.com/FlatbreadLabs/flatbread/blob/main/CONTRIBUTING.md) -for release steps, including version bumps and publishing. +- It is not a hosted CMS, dashboard, or writing UI. +- It is not a general-purpose database or GraphQL platform. Transactions, + detailed access control, and many concurrent writers are outside its scope. +- It does not reload its own packages. `flatbread start --watch` picks up + valid content and config changes while you work, but a change to a + Flatbread package needs a rebuild and a restart. The + [local development loop](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/local-dev-loop.md) + maps the exact watch boundaries. + +## Find the next detail + +| If you need to… | Read… | +| ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | +| Understand where Flatbread fits | [Positioning](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/positioning.md) | +| Learn the content vocabulary | [Glossary](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/glossary.md) | +| Set up agent memory | [Proof README](https://github.com/FlatbreadLabs/flatbread/blob/main/packages/proof/README.md) | +| Run the working example app | [Next.js example](https://github.com/FlatbreadLabs/flatbread/blob/main/examples/nextjs/README.md) | +| Use filters, sorting, pagination, or field overrides | [Query reference](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/query-reference.md) | +| Know what watch mode reloads | [Local development loop](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/local-dev-loop.md) | +| Keep or move your data | [Data ownership](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/data-ownership.md) | +| Export JSON or CSV through the core API | [Snapshot export](https://github.com/FlatbreadLabs/flatbread/blob/main/docs/json-export.md) | +| Build and test this monorepo | [Contributing](https://github.com/FlatbreadLabs/flatbread/blob/main/CONTRIBUTING.md) | + +## Contributing + +This monorepo uses Node 20.19+ and pnpm 10.33.x. Start with +[CONTRIBUTING.md](https://github.com/FlatbreadLabs/flatbread/blob/main/CONTRIBUTING.md), +and run `pnpm verify` before opening a pull request that changes source, +tests, package metadata, or CI. diff --git a/packages/proof/README.md b/packages/proof/README.md index 84e22992..98352bd8 100644 --- a/packages/proof/README.md +++ b/packages/proof/README.md @@ -1,77 +1,257 @@ # `@flatbread/proof` -Git-native memory for coding agents. Installing this package gives you three -things: - -- **Record types.** An agent records the work it is doing as an **Effort**, - then writes what it learns against that Effort: **Issues**, **Findings**, - **Decisions**, **Constraints**, **Risks**, **Citations**, and **Blobs**. -- **Write operations.** A typed mutation turns into Markdown files on disk, and - the writer checks the links between records before it commits them. -- **A Flatbread content model.** `proofContent()` adds those eight record - types to a Flatbread configuration, so the same files come back as a typed - graph you can query and page through. - -Every record is a Markdown file in your repository, so you commit, diff, -review, and revert an agent's reasoning the same way you handle code, and the -next session can read it back. - -Writes go through a journal, so a change that touches several files either -finishes in full or leaves nothing behind: if the process dies mid-write, the -next run restores the earlier contents of the unfinished change. - -Version 1 supports these actions: `CreateEffort`, `SetEffortStatus`, -`WriteIssue`, `WriteFinding`, `WriteDecision`, `WriteConstraint`, `WriteRisk`, -`WriteCitation`, `WriteBlob`, `Supersede`, `Invalidate`, `ResolveIssue`, -`AcceptDecision`, `MitigateRisk`, `SetRiskState`, and `Retract`. `Retract` -hides a record that should not have stayed on the live graph without deleting -the file. - -An Issue, Finding, Decision, Constraint, or Risk may name Citation ids in -`cites` (Flatbread `refs`). A Citation body alone is valid (e.g. a URL); an -optional `blob` ref attaches a long payload such as a document, JSON, or image. - -## Where records live, and what to ignore - -`proofContent()` stores the graph under `.flatbread-proof` in your -project root. Pass a path to choose another root: -`proofContent('path/to/graph')`. - -Two paths hold working state that Git should not track: the write journal at -`/.journal`, and the derived read cache at -`.flatbread/proof/read-cache`. Nothing adds them to `.gitignore` for -you, so add these lines yourself: +Proof keeps coding agents and the people they work with aligned. It stores +the facts and reasons that must outlive one chat session as a lightweight, +portable, Git-tracked knowledge graph. + +Each record is a Markdown file in your repository. The next agent session and +your coworkers read the same open questions, evidence, decisions, +constraints, and risks. Git supplies the collaboration tools you already +know: branches, diffs, reviews, history, and reverts. + +Proof is built on [Flatbread](https://github.com/FlatbreadLabs/flatbread), +which turns files in Git into a typed relational graph. This package holds +the Proof record model, writer, reader, and packaged agent skill. End users +install the public `flatbread` package: it provides the `flatbread proof` +CLI and re-exports `proofContent()`. + +Proof is not a transcript store, task tracker, hosted memory service, or +authoring interface. It keeps durable project knowledge — the small set of +reasons a future session or coworker would otherwise have to reconstruct. + +## The record model + +An **Effort** anchors one coherent thread of work — a feature, migration, +investigation, or refactor. Every other record belongs to exactly one Effort, +which keeps reads small and gives each piece of evidence a clear home. + +| Record | What it holds | +| -------------- | ----------------------------------------------------------- | +| **Effort** | One coherent thread of work | +| **Issue** | A question, defect, gap, or blocker that needs an answer | +| **Finding** | An observation grounded in code, users, or runtime behavior | +| **Decision** | A commitment among alternatives | +| **Constraint** | A hard or soft boundary on the decision space | +| **Risk** | A possible negative outcome, with likelihood and severity | +| **Citation** | An external source or reference, often a URL | +| **Blob** | Attached content such as a document, JSON payload, or image | + +Typed relations preserve the reasoning between records. A Decision can +`derive_from` the Findings, Constraints, and Issues it responds to. A Finding +can `invalidate` an older Finding or Decision. Records cite evidence through +`cites`, which names Citation records; a Citation can attach one Blob. Proof +validates every link and keeps it within one Effort. The +[Proof glossary](./skills/proof/glossary.md) gives the exact meanings. + +## First success + +Proof requires Node 20.19 or newer. Run its commands from the directory that +contains your project's one `flatbread.config.*` file. + +### 1. Install the skill and the matching runtime + +The skill and the `flatbread` package must use the same release. Read +[`skills/proof/release.json`](./skills/proof/release.json) for the current +`gitTag` and `flatbreadVersion` and use those +values exactly: + +```bash +npx skills add https://github.com/FlatbreadLabs/flatbread/tree/v1.1.0/packages/proof/skills/proof --skill proof +npm install --save-dev flatbread@1.1.0 +``` + +`npx skills add` runs the `skills` CLI, which copies the pinned skill folder +from that release tag into your project so agent tools can load it. The +[setup guide](./skills/proof/setup.md) gives the equivalent pnpm, Yarn, and +Bun commands. + +### 2. Add the Proof content model + +Create or update `flatbread.config.js`, keeping any content entries the +project already has: + +```js +import { + defineConfig, + sourceFilesystem, + transformerMarkdown, + proofContent, +} from 'flatbread'; + +export default defineConfig({ + source: sourceFilesystem(), + transformer: transformerMarkdown(), + content: [ + // Keep existing entries here. + ...proofContent(), + ], +}); +``` + +`proofContent()` adds eight collections under `.flatbread-proof/`. Pass a +path — `proofContent('path/to/graph')` — when the project needs another +root. Proof read and write commands require this complete preset in the +config; `bootstrap` reports when it is missing or incomplete. + +### 3. Ignore the working state + +Add two lines to `.gitignore`. Proof records stay tracked; only the write +journal and the derived read cache stay out of Git: ```gitignore **/.flatbread-proof/.journal/ **/.flatbread/proof/read-cache/ ``` -For a custom root, replace `.flatbread-proof` with that root. The read cache -path stays the same. +For a custom graph root, replace `.flatbread-proof` in the first line. The +read cache path never changes. -`flatbread proof bootstrap` reports what is still missing — the config entry -or either ignore rule. `flatbread proof bootstrap --verify` reports the same -and exits nonzero when anything is missing, which makes it usable in CI. +### 4. Verify the setup -## The domain model and the packaged skill +```bash +npx flatbread proof bootstrap +npx flatbread proof bootstrap --verify +``` -Read [`skills/proof/glossary.md`](./skills/proof/glossary.md) for -the portable Proof domain model. +The first command reports what is still missing — the config entry or either +ignore rule. The second prints one JSON object with `"status":"ready"` when +activation is complete, and exits nonzero when it is not, which makes it +usable in CI. Bootstrap is report-only: it never creates or edits project +files. -The packaged Agent Skill is in `skills/proof/`. The repository copy in -`.agents/skills/proof/` is generated from those files. Run -`pnpm skills:sync` from the repository root after changing the skill. +### 5. Write and read the first record -## Install the Proof skill +Create an Effort: -Install from a release tag, then activate the skill for setup: +```bash +npx flatbread proof write '{"type":"CreateEffort","title":"Choose a search index","body":"Track evidence, constraints, decisions, and open work."}' +``` + +The command prints one JSON object. Save `artifacts[0].id` — later records +name this Effort by that ID. Then list active Efforts: ```bash -npx skills add https://github.com/FlatbreadLabs/flatbread/tree//packages/proof/skills/proof --skill proof -npm install --save-dev flatbread@ +npx flatbread proof list --status active ``` -The tag and version come from `gitTag` and `flatbreadVersion` in -`skills/proof/release.json`. See `skills/proof/setup.md` for the -equivalent `pnpm`, `yarn`, and `bun` commands. +Every read returns a bounded JSON envelope. Open the Markdown file named by +its `artifact_path` to read the digest. + +## One session loop + +An agent resuming work follows the same bounded loop each time. + +1. **Resume.** `flatbread proof list --status active` finds the live + Efforts. For each relevant one, + `flatbread proof records --kinds issue,decision --limit 10` + summarizes its state, and + `flatbread proof blocking-decisions ` narrows to proposed + Decisions that derive from open blocker Issues. +2. **Zoom in only when needed.** Browse digests excerpt record bodies. Read + one full body with `flatbread proof get `, or follow a superseded + record to its current head with `flatbread proof get --resolve head`. + Reads cap at 25 primary records, one relation hop, 50 displayed edges, + and a 64 KiB digest; check `complete`, `page.has_more`, and `cap_reasons` + in the envelope before treating a digest as the whole story. +3. **Write only durable knowledge.** All 16 typed mutations go through one + command: `flatbread proof write ''`. Before a create or a body edit + adds a claim, the packaged skill applies a four-part gate — future need, + durable effect, causal value, and unique signal — and writes only when + all four hold. That gate is agent policy; the CLI does not enforce it. + Routine progress notes belong in the pull request or issue instead. +4. **Read your own write when it matters.** Each mutation returns a + `generation` token. Pass it back as + `--strict-min-generation ` to get data at or after that + generation; Proof waits up to 3000 ms by default, then fails with + `PROOF_GENERATION_WAIT_TIMEOUT`. Do not build a polling loop. +5. **Close the loop.** Use lifecycle mutations when the team commits to a + choice or resolves an Issue. One default deserves care: `AcceptDecision` + sets `rejectSiblings` to `true`, which rejects every other proposed + Decision in the same Effort — pass `"rejectSiblings":false` unless that + is what you mean. When a record should never have entered the graph, use + `Retract`: the file and reason stay in history, but browse reads omit the + record. Do not delete record files or hand-edit frontmatter. + +## Files, the journal, and recovery + +Tracked records live in eight directories under the graph root: + +```text +.flatbread-proof/ +├── efforts/ +├── issues/ +├── findings/ +├── decisions/ +├── constraints/ +├── risks/ +├── citations/ +└── blobs/ +``` + +A single mutation may touch several of these files, because Proof +materializes reverse links and lifecycle changes together. The writer +validates IDs, record kinds, and Effort boundaries first, then applies the +change through a journal at `/.journal/`. + +If a process stops mid-write, the journal makes the change safe — but +recovery has an exact timing: **a later `flatbread proof write` or a +Flatbread live-server start runs recovery; a read command alone does not.** +Recovery rolls back an uncommitted change or finishes publishing a committed +one. The generation token advances only after a write is published. Never +edit the journal directory. + +Record bodies may be edited by hand — the reindexer validates and repairs +projections — but hand edits bypass the journal and do not advance its +generation token. Frontmatter must only change through `flatbread proof write`. + +## Working with coworkers + +Commit the tracked `.flatbread-proof/` records with the code they explain. +Reviewers then see the reasoning and the implementation in one pull request, +and the next agent session starts from the merged graph. Proof does not +create Git commits, resolve merge conflicts, or run a hosted multi-writer +service; your normal Git workflow decides when records are shared. + +## Command map + +| Goal | Command | +| ----------------------------- | ------------------------------------------------------------------- | +| Check setup | `flatbread proof bootstrap --verify` | +| Find active work | `flatbread proof list --status active` | +| Browse one Effort | `flatbread proof records ` | +| Read one full record | `flatbread proof get ` | +| Follow selected links | `flatbread proof relations --relations ` | +| Find choices tied to blockers | `flatbread proof blocking-decisions ` | +| Apply a typed mutation | `flatbread proof write ''` | +| Prune old derived digests | `flatbread proof cache prune` | + +Every command prints one JSON object to standard output; errors print JSON +to standard error and exit with status 1. The +[full API reference](./skills/proof/reference.md) lists all 16 mutations, +read flags, lifecycle states, relation names, paging rules, and error codes. + +## Optional explorer + +With a complete `proofContent()` preset in config, the `flatbread` package +can serve a visual graph explorer: + +```bash +npx flatbread start --watch --open +``` + +When the explorer's prebuilt assets are present, it serves at +`http://localhost:5057/` and the GraphQL endpoint stays at +`http://localhost:5057/graphql`. When those assets are missing, `--open` +opens the GraphQL path instead. + +## Package and skill maintenance + +The canonical Agent Skill lives in [`skills/proof/`](./skills/proof/): +[`SKILL.md`](./skills/proof/SKILL.md) for the agent workflow and write gate, +[`setup.md`](./skills/proof/setup.md) for activation, +[`reference.md`](./skills/proof/reference.md) for the full API, and +[`glossary.md`](./skills/proof/glossary.md) for record and relation +meanings. The repository copy at `.agents/skills/proof/` is generated from +these files — do not edit it by hand. After changing the canonical skill in +this monorepo, run `pnpm skills:sync`, then `pnpm skills:check` and +`pnpm skills:pack-check`.