Document reserved import property names and large-import page sizing

.claude/skills/build-plugin/SKILL.md

@@ -62,7 +62,7 @@ Before writing a single file, understand and explore the API. **Use `AskUserQues
 
 1. **Find the docs** — Gather URLs or spec files from the user, then fetch and read them.
 2. **Identify the object model** — What are the core entities? (e.g. installations, devices, sites). These become the **indexed objects** in SquaredUp — available for drilldown, search, scoping dashboards, and use as variables.
-3. **Find the list endpoints** — Used to import objects. Prefer fetching **50–250 records per page** across multiple requests — SquaredUp has a per-page timeout but supports as many paged requests as needed.
+3. **Find the list endpoints** — Used to import objects. Note the page-size parameter's maximum, and estimate how many objects the largest environment holds (ask the user if unsure). An import run is capped at ~220 pages and ~6MB per page, with a per-page timeout. Start from **50–250 records per page** when that keeps the largest environment comfortably under ~200 pages; tens of thousands of objects need a larger `pageSize` with lean rows — see [index-defs.md](references/index-defs.md#large-imports--choosing-pagesize).
 4. **Find the data endpoints** — These power data streams. For each, record three things; Phase 2 turns the last two into the stream's `timeframes`:
     - **Scoping** — scoped to a single object, multiple objects, or global (no object context).
     - **Time-range control** — does the endpoint accept a queryable time range at all (a `from`/`to`, `start`/`end`, or `period` parameter), or only return a fixed snapshot / current values?

.claude/skills/build-plugin/references/checkpoints.md

@@ -56,7 +56,12 @@ squaredup objects --matches '{"sourceType":{"type":"equals","value":"<Object Typ
 Notes:
 
 - `index` / `index-status` are **folder-independent** — they need only `--datasource-id <id>` (not `--plugin-id`), since the re-index endpoint is keyed purely on the datasource.
-- **Branch on the JSON, not the exit code.** `index-status` exits `0` on every clean read regardless of import state. A failed import is `done: true, succeeded: false` — *not* a non-zero exit. Read the run-level `message` and the per-step `steps[]` (each step has `name`, `status`, `errorReason`, `totalObjectsReceived`, `totalObjectsWritten`) to see **which** step broke and why, fix that import stream, and re-trigger.
+- **Branch on the JSON, not the exit code.** `index-status` exits `0` on every clean read regardless of import state. A failed import is `done: true, succeeded: false` — *not* a non-zero exit. Read the run-level `message` and the per-step `steps[]` (each step has `name`, `status`, `errorReason`, `totalObjectsReceived`, `totalObjectsWritten`) to see **which** step broke and why, fix that import stream, and re-trigger. Three platform errors read as noise but have plugin-side causes:
+    - `Payload failed schema validation` with `/vertices/N/<prop>` — a [reserved property name](index-defs.md) (`id`, `state`, `__…`) in `objectMapping.properties`.
+    - `Function.ResponseSizeTooLarge` — one page is over ~6MB; lower `pageSize` or trim the row.
+    - `'call-chain-length' reached threshold of 220` — too many pages in the run; raise `pageSize`.
+
+    For the last two, see [Large imports](index-defs.md#large-imports--choosing-pagesize).
 - **`status` is the run's lifecycle, not the outcome.** While running it's `ready`/`inProgress`; once `done` it's one of `succeeded`, `failed`, `warning`, or `cancelled`. `succeeded` and `warning` both report `succeeded: true` (a `warning` run finished but a step emitted warnings — check the `steps[]`); `failed` and `cancelled` report `succeeded: false`. A datasource that has never imported reports `status: "notRun"`, `done: false`.
 - `--since` is **exclusive** (`scheduledStart > since`): always pass the `since` from `index` so `done` can't latch on a stale previous run.
 - If `index` reports `alreadyRunning: true`, it adopted the in-flight run — poll with the `since` it returned. Imports can take several minutes (object import allows up to ~10 min); use a generous overall timeout.

.claude/skills/build-plugin/references/index-defs.md

@@ -60,6 +60,12 @@ Defines what gets imported into the SquaredUp graph.
 - `objectMapping.properties` are extra fields stored on the graph node, accessible in scripts as `object.propName`.
 - Use `{ "targetProp": "sourceProp" }` syntax when the column name differs from the desired property name.
 - **Never map a column into `properties` that is already mapped as `id`, `name`, or `type`.** The id's raw value is always available on every object as `rawId` (`{{object.rawId}}` in templates, `context.objects[N].rawId` in scripts — and as a scalar, unlike user-defined properties, which arrive as arrays), and the name as `name`. Adding e.g. `{ "projectId": "id" }` to `properties` when `"id": "id"` already exists creates a duplicate that has to be re-indexed to take effect and otherwise sits as dead config — use `rawId`/`name` instead.
+- **Never use a reserved name as a property name.** The property name — a string entry, or the *key* in `{ "targetProp": "sourceProp" }` — becomes a property on the graph object, and the graph owns some names. Using one deploys cleanly, then fails the import with a raw schema error that doesn't mention your mapping:
+    - `id`, and any `__`-prefixed name — always rejected: `Payload failed schema validation` … `/vertices/0/id` … `boolean schema is false`. You don't need it — the object's id is already `rawId` (above).
+    - `state` — reserved by the platform; an API status (`"active"`) or a postal state (`"OH"`) fails with `must be equal to one of the allowed values`. Rename it: `{ "status": "state" }`, or `{ "region": "state" }` for an address.
+    - `rawId`, `sourceId`, `sourceType`, `name`, `label` — silently overwritten by the platform, so the mapping does nothing.
+
+    A reserved name is fine as the *source column* (`{ "status": "state" }` reads a column called `state`). `squaredup validate` rejects `id`, `state` and `__` names; it can't catch the silent overwrites.
 - The `objectMapping.sourceType` column value **must** match an entry in `objectTypes` in `metadata.json`. For dynamic ones add these based on API response data later.
 - `frequencyMinutes` — controls re-import interval. Defaults to `720` (12 hours).
 
@@ -79,6 +85,30 @@ Building and testing a dependent step needs objects from its dependency to alrea
 
 ---
 
+## Large imports — choosing `pageSize`
+
+An import step fetches one page at a time and writes each page's objects to the graph before asking for the next, so a step has no object-count limit of its own. Two limits bound the stream's [`paging.pageSize`](data-streams.md#pagination) from opposite sides instead:
+
+| Limit | What it counts | `errorReason` when hit | Pushes `pageSize` |
+| --- | --- | --- | --- |
+| **~6MB per page** | One page, measured after SquaredUp formats it — larger than the raw API body. That page's objects are then written to the graph in a single request. | `Function.ResponseSizeTooLarge` | down |
+| **~220 calls per import run** | Each page is one more call in a chain. A dependent step continues the chain of the step it `dependsOn`. | `'call-chain-length' reached threshold of 220` | up |
+
+Each page must also return within the ~28s per-page timeout, another reason not to max out `pageSize`.
+
+Size it for the **largest environment the plugin will meet**, not your test tenant — a 500-object test account passes at any page size:
+
+1. **Pages** — `objects ÷ pageSize`, plus the pages of every step this one transitively `dependsOn`, must stay comfortably under ~200.
+2. **Bytes** — `pageSize × bytes per row` should stay under ~2MB of raw API response, leaving headroom for formatting. Measure a real page during exploration, e.g. `curl … | wc -c`.
+
+A worked example: a CrowdStrike import of ~92,700 hosts overflowed 6MB at 1,000 hosts per page. 250 per page would have needed ~371 calls, so 500 per page with a lean row (~186 calls) was the size that fit both limits.
+
+**Keep import rows lean.** An import stream exists to feed `objectMapping` — return only the id, name, type and mapped properties. If the API has a field-selection parameter (`fields`, `select`, `$select`), use it, and leave nested blobs and anything only a dashboard needs to a separate data stream. Smaller rows let a larger `pageSize` fit under 6MB, which is what lets a big inventory fit under the call cap.
+
+**If no `pageSize` satisfies both**, import less rather than fighting the limits: filter at the API (e.g. only active or managed hosts), or import a subset and query the rest live through unscoped data streams. Raise it with the user — the call cap is a platform setting, not something a plugin can change.
+
+---
+
 ## Stream-level requirements
 
 Import streams are ordinary data stream files in `dataStreams/` — they need `name`, `displayName`, `description`, and `tags` like any other stream. Mark import-only streams as hidden so they don't clutter the tile editor:

REVIEW.md

@@ -104,6 +104,8 @@ When suggesting changes:
 ### Source types (indexDefinitions/)
 
 - Name source types after how they are referred to in the upstream product or API (e.g. `agent`, `device`). Do not prefix them with the plugin name (e.g. avoid `NinjaOne Device`). A separate friendly display name can be configured if needed (via custom_types.json).
+- objectMapping.properties - Never use `id`, `state`, or a `__`-prefixed name as a property name; they fail the import at graph write. Rename instead, e.g. `{ "status": "state" }`. `rawId`, `sourceId`, `sourceType`, `name` and `label` are silently overwritten, so mapping them does nothing.
+- pageSize - For APIs where a tenant can hold tens of thousands of objects, check the import stream's `pageSize` against both import limits: ~6MB per page and ~220 pages per import run. See the build-plugin skill's index-defs.md, "Large imports".
 
 ### Documentation - (docs/README.md)