diff --git a/.changeset/datasync-conflict-dual-source-c13-c15.md b/.changeset/datasync-conflict-dual-source-c13-c15.md new file mode 100644 index 0000000000..b2900997ed --- /dev/null +++ b/.changeset/datasync-conflict-dual-source-c13-c15.md @@ -0,0 +1,44 @@ +--- +"@objectstack/spec": major +--- + +BREAKING(spec): 退役 L1「Simple Sync」整层(`@objectstack/spec/automation` 的 `DataSyncConfig` 一族,17 个导出名),并把 `@objectstack/spec/integration` 的冲突策略枚举改名 `ConflictResolution` → `ConnectorConflictResolution`;裸名 `ConflictResolution` 现在全包唯一地指 `@objectstack/spec/ui` 的离线同步概念 (#4738, #4535 C13+C15) + +`DataSyncConfig(Schema)` 曾由 `./automation` 与 `./integration` 各自导出一个声明,`ConflictResolution(Schema)` 更是**三个入口三个声明**(#4411 陷阱): + +| 名字 | 入口 | 词表/形状 | 处置 | +|:--|:--|:--|:--| +| `DataSyncConfig` | `./automation`(**删除**) | 19 键,direction=push/pull,batchSize 默认 100 | 随 L1 整层退役 | +| `DataSyncConfig` | `./integration`(**保名不动**) | 9 键,direction=import/export/bidirectional,batchSize 默认 1000 | 唯一真源(`ConnectorSchema.syncConfig` 活解析路径) | +| `ConflictResolution` | `./automation`(**删除**) | `destination_wins` / `merge` 等 5 值 | 随 L1 整层退役 | +| `ConflictResolution` | `./integration`(**改名**) | `target_wins` 等 4 值 | → `ConnectorConflictResolution(Schema)`,枚举值逐字不变 | +| `ConflictResolution` | `./ui`(**一字不动**) | `client_wins` / `server_wins` / `manual` / `last_write_wins` | 裸名唯一归属(objectui 实活消费) | + +**automation 侧是叙事层,不是实现**:L1「Simple Sync」只存在于 `SYNC_ARCHITECTURE.md` 的三层故事里 —— 三仓(objectstack / cloud / objectui)import 语句级零消费者,没有任何引擎解析或执行过 `DataSyncConfig`,8 个 def 从元数据根真 Zod 图不可达(#4650 门禁实测)。整文件删除:`DataSyncConfig(Schema)`、`ConflictResolution(Schema)`、`SyncDirection(Schema)`、`SyncMode(Schema)`、`DataSourceConfig(Schema)`、`DataDestinationConfig(Schema)`、`SyncExecutionStatus(Schema)`、`SyncExecutionResult(Schema)`、`Sync` 工厂。 + +## FROM → TO + +```ts +// FROM —— 编译期起以 TS2305 失败(实测三仓零命中,预期无人受影响) +import { DataSyncConfig, ConflictResolution, Sync } from '@objectstack/spec/automation'; +``` + +- 若你要的是**连接器同步策略配置**(唯一活着的服务端 sync 面): + + ```ts + // TO —— ConnectorSchema.syncConfig 的类型;裸名保持不变 + import { DataSyncConfig, ConnectorConflictResolution } from '@objectstack/spec/integration'; + ``` + +- 若你要的是**多源转换管道**:`import { ETLPipeline } from '@objectstack/spec/automation'`。 +- 若你要的是**客户端离线冲突策略**:`import { ConflictResolution } from '@objectstack/spec/ui'`(本次未动)。 + +```ts +// FROM —— integration 侧旧名,编译期起以 TS2305 失败 +import { ConflictResolution, ConflictResolutionSchema } from '@objectstack/spec/integration'; + +// TO —— 同一声明、同一词表,只是名字带上了域前缀 +import { ConnectorConflictResolution, ConnectorConflictResolutionSchema } from '@objectstack/spec/integration'; +``` + +**零元数据迁移**:integration 改名只动 TS 导出名,`connectors[].syncConfig.conflictResolution` 的取值域(`source_wins` / `target_wins` / `latest_wins` / `manual`)逐字节不变,已发布的 connector 元数据原样解析(def 改名走 `RENAMED_DEFS` 承接表,0-key carry);automation 删除侧没有任何存量元数据可迁 —— 无解析站点即无作者,conversion 写不出能跑到的(不在 stack 树,`converge-activation-event-schema` 先例论证)。相邻雷勿踩:`@objectstack/spec/api` 的 `ConflictResolutionStrategy`(路由冲突,`error` / `priority` / `first-wins` / `last-wins`)是第四个同族概念、不同名,本次未动。 diff --git a/content/docs/getting-started/quick-reference.mdx b/content/docs/getting-started/quick-reference.mdx index 3de1aa9626..87d4232b24 100644 --- a/content/docs/getting-started/quick-reference.mdx +++ b/content/docs/getting-started/quick-reference.mdx @@ -146,7 +146,7 @@ REST/GraphQL endpoints, real-time subscriptions, and discovery. | **[Registry](/docs/references/api/registry)** | `registry.zod.ts` | Registry | API registry | | **[Storage](/docs/references/api/storage)** | `storage.zod.ts` | Storage | API storage operations | -## Automation Protocol (7 schemas) +## Automation Protocol (5 schemas) Flows, state machines, approvals, and integrations. @@ -157,7 +157,6 @@ Flows, state machines, approvals, and integrations. | **[State Machine](/docs/references/automation/state-machine)** | `state-machine.zod.ts` | StateMachine | State machine definitions | | **[Webhook](/docs/references/automation/webhook)** | `webhook.zod.ts` | Webhook | Outbound webhooks | | **[ETL](/docs/references/automation/etl)** | `etl.zod.ts` | ETLPipeline | Data transformation pipelines | -| **[Sync](/docs/references/automation/sync)** | `sync.zod.ts` | DataSyncConfig, SyncMode | Bi-directional data sync | ## Security Protocol (3 schemas) diff --git a/content/docs/references/automation/connector.mdx b/content/docs/references/automation/connector.mdx deleted file mode 100644 index 3128787833..0000000000 --- a/content/docs/references/automation/connector.mdx +++ /dev/null @@ -1,48 +0,0 @@ ---- -title: Connector -description: Connector protocol schemas ---- - -{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} - -## TypeScript Usage - -```typescript -import { DataSyncConfigSchema } from '@objectstack/spec/automation'; -import type { DataSyncConfig } from '@objectstack/spec/automation'; - -// Validate data -const result = DataSyncConfigSchema.parse(data); -``` - ---- - -## DataSyncConfig - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **name** | `string` | ✅ | Sync configuration name (snake_case) | -| **label** | `string` | optional | Sync display name | -| **description** | `string` | optional | Sync description | -| **source** | `{ object?: string; filters?: any; fields?: string[]; connectorInstanceId?: string; … }` | ✅ | Data source | -| **destination** | `{ object?: string; connectorInstanceId?: string; operation: Enum<'insert' \| 'update' \| 'upsert' \| 'delete' \| 'sync'>; mapping?: Record \| { source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }; defaultValue?: any }[]; … }` | ✅ | Data destination | -| **direction** | `Enum<'push' \| 'pull' \| 'bidirectional'>` | optional | Sync direction | -| **syncMode** | `Enum<'full' \| 'incremental' \| 'realtime'>` | optional | Sync mode | -| **conflictResolution** | `Enum<'source_wins' \| 'destination_wins' \| 'latest_wins' \| 'manual' \| 'merge'>` | optional | Conflict resolution | -| **schedule** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | optional | Cron schedule | -| **enabled** | `boolean` | optional | Sync enabled | -| **changeTrackingField** | `string` | optional | Field for change tracking | -| **batchSize** | `integer` | optional | Batch size for processing | -| **retry** | `{ maxAttempts?: integer; backoffMs?: integer }` | optional | Retry configuration | -| **validation** | `{ required?: string[]; unique?: string[]; custom?: { name: string; condition: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; message: string }[] }` | optional | Validation rules | -| **errorHandling** | `{ onValidationError?: Enum<'skip' \| 'fail' \| 'log'>; onSyncError?: Enum<'skip' \| 'fail' \| 'retry'>; notifyOnError?: string[] }` | optional | Error handling | -| **optimization** | `{ parallelBatches?: boolean; cacheEnabled?: boolean; compressionEnabled?: boolean }` | optional | Performance optimization | -| **audit** | `{ logLevel?: Enum<'none' \| 'error' \| 'warn' \| 'info' \| 'debug'>; retainLogsForDays?: integer; trackChanges?: boolean }` | optional | Audit configuration | -| **tags** | `string[]` | optional | Sync tags | -| **metadata** | `Record` | optional | Custom metadata | - - ---- - diff --git a/content/docs/references/automation/etl.mdx b/content/docs/references/automation/etl.mdx index 958317067e..f5554cef69 100644 --- a/content/docs/references/automation/etl.mdx +++ b/content/docs/references/automation/etl.mdx @@ -9,13 +9,15 @@ ETL (Extract, Transform, Load) Pipeline Protocol - LEVEL 2: Data Engineering Inspired by modern data integration platforms like Airbyte, Fivetran, and Apache NiFi. -**Positioning in 3-Layer Architecture:** +**Positioning in the sync/integration layering** (L1 "Simple Sync" was -- **L1: Simple Sync** (automation/sync.zod.ts) - Business users - Sync Salesforce to Sheets +retired in #4738 — narrative-only, zero consumers; see -- **L2: ETL Pipeline** (THIS FILE) - Data engineers - Aggregate 10 sources to warehouse +`packages/spec/docs/SYNC_ARCHITECTURE.md`): -- **L3: Enterprise Connector** (integration/connector.zod.ts) - System integrators - Full SAP integration +- **ETL Pipeline** (THIS FILE) - Data engineers - Aggregate 10 sources to warehouse + +- **Enterprise Connector** (integration/connector.zod.ts) - System integrators - Full SAP integration; connector-attached sync via `syncConfig` ETL pipelines enable automated data synchronization between systems, transforming @@ -45,17 +47,11 @@ Supports complex operations: joins, aggregations, filtering, custom SQL. - Legacy system migration with transformation -**When to downgrade:** - -- Simple 1:1 sync → Use [Simple Sync](./sync.zod.ts) - **When to upgrade:** - Need full connector lifecycle (auth, webhooks, rate limits) → Use [Enterprise Connector](/docs/references/integration/connector) -See also: `./sync.zod.ts` for Level 1 (simple sync) - -See also: [../[integration/connector.zod.ts](/docs/references/integration/connector)](/docs/references/integration/connector) for Level 3 (enterprise integration) +See also: [../[integration/connector.zod.ts](/docs/references/integration/connector)](/docs/references/integration/connector) for the Enterprise Connector layer ## Use Cases diff --git a/content/docs/references/automation/index.mdx b/content/docs/references/automation/index.mdx index 604be92f4b..626af5d65d 100644 --- a/content/docs/references/automation/index.mdx +++ b/content/docs/references/automation/index.mdx @@ -18,7 +18,6 @@ This section contains all protocol schemas for the automation layer of ObjectSta - diff --git a/content/docs/references/automation/meta.json b/content/docs/references/automation/meta.json index 1ec491015b..57ef1974b1 100644 --- a/content/docs/references/automation/meta.json +++ b/content/docs/references/automation/meta.json @@ -10,10 +10,7 @@ "time-relative-trigger", "---Integration & Data---", "bpmn-interop", - "connector", "etl", - "offline", - "sync", "webhook", "---Approvals & Jobs---", "approval", diff --git a/content/docs/references/automation/offline.mdx b/content/docs/references/automation/offline.mdx deleted file mode 100644 index 5f6e89e9f6..0000000000 --- a/content/docs/references/automation/offline.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Offline -description: Offline protocol schemas ---- - -{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} - -## TypeScript Usage - -```typescript -import { ConflictResolutionSchema } from '@objectstack/spec/automation'; -import type { ConflictResolution } from '@objectstack/spec/automation'; - -// Validate data -const result = ConflictResolutionSchema.parse(data); -``` - ---- - -## ConflictResolution - -### Allowed Values - -* `source_wins` -* `destination_wins` -* `latest_wins` -* `manual` -* `merge` - - ---- - diff --git a/content/docs/references/automation/sync.mdx b/content/docs/references/automation/sync.mdx deleted file mode 100644 index b644e2abe7..0000000000 --- a/content/docs/references/automation/sync.mdx +++ /dev/null @@ -1,239 +0,0 @@ ---- -title: Sync -description: Sync protocol schemas ---- - -{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} - -Data Sync Protocol - LEVEL 1: Simple Synchronization - -Inspired by Salesforce Connect, Segment Sync, and Census Reverse ETL. - -**Positioning in 3-Layer Architecture:** - -- **L1: Simple Sync** (THIS FILE) - Business users - Sync Salesforce to Sheets - -- **L2: ETL Pipeline** (automation/etl.zod.ts) - Data engineers - Aggregate 10 sources to warehouse - -- **L3: Enterprise Connector** (integration/connector.zod.ts) - System integrators - Full SAP integration - -Data sync provides bidirectional or unidirectional data synchronization - -between ObjectStack and external systems, maintaining data consistency - -across platforms. - -**SCOPE: Simple field mappings only. NO complex transformations.** - -For complex transformations (joins, aggregates, custom SQL), use ETL Pipeline (Level 2). - -## When to Use This Layer - -**Use Simple Sync when:** - -- Syncing 1:1 fields between two systems - -- Simple field transformations (uppercase, cast, etc.) - -- No complex logic required - -- Business users need to configure integrations - -**Examples:** - -- Salesforce Contact ↔ Google Sheets - -- HubSpot Company ↔ CRM Account - -- Shopify Orders → Accounting System - -**When to upgrade:** - -- Need multi-source joins → Use [ETL Pipeline](./etl.zod.ts) - -- Need complex authentication/webhooks → Use [Enterprise Connector](/docs/references/integration/connector) - -- Need aggregations or data warehousing → Use [ETL Pipeline](./etl.zod.ts) - -See also: `./etl.zod.ts` for Level 2 (data engineering) - -See also: [../[integration/connector.zod.ts](/docs/references/integration/connector)](/docs/references/integration/connector) for Level 3 (enterprise integration) - -## Use Cases - -1. **CRM Integration** - -- Sync contacts between ObjectStack and Salesforce - -- Keep opportunity data synchronized - -- Bidirectional updates - -2. **Customer Data Platform (CDP)** - -- Sync user profiles to Segment - -- Enrichment data from Clearbit - -- Marketing automation sync - -3. **Operational Analytics** - -- Sync production data to analytics warehouse - -- Real-time dashboards - -- Business intelligence - -See also: https://help.salesforce.com/s/articleView?id=sf.platform_connect_about.htm - -See also: https://segment.com/docs/connections/sync/ - -See also: https://www.getcensus.com/ - -@example - -```typescript - -const contactSync: DataSyncConfig = \{ - -name: 'salesforce_contact_sync', - -label: 'Salesforce Contact Sync', - -source: \{ - -object: 'contact', - -filters: \{ status: 'active' \} - -\}, - -destination: \{ - -connector: 'salesforce', - -operation: 'upsert_contact', - -mapping: \{ - -first_name: 'FirstName', - -last_name: 'LastName', - -email: 'Email' - -\} - -\}, - -syncMode: 'incremental', - -schedule: '0 * * * *' // Hourly - -\} - -``` - - -**Source:** `packages/spec/src/automation/sync.zod.ts` - - -## TypeScript Usage - -```typescript -import { DataDestinationConfigSchema, DataSourceConfigSchema, SyncDirectionSchema, SyncExecutionResultSchema, SyncExecutionStatusSchema, SyncModeSchema } from '@objectstack/spec/automation'; -import type { DataDestinationConfig, DataSourceConfig, SyncDirection, SyncExecutionResult, SyncExecutionStatus, SyncMode } from '@objectstack/spec/automation'; - -// Validate data -const result = DataDestinationConfigSchema.parse(data); -``` - ---- - -## DataDestinationConfig - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **object** | `string` | optional | ObjectStack object name | -| **connectorInstanceId** | `string` | optional | Connector instance ID | -| **operation** | `Enum<'insert' \| 'update' \| 'upsert' \| 'delete' \| 'sync'>` | ✅ | Sync operation | -| **mapping** | `Record \| { source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }; defaultValue?: any }[]` | optional | Field mappings | -| **externalResource** | `string` | optional | External resource ID | -| **matchKey** | `string[]` | optional | Match key fields | - - ---- - -## DataSourceConfig - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **object** | `string` | optional | ObjectStack object name | -| **filters** | `any` | optional | Filter conditions | -| **fields** | `string[]` | optional | Fields to sync | -| **connectorInstanceId** | `string` | optional | Connector instance ID | -| **externalResource** | `string` | optional | External resource ID | - - ---- - -## SyncDirection - -### Allowed Values - -* `push` -* `pull` -* `bidirectional` - - ---- - -## SyncExecutionResult - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **id** | `string` | ✅ | Execution ID | -| **syncName** | `string` | ✅ | Sync name | -| **status** | `Enum<'pending' \| 'running' \| 'completed' \| 'partial' \| 'failed' \| 'cancelled'>` | ✅ | Execution status | -| **startedAt** | `string` | ✅ | Start time | -| **completedAt** | `string` | optional | Completion time | -| **durationMs** | `number` | optional | Duration in ms | -| **stats** | `{ recordsProcessed: integer; recordsInserted: integer; recordsUpdated: integer; recordsDeleted: integer; … }` | optional | Execution statistics | -| **errors** | `{ recordId?: string; field?: string; message: string; code?: string }[]` | optional | Errors | -| **logs** | `string[]` | optional | Execution logs | - - ---- - -## SyncExecutionStatus - -### Allowed Values - -* `pending` -* `running` -* `completed` -* `partial` -* `failed` -* `cancelled` - - ---- - -## SyncMode - -### Allowed Values - -* `full` -* `incremental` -* `realtime` - - ---- - diff --git a/content/docs/references/integration/connector.mdx b/content/docs/references/integration/connector.mdx index 7e6167993d..89b4fef6d5 100644 --- a/content/docs/references/integration/connector.mdx +++ b/content/docs/references/integration/connector.mdx @@ -13,13 +13,15 @@ Connectors enable ObjectStack to sync data with SaaS apps, databases, file stora and message queues through a unified protocol. -**Positioning in 3-Layer Architecture:** +**Positioning in the sync/integration layering** (L1 "Simple Sync" was -- **L1: Simple Sync** (automation/sync.zod.ts) - Business users - Sync Salesforce to Sheets +retired in #4738 — narrative-only, zero consumers; see -- **L2: ETL Pipeline** (automation/etl.zod.ts) - Data engineers - Aggregate 10 sources to warehouse +`packages/spec/docs/SYNC_ARCHITECTURE.md`): -- **L3: Enterprise Connector** (THIS FILE) - System integrators - Full SAP integration +- **ETL Pipeline** (automation/etl.zod.ts) - Data engineers - Aggregate 10 sources to warehouse + +- **Enterprise Connector** (THIS FILE) - System integrators - Full SAP integration; connector-attached sync via `syncConfig` **SCOPE: Most comprehensive integration layer.** @@ -95,13 +97,9 @@ Authentication is now imported from the canonical `auth/config.zod.ts`. **When to downgrade:** -- Simple field sync → Use [Simple Sync](/docs/references/automation/sync) - - Data transformation only → Use [ETL Pipeline](/docs/references/automation/etl) -See also: [../[automation/sync.zod.ts](/docs/references/automation/sync)](/docs/references/automation/sync) for Level 1 (simple sync) - -See also: [../[automation/etl.zod.ts](/docs/references/automation/etl)](/docs/references/automation/etl) for Level 2 (data engineering) +See also: [../[automation/etl.zod.ts](/docs/references/automation/etl)](/docs/references/automation/etl) for the ETL Pipeline layer (data engineering) ## There is no "Trigger Registry" alternative @@ -123,7 +121,11 @@ per-provider template cluster). The same defect class as the somewhere enforced. Lightweight cases are served HERE — a connector instance -with simple `auth` — or by `[automation/sync.zod.ts](/docs/references/automation/sync)` / `etl.zod.ts` below. +with simple `auth` — or by `[automation/etl.zod.ts](/docs/references/automation/etl)` for transformation + +pipelines. (The automation-side L1 "Simple Sync" file was itself retired as + +a dead end of the same class in #4738.) **Source:** `packages/spec/src/integration/connector.zod.ts` @@ -132,8 +134,8 @@ with simple `auth` — or by `[automation/sync.zod.ts](/docs/references/automati ## TypeScript Usage ```typescript -import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorErrorCategorySchema, ConnectorFieldMappingSchema, ConnectorHealthSchema, ConnectorRateLimitConfigSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RateLimitStrategySchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration'; -import type { CircuitBreakerConfig, Connector, ConnectorErrorCategory, ConnectorFieldMapping, ConnectorHealth, ConnectorRateLimitConfig, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RateLimitStrategy, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration'; +import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorConflictResolutionSchema, ConnectorErrorCategorySchema, ConnectorFieldMappingSchema, ConnectorHealthSchema, ConnectorRateLimitConfigSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RateLimitStrategySchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration'; +import type { CircuitBreakerConfig, Connector, ConnectorConflictResolution, ConnectorErrorCategory, ConnectorFieldMapping, ConnectorHealth, ConnectorRateLimitConfig, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RateLimitStrategy, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration'; // Validate data const result = CircuitBreakerConfigSchema.parse(data); @@ -205,6 +207,20 @@ Circuit breaker configuration | **outputSchema** | `Record` | optional | Output schema (JSON Schema) | +--- + +## ConnectorConflictResolution + +Conflict resolution strategy + +### Allowed Values + +* `source_wins` +* `target_wins` +* `latest_wins` +* `manual` + + --- ## ConnectorErrorCategory diff --git a/content/docs/references/integration/meta.json b/content/docs/references/integration/meta.json index d443f3db08..4fc44358aa 100644 --- a/content/docs/references/integration/meta.json +++ b/content/docs/references/integration/meta.json @@ -3,8 +3,6 @@ "pages": [ "---Connectors---", "connector", - "connector-auth", - "---Transport & Storage---", - "offline" + "connector-auth" ] } \ No newline at end of file diff --git a/content/docs/references/integration/offline.mdx b/content/docs/references/integration/offline.mdx deleted file mode 100644 index 9877357420..0000000000 --- a/content/docs/references/integration/offline.mdx +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: Offline -description: Offline protocol schemas ---- - -{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} - -## TypeScript Usage - -```typescript -import { ConflictResolutionSchema } from '@objectstack/spec/integration'; -import type { ConflictResolution } from '@objectstack/spec/integration'; - -// Validate data -const result = ConflictResolutionSchema.parse(data); -``` - ---- - -## ConflictResolution - -Conflict resolution strategy - -### Allowed Values - -* `source_wins` -* `target_wins` -* `latest_wins` -* `manual` - - ---- - diff --git a/content/docs/references/shared/mapping.mdx b/content/docs/references/shared/mapping.mdx index f81b0c485d..33c35d1f93 100644 --- a/content/docs/references/shared/mapping.mdx +++ b/content/docs/references/shared/mapping.mdx @@ -7,7 +7,7 @@ description: Mapping protocol schemas Base Field Mapping Protocol -Shared by: ETL, Sync, Connector, External Lookup +Shared by: ETL, Connector, External Lookup This module provides the canonical field mapping schema used across @@ -17,8 +17,6 @@ ObjectStack for data transformation and synchronization. - ETL pipelines (data/mapping.zod.ts) -- Data synchronization (automation/sync.zod.ts) - - Integration connectors (integration/connector.zod.ts) - External lookups (data/external-lookup.zod.ts) diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.md b/docs/audits/2026-07-unknown-key-strictness-ledger.md index 0b8a772253..a7e61cbeb4 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.md @@ -477,12 +477,12 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts). | `validation.zod.ts` | 6 | authorable | **strict as of #4001 batch 3b** — a `z.lazy()` discriminated union, so the one-call conversion does not apply: each of the six variants builds its own `strictObject` from a shared `BASE_VALIDATION_SHAPE`. Closing the base alone would have rejected correctly but suggested from the SHARED keys only, so a typo of a variant's own key (`transtions` → `transitions`) would get no rename. Site count 1 → 6 because the six variants are now object sites in their own right. The ADR-0010 envelope lives in the shared shape, so all six inherit it | | `field-value.zod.ts` / `seed.zod.ts` | 1+1 | mixed (p) | `seed` is strict (registered-types batch) | -### `automation/` — 86 sites +### `automation/` — 74 sites | File | Sites | Class | Note | |---|---|---|---| | `flow.zod.ts` | 11 | authorable | **strict as of #4001** (4 schemas; `FlowVersionHistorySchema` is runtime — stays tolerant) | -| `sync.zod.ts` / `etl.zod.ts` | 12+10 | authorable (p) | authored pipelines — **candidates** | +| `etl.zod.ts` | 10 | authorable (p) | authored pipelines — **candidate**. **−12 at #4738**: `sync.zod.ts` (the L1 "Simple Sync" file — `DataSyncConfig`, its `ConflictResolution` enum and satellites, formerly this row's co-candidate) was deleted whole rather than hardened: three-repo zero importers, no parse site, defs unreachable from the metadata-type roots (#4650 gate), so there was no author for strictness to protect (#4535 C13+C15). The integration-side `ConflictResolution` → `ConnectorConflictResolution` rename in the same change is name-only and moves no sites | | `execution.zod.ts` | 13 | wire | run-state envelopes — never strict. +5 at #4354 (the run-summary family: step metrics / skip reason / per-node / per-gate / the summary itself) — engine-emitted telemetry read by the Console and by operator queries, nobody authors them, so the `wire` verdict covers them unchanged | | `state-machine.zod.ts` | 6 | authorable (p) | **−1 at #4658**: the orphan `EventSchema` (`{ type, schema }`, an XState-style signal declaration nothing referenced — `StateMachineSchema` names event types as `on:` record keys) was deleted rather than converged with `kernel/events/core.zod.ts`'s envelope `EventSchema`, whose key set it did not intersect (#4535 C6). The remaining 6 sites and their verdict are unchanged | | `control-flow.zod.ts` | 5 | authorable (p) | validated structurally by `validateControlFlow`. **−1 at #4661**: `RetryPolicySchema` moved out to `shared/retry-policy.zod.ts` — `./automation` and `./system` published the same name for two different declarations (#4411), so the retry policy converged onto one. The site still exists and is still non-strict and authorable; it is simply no longer in a directory this ledger sections. ⚠️ That is a coverage gap worth knowing about: this audit sections `ui/` / `data/` / `automation/` / `security/` / `studio/` only, so a `shared/` shape is unaudited by construction. The tolerance is deliberate here — the `retryDelayMs` → `backoffMs` rename is tombstoned via `retiredKey()` precisely because a non-strict parent would otherwise swallow the old spelling | diff --git a/packages/qa/dogfood/test/expression-conformance.ledger.ts b/packages/qa/dogfood/test/expression-conformance.ledger.ts index 2efecb761e..d1d42a2e4c 100644 --- a/packages/qa/dogfood/test/expression-conformance.ledger.ts +++ b/packages/qa/dogfood/test/expression-conformance.ledger.ts @@ -160,12 +160,16 @@ export const EXPRESSION_SURFACE: ExprSurface[] = [ }, { id: 'cel-flow', - summary: 'flow / sync / loader branching + filter predicates', + summary: 'flow / loader branching + filter predicates', dialect: 'cel', mode: 'interpret', state: 'enforced', failPolicy: 'throw', enforcement: '@objectstack/formula celEngine (interpret) via the automation runtime', covers: [ 'automation/flow.zod.ts:condition', - 'automation/sync.zod.ts:condition', + // `automation/sync.zod.ts:condition` (custom validation predicates on + // the L1 "Simple Sync" DataSyncConfig) left with the whole file in + // #4738 — the L1 layer was narrative-only, so no engine ever evaluated + // that predicate. Connector-attached sync (`ConnectorSchema.syncConfig`) + // declares no expression surface; nothing to re-point at. // `kernel/metadata-loader.zod.ts:filter` (on MetadataLoadOptions and // MetadataExportOptions) was removed with the rest of that file's // zero-consumer duplicate envelope family in #4411. The surviving diff --git a/packages/spec/PROTOCOL_MAP.md b/packages/spec/PROTOCOL_MAP.md index edffaf5066..795fabcefc 100644 --- a/packages/spec/PROTOCOL_MAP.md +++ b/packages/spec/PROTOCOL_MAP.md @@ -71,7 +71,6 @@ This document serves as the **Grand Map** of the ObjectStack specification. It l | [`approval.zod.ts`](src/automation/approval.zod.ts) | ⭐ | **Approval Node**. Flow node config for human approval pauses. | | [`webhook.zod.ts`](src/automation/webhook.zod.ts) | ⭐ | **Webhooks**. Outbound HTTP notification configuration. | | [`etl.zod.ts`](src/automation/etl.zod.ts) | | **ETL Jobs**. Extract-Transform-Load definitions. | -| [`sync.zod.ts`](src/automation/sync.zod.ts) | | **Data Sync**. Bi-directional synchronization rules. | --- diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index b68d83f137..2e60e0b407 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -2112,19 +2112,11 @@ "ConcurrencyPolicy (type)", "ConcurrencyPolicyParsed (type)", "ConcurrencyPolicySchema (const)", - "ConflictResolution (type)", - "ConflictResolutionSchema (const)", "CreateRecordConfig (type)", "CreateRecordConfigParsed (type)", "CreateRecordConfigSchema (const)", "DEFAULT_FLOW_FUNCTION_EFFECT (const)", "DEPRECATED_APPROVER_TYPES (const)", - "DataDestinationConfig (type)", - "DataDestinationConfigSchema (const)", - "DataSourceConfig (type)", - "DataSourceConfigSchema (const)", - "DataSyncConfig (type)", - "DataSyncConfigSchema (const)", "DecisionCondition (type)", "DecisionConditionSchema (const)", "DecisionConfig (type)", @@ -2268,15 +2260,6 @@ "SubflowConfig (type)", "SubflowConfigParsed (type)", "SubflowConfigSchema (const)", - "Sync (const)", - "SyncDirection (type)", - "SyncDirectionSchema (const)", - "SyncExecutionResult (type)", - "SyncExecutionResultSchema (const)", - "SyncExecutionStatus (type)", - "SyncExecutionStatusSchema (const)", - "SyncMode (type)", - "SyncModeSchema (const)", "TIME_RELATIVE_DEFAULT_CRON (const)", "TIME_RELATIVE_DEFAULT_MAX_RECORDS (const)", "TRY_CATCH_NODE_TYPE (const)", @@ -3900,11 +3883,11 @@ "CONNECTOR_UPSTREAM_UNAVAILABLE (const)", "CircuitBreakerConfig (type)", "CircuitBreakerConfigSchema (const)", - "ConflictResolution (type)", - "ConflictResolutionSchema (const)", "Connector (type)", "ConnectorActionDescriptor (interface)", "ConnectorActionSchema (const)", + "ConnectorConflictResolution (type)", + "ConnectorConflictResolutionSchema (const)", "ConnectorDescriptor (interface)", "ConnectorErrorCategory (type)", "ConnectorErrorCategorySchema (const)", diff --git a/packages/spec/authorable-surface.json b/packages/spec/authorable-surface.json index 3fbabd7ab8..1f5bc9171b 100644 --- a/packages/spec/authorable-surface.json +++ b/packages/spec/authorable-surface.json @@ -2196,36 +2196,6 @@ "automation/CreateRecordConfig:fields", "automation/CreateRecordConfig:objectName", "automation/CreateRecordConfig:outputVariable", - "automation/DataDestinationConfig:connectorInstanceId", - "automation/DataDestinationConfig:externalResource", - "automation/DataDestinationConfig:mapping", - "automation/DataDestinationConfig:matchKey", - "automation/DataDestinationConfig:object", - "automation/DataDestinationConfig:operation", - "automation/DataSourceConfig:connectorInstanceId", - "automation/DataSourceConfig:externalResource", - "automation/DataSourceConfig:fields", - "automation/DataSourceConfig:filters", - "automation/DataSourceConfig:object", - "automation/DataSyncConfig:audit", - "automation/DataSyncConfig:batchSize", - "automation/DataSyncConfig:changeTrackingField", - "automation/DataSyncConfig:conflictResolution", - "automation/DataSyncConfig:description", - "automation/DataSyncConfig:destination", - "automation/DataSyncConfig:direction", - "automation/DataSyncConfig:enabled", - "automation/DataSyncConfig:errorHandling", - "automation/DataSyncConfig:label", - "automation/DataSyncConfig:metadata", - "automation/DataSyncConfig:name", - "automation/DataSyncConfig:optimization", - "automation/DataSyncConfig:retry", - "automation/DataSyncConfig:schedule", - "automation/DataSyncConfig:source", - "automation/DataSyncConfig:syncMode", - "automation/DataSyncConfig:tags", - "automation/DataSyncConfig:validation", "automation/DecisionCondition:expression", "automation/DecisionCondition:label", "automation/DecisionConfig:conditions", @@ -2505,15 +2475,6 @@ "automation/SubflowConfig:flowName", "automation/SubflowConfig:input", "automation/SubflowConfig:outputVariable", - "automation/SyncExecutionResult:completedAt", - "automation/SyncExecutionResult:durationMs", - "automation/SyncExecutionResult:errors", - "automation/SyncExecutionResult:id", - "automation/SyncExecutionResult:logs", - "automation/SyncExecutionResult:startedAt", - "automation/SyncExecutionResult:stats", - "automation/SyncExecutionResult:status", - "automation/SyncExecutionResult:syncName", "automation/TimeRelativeTrigger:dateField", "automation/TimeRelativeTrigger:filter", "automation/TimeRelativeTrigger:maxRecords", diff --git a/packages/spec/docs/SYNC_ARCHITECTURE.md b/packages/spec/docs/SYNC_ARCHITECTURE.md index 44298a91fa..80387680fd 100644 --- a/packages/spec/docs/SYNC_ARCHITECTURE.md +++ b/packages/spec/docs/SYNC_ARCHITECTURE.md @@ -1,92 +1,53 @@ # Data Synchronization Architecture -ObjectStack implements a **3-layer architecture** for data synchronization and integration, designed to serve different audiences and use cases. +ObjectStack implements a **2-layer architecture** for data synchronization and integration, designed to serve different audiences and use cases. + +> **History note (v17):** this document used to describe a 3-layer architecture whose +> first layer — **"L1: Simple Sync"** (`DataSyncConfig` in `automation/sync.zod.ts`) — +> was retired in #4738. See [Retired: L1 Simple Sync](#retired-l1-simple-sync-v17) +> for what happened and what to use instead. The historical L2/L3 numbering is kept +> in the level headings so older references stay legible. ## Overview | Level | Protocol | File | Audience | Use Case | Complexity | |-------|----------|------|----------|----------|------------| -| **L1: Simple Sync** | `DataSyncConfig` | `automation/sync.zod.ts` | Business users | Sync Salesforce to Google Sheets | ⭐ Simple | | **L2: ETL Pipeline** | `ETLPipeline` | `automation/etl.zod.ts` | Data engineers | Aggregate 10 sources to data warehouse | ⭐⭐ Moderate | | **L3: Enterprise Connector** | `Connector` | `integration/connector.zod.ts` | System integrators | Full SAP integration with advanced features | ⭐⭐⭐ Advanced | --- -## Level 1: Simple Sync - -**File:** `packages/spec/src/automation/sync.zod.ts` -**Audience:** Business users, citizen developers -**Complexity:** ⭐ Simple - -### Purpose +## Retired: L1 Simple Sync (v17) -Simple, user-friendly synchronization between two systems. Designed for business users who need straightforward data sync without writing code. +**Removed in:** #4738 (dual-source ledger #4535, clusters C13+C15) +**Was:** `DataSyncConfig` + `ConflictResolution` + the `Sync` factory in `packages/spec/src/automation/sync.zod.ts` -### Key Features +The L1 layer was **narrative-only**: no engine ever parsed, scheduled or executed a +`DataSyncConfig` — the schema had zero importers across objectstack, cloud and +objectui, was unreachable from the metadata-type roots (#4650 gate), and existed +solely in this document's 3-layer story. Keeping a documented authoring surface +that nothing enforces is exactly the `declared ≠ enforced` gap Prime Directive #10 +forbids, and its `DataSyncConfig` / `ConflictResolution` names collided with the +live declarations in `integration/connector.zod.ts` and `ui/offline.zod.ts` (the +#4411 dual-source trap). -- ✅ Bidirectional or unidirectional sync (push/pull) -- ✅ Simple field mappings (name mapping only) -- ✅ Basic filters -- ✅ Scheduled or real-time sync -- ❌ NO complex transformations (use ETL for that) -- ❌ NO multi-source joins (use ETL for that) +**What to use instead:** -### Use Cases - -1. **CRM Integration** - Sync contacts between ObjectStack and Salesforce -2. **Marketing Automation** - Push leads to HubSpot -3. **Data Export** - Sync orders to Google Sheets for reporting - -### Example - -```typescript -import { DataSyncConfig } from '@objectstack/spec/automation'; - -const salesforceContactSync: DataSyncConfig = { - name: 'salesforce_contact_sync', - label: 'Salesforce Contact Sync', - - // Source: ObjectStack - source: { - object: 'contact', - filters: { status: 'active' } - }, - - // Destination: Salesforce - destination: { - connectorInstanceId: 'salesforce_production', - externalResource: 'Contact', - operation: 'upsert', - mapping: { - first_name: 'FirstName', - last_name: 'LastName', - email: 'Email', - phone: 'Phone' - }, - matchKey: ['email'] - }, - - direction: 'bidirectional', - syncMode: 'incremental', - conflictResolution: 'latest_wins', - schedule: '0 * * * *', // Hourly - enabled: true -}; -``` - -### Best Practices - -- Use for **single-source to single-destination** sync -- Keep mappings **simple** (field renaming only) -- Use **incremental mode** for large datasets -- Set appropriate **conflict resolution** strategy +- **Connector-attached sync** — `ConnectorSchema.syncConfig` + (`integration/connector.zod.ts`): the live, parsed sync-strategy surface + (strategy, direction, schedule, `conflictResolution`, batching, delete mode). +- **Transformation pipelines** — `ETLPipeline` (`automation/etl.zod.ts`) for + multi-source, multi-stage data movement. +- **Client offline sync** — `SyncConfigSchema` / `ConflictResolution` + (`ui/offline.zod.ts`): a *different* concept (client/server conflict handling) + that now owns the bare `ConflictResolution` name package-wide. --- ## Level 2: ETL Pipeline -**File:** `packages/spec/src/automation/etl.zod.ts` -**Audience:** Data engineers, analytics teams +**File:** `packages/spec/src/automation/etl.zod.ts` +**Audience:** Data engineers, analytics teams **Complexity:** ⭐⭐ Moderate ### Purpose @@ -117,7 +78,7 @@ import { ETLPipeline } from '@objectstack/spec/automation'; const dataWarehousePipeline: ETLPipeline = { name: 'customer_360_pipeline', label: 'Customer 360 Data Warehouse Pipeline', - + // Extract from Salesforce source: { type: 'api', @@ -130,7 +91,7 @@ const dataWarehousePipeline: ETLPipeline = { cursorField: 'LastModifiedDate' } }, - + // Transform: Join with support tickets, aggregate metrics transformations: [ { @@ -158,7 +119,7 @@ const dataWarehousePipeline: ETLPipeline = { } } ], - + // Load to Snowflake destination: { type: 'warehouse', @@ -171,7 +132,7 @@ const dataWarehousePipeline: ETLPipeline = { writeMode: 'upsert', primaryKey: ['customer_id'] }, - + syncMode: 'incremental', schedule: '0 2 * * *', // Daily at 2 AM enabled: true @@ -205,8 +166,8 @@ const dataWarehousePipeline: ETLPipeline = { ## Level 3: Enterprise Connector -**File:** `packages/spec/src/integration/connector.zod.ts` -**Audience:** System integrators, enterprise architects +**File:** `packages/spec/src/integration/connector.zod.ts` +**Audience:** System integrators, enterprise architects **Complexity:** ⭐⭐⭐ Advanced ### Purpose @@ -220,7 +181,7 @@ Complete, production-grade integration with external systems. Includes authentic - ✅ **Rate Limiting**: Token bucket, leaky bucket algorithms - ✅ **Retry Policies**: Exponential backoff, circuit breaker - ✅ **Field Mapping**: With transformations and data type conversion -- ✅ **Conflict Resolution**: Multiple strategies +- ✅ **Conflict Resolution**: Multiple strategies (`ConnectorConflictResolution`) - ✅ **Security**: Signature verification, encryption - ✅ **Monitoring**: Health checks, metrics, logging @@ -241,7 +202,7 @@ const sapConnector: Connector = { label: 'SAP ERP Integration', type: 'saas', description: 'Enterprise-grade SAP ERP integration', - + // OAuth2 Authentication authentication: { type: 'oauth2', @@ -251,7 +212,7 @@ const sapConnector: Connector = { clientSecret: process.env.SAP_CLIENT_SECRET!, scopes: ['read:orders', 'write:orders'] }, - + // Data Sync Configuration syncConfig: { strategy: 'incremental', @@ -263,7 +224,7 @@ const sapConnector: Connector = { batchSize: 1000, deleteMode: 'soft_delete' }, - + // Field Mappings with Transformations fieldMappings: [ { @@ -284,7 +245,7 @@ const sapConnector: Connector = { syncMode: 'bidirectional' } ], - + // Webhooks for Real-time Events webhooks: [ { @@ -302,7 +263,7 @@ const sapConnector: Connector = { isActive: true } ], - + // Rate Limiting rateLimitConfig: { strategy: 'token_bucket', @@ -311,7 +272,7 @@ const sapConnector: Connector = { burstCapacity: 150, respectUpstreamLimits: true }, - + // Retry Configuration retryConfig: { strategy: 'exponential_backoff', @@ -323,7 +284,7 @@ const sapConnector: Connector = { retryOnNetworkError: true, jitter: true }, - + connectionTimeoutMs: 30000, requestTimeoutMs: 60000, status: 'active', @@ -365,20 +326,14 @@ const sapConnector: Connector = { | Do you need real-time webhooks? | **Yes** → L3 (Connector) | | Do you need advanced authentication (OAuth2, SAML)? | **Yes** → L3 (Connector) | | Do you need rate limiting and retry policies? | **Yes** → L3 (Connector) | -| Is it a simple point-to-point sync? | **Yes** → L1 (Simple Sync) | -| Are you a business user with no coding? | **Yes** → L1 (Simple Sync) | +| Is it a simple point-to-point sync with an external system? | **Yes** → L3 (Connector) with `syncConfig` | | Are you building a data warehouse pipeline? | **Yes** → L2 (ETL) | | Are you integrating with an enterprise system? | **Yes** → L3 (Connector) | +| Do you need client-side offline sync? | **Yes** → `ui/offline.zod.ts` (a separate protocol, not this layering) | ### Common Patterns -#### Pattern 1: CRM Sync (L1) -``` -ObjectStack → Simple Sync → Salesforce -``` -Use **L1 Simple Sync** for straightforward bidirectional sync. - -#### Pattern 2: Analytics Pipeline (L2) +#### Pattern 1: Analytics Pipeline (L2) ``` Salesforce → ETL → Transform → Snowflake HubSpot ↗ ↘ Analytics Dashboard @@ -386,19 +341,20 @@ Stripe ↗ ``` Use **L2 ETL Pipeline** for multi-source data warehousing. -#### Pattern 3: Enterprise Integration (L3) +#### Pattern 2: Enterprise Integration (L3) ``` ObjectStack ↔ Enterprise Connector ↔ SAP ↓ Webhooks, Auth, Rate Limiting ``` -Use **L3 Enterprise Connector** for production-grade integrations. +Use **L3 Enterprise Connector** for production-grade integrations — including +straightforward point-to-point sync, via a connector instance with simple `auth` +and a `syncConfig`. -#### Pattern 4: Hybrid Approach +#### Pattern 3: Hybrid Approach ``` External API → L3 Connector → ObjectStack ObjectStack → L2 ETL → Data Warehouse -ObjectStack → L1 Sync → Google Sheets ``` Combine levels for complex scenarios. @@ -406,16 +362,17 @@ Combine levels for complex scenarios. ## Migration Guide -### From L1 to L2 +### From L3 (`syncConfig`) to L2 -When your simple sync needs complex transformations: +When a connector's declarative sync needs complex transformations: -**Before (L1):** +**Before (L3 `syncConfig`):** ```typescript -const sync: DataSyncConfig = { - name: 'order_sync', - source: { object: 'order' }, - destination: { object: 'analytics_order' } +const connector: Connector = { + name: 'orders', + type: 'saas', + authentication: { type: 'api-key', ... }, + syncConfig: { strategy: 'incremental', direction: 'import' } }; ``` @@ -423,7 +380,7 @@ const sync: DataSyncConfig = { ```typescript const pipeline: ETLPipeline = { name: 'order_analytics_pipeline', - source: { type: 'object', config: { object: 'order' } }, + source: { type: 'api', connector: 'orders' }, transformations: [ { type: 'aggregate', config: { groupBy: ['customer_id'] } } ], @@ -455,11 +412,6 @@ const connector: Connector = { ## API Reference -### Level 1: Simple Sync -- [DataSyncConfig Schema](../src/automation/sync.zod.ts) -- [Field Mapping](../src/automation/sync.zod.ts#L97) -- [Sync Execution Result](../src/automation/sync.zod.ts#L380) - ### Level 2: ETL Pipeline - [ETLPipeline Schema](../src/automation/etl.zod.ts) - [ETL Transformations](../src/automation/etl.zod.ts#L151) diff --git a/packages/spec/dual-source-exports.baseline.json b/packages/spec/dual-source-exports.baseline.json index 6b3e88cf66..9d5f137d71 100644 --- a/packages/spec/dual-source-exports.baseline.json +++ b/packages/spec/dual-source-exports.baseline.json @@ -2,10 +2,6 @@ "_comment": "Accepted cross-entry DUAL-SOURCE exports of @objectstack/spec (#4446): names that two or more public entry points export for DIFFERENT declarations, so which type a consumer gets depends on the import path — the #4411 trap. Shrink-only ratchet, judged by symbol identity (a re-export of one declaration from many entries is fine and not listed). A NEW name here fails check:dual-source-exports: converge on one declaration and re-export it, or rename one side — growing this list needs maintainer sign-off and shows up as this file in the diff. An entry that stops being dual-source fails until its line is deleted. Regenerate with: tsx scripts/check-dual-source-exports.ts --update (after pnpm build).", "entries": [ "ActionLocationSchema — [./studio (const)] ≠ [./ui (const)]", - "ConflictResolution — [./automation (type)] ≠ [./integration (type)] ≠ [./ui (type)]", - "ConflictResolutionSchema — [./automation (const)] ≠ [./integration (const)] ≠ [./ui (const)]", - "DataSyncConfig — [./automation (type)] ≠ [./integration (type)]", - "DataSyncConfigSchema — [./automation (const)] ≠ [./integration (const)]", "EnvironmentArtifact — [./cloud (type)] ≠ [./system (type)]", "EnvironmentArtifactInput — [./cloud (type)] ≠ [./system (type)]", "EnvironmentArtifactSchema — [./cloud (const)] ≠ [./system (const)]", diff --git a/packages/spec/json-schema.manifest.json b/packages/spec/json-schema.manifest.json index b1a87203ef..774c112295 100644 --- a/packages/spec/json-schema.manifest.json +++ b/packages/spec/json-schema.manifest.json @@ -514,11 +514,7 @@ "automation/BpmnVersion", "automation/Checkpoint", "automation/ConcurrencyPolicy", - "automation/ConflictResolution", "automation/CreateRecordConfig", - "automation/DataDestinationConfig", - "automation/DataSourceConfig", - "automation/DataSyncConfig", "automation/DecisionCondition", "automation/DecisionConfig", "automation/DecisionOutputDef", @@ -567,10 +563,6 @@ "automation/StateMachine", "automation/StateNode", "automation/SubflowConfig", - "automation/SyncDirection", - "automation/SyncExecutionResult", - "automation/SyncExecutionStatus", - "automation/SyncMode", "automation/TimeRelativeTrigger", "automation/Transition", "automation/TryCatchConfig", @@ -867,9 +859,9 @@ "identity/User", "identity/VerificationToken", "integration/CircuitBreakerConfig", - "integration/ConflictResolution", "integration/Connector", "integration/ConnectorAction", + "integration/ConnectorConflictResolution", "integration/ConnectorErrorCategory", "integration/ConnectorFieldMapping", "integration/ConnectorHealth", diff --git a/packages/spec/scripts/lib/renamed-defs.ts b/packages/spec/scripts/lib/renamed-defs.ts index 911e706350..07d61ce29f 100644 --- a/packages/spec/scripts/lib/renamed-defs.ts +++ b/packages/spec/scripts/lib/renamed-defs.ts @@ -68,6 +68,17 @@ export const RENAMED_DEFS: Readonly> = { // target here. 'integration/FieldMapping': 'integration/ConnectorFieldMapping', // 7 keys carried 'data/FieldMapping': 'data/ImportFieldMapping', // 4 keys carried + + // #4738 / ADR-0112 D9a — `ConflictResolution` was published by THREE defs at + // once (automation ≠ integration ≠ ui, three disjoint vocabularies). The + // connector-sync side takes the domain prefix (0-key carry: it is an enum + // def, no authorable properties). The automation side was retired outright + // with the rest of `automation/sync.zod.ts` in the same change (deliberate + // manifest removal, NOT carried here — a real retirement must never ride the + // rename table). `ui/ConflictResolution` keeps the bare name: it is a + // distinct concept (client/server offline sync) and the only side with + // cross-repo consumers (objectui useOffline + re-export + parity ratchet). + 'integration/ConflictResolution': 'integration/ConnectorConflictResolution', }; /** diff --git a/packages/spec/src/automation/etl.zod.ts b/packages/spec/src/automation/etl.zod.ts index c5934f876e..db56d8e030 100644 --- a/packages/spec/src/automation/etl.zod.ts +++ b/packages/spec/src/automation/etl.zod.ts @@ -8,10 +8,11 @@ import { CronExpressionInputSchema } from '../shared/expression.zod'; * * Inspired by modern data integration platforms like Airbyte, Fivetran, and Apache NiFi. * - * **Positioning in 3-Layer Architecture:** - * - **L1: Simple Sync** (automation/sync.zod.ts) - Business users - Sync Salesforce to Sheets - * - **L2: ETL Pipeline** (THIS FILE) - Data engineers - Aggregate 10 sources to warehouse - * - **L3: Enterprise Connector** (integration/connector.zod.ts) - System integrators - Full SAP integration + * **Positioning in the sync/integration layering** (L1 "Simple Sync" was + * retired in #4738 — narrative-only, zero consumers; see + * `packages/spec/docs/SYNC_ARCHITECTURE.md`): + * - **ETL Pipeline** (THIS FILE) - Data engineers - Aggregate 10 sources to warehouse + * - **Enterprise Connector** (integration/connector.zod.ts) - System integrators - Full SAP integration; connector-attached sync via `syncConfig` * * ETL pipelines enable automated data synchronization between systems, transforming * data as it moves from source to destination. @@ -32,14 +33,10 @@ import { CronExpressionInputSchema } from '../shared/expression.zod'; * - Multi-region databases → Consolidated reporting * - Legacy system migration with transformation * - * **When to downgrade:** - * - Simple 1:1 sync → Use {@link file://./sync.zod.ts | Simple Sync} - * * **When to upgrade:** * - Need full connector lifecycle (auth, webhooks, rate limits) → Use {@link file://../integration/connector.zod.ts | Enterprise Connector} - * - * @see {@link file://./sync.zod.ts} for Level 1 (simple sync) - * @see {@link file://../integration/connector.zod.ts} for Level 3 (enterprise integration) + * + * @see {@link file://../integration/connector.zod.ts} for the Enterprise Connector layer * * ## Use Cases * diff --git a/packages/spec/src/automation/index.ts b/packages/spec/src/automation/index.ts index f44fe4518a..1821c74908 100644 --- a/packages/spec/src/automation/index.ts +++ b/packages/spec/src/automation/index.ts @@ -22,7 +22,15 @@ export * from './etl.zod'; // DeclarativeConnectorEntrySchema. One capability, one contract // (Prime Directive #12); the #4480 template cluster fell the same way. export * from './time-relative-trigger.zod'; -export * from './sync.zod'; +// `sync.zod.ts` (L1 "Simple Sync": DataSyncConfig, its ConflictResolution enum +// and the Sync factory) was removed here (#4738, ledger #4535 C13+C15). The L1 +// layer was narrative-only — zero importers across objectstack / cloud / +// objectui, no engine ever parsed or executed a DataSyncConfig, and the def was +// unreachable from the metadata-type roots (#4650 gate). Connector-attached +// sync config is `ConnectorSchema.syncConfig` (integration/connector.zod.ts, +// the live parse path); multi-step transformation is `etl.zod.ts`. The bare +// `ConflictResolution` name now belongs solely to `@objectstack/spec/ui` +// (offline sync), which objectui consumes. export * from './state-machine.zod'; export * from './node-executor.zod'; export * from './flow-node-expression-paths'; diff --git a/packages/spec/src/automation/sync-retirement.test.ts b/packages/spec/src/automation/sync-retirement.test.ts new file mode 100644 index 0000000000..c413d8c2e2 --- /dev/null +++ b/packages/spec/src/automation/sync-retirement.test.ts @@ -0,0 +1,230 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, it, expect } from 'vitest'; + +// ─── [#4738] L1 "Simple Sync" is gone; `ConflictResolution` has ONE owner ─── +// +// Dual-source ledger #4535, clusters C13+C15 (one implementation unit — the +// two clusters shared their source files). Before this change: +// +// `DataSyncConfig(Schema)` — ./automation ≠ ./integration (two disjoint +// encodings of "sync with an external system": push/pull vs +// import/export/bidirectional, batchSize default 100 vs 1000, …) +// `ConflictResolution(Schema)` — ./automation ≠ ./integration ≠ ./ui (THREE +// declarations, three vocabularies: destination_wins+merge vs target_wins +// vs client_wins/server_wins/last_write_wins) +// +// Which type a consumer got depended on the import path — the #4411 trap. +// Resolution (maintainer-ruled, #4738): +// +// - automation/sync.zod.ts was REMOVED whole (the "L1 Simple Sync" layer: +// DataSyncConfig, its ConflictResolution enum, SyncDirection/SyncMode, +// DataSource/DataDestinationConfig, SyncExecutionStatus/Result, `Sync`). +// It was narrative-only: zero importers in objectstack / cloud / objectui, +// no engine ever parsed a DataSyncConfig, defs unreachable from the +// metadata-type roots (#4650 gate). +// - integration's enum was RENAMED `ConnectorConflictResolution(Schema)` +// (ADR-0112 D9a prefixing, RENAMED_DEFS carry). `DataSyncConfig` stays +// integration-owned under its bare name — it is on the live parse path +// (`ConnectorSchema.syncConfig`). +// - ui keeps the bare `ConflictResolution(Schema)` UNTOUCHED: a distinct +// concept (offline client/server sync) and the only side with cross-repo +// consumers (objectui useOffline.ts + types re-export + a parity ratchet +// that pins "must stay a spec export"). Renaming the ui side would replay +// the objectui#3235 downstream breakage — that "tidy-up" is the wrong-case +// this pin exists to catch. +// - `@objectstack/spec/api`'s `ConflictResolutionStrategy` (route conflicts) +// is a FOURTH relative under a different name; it is outside the baseline +// and must not be touched by any of this. +// +// #4642 established that a compile-time conditional-type pin in this package +// is a no-op (tsconfig excludes `**/*.test.ts`; vitest never enables +// `typecheck`), so the load-bearing pin is the compiler-API test below, with +// anti-vacuity guards; sabotage-verified in the PR (re-adding an automation +// export, re-introducing a bare-name re-export on ./integration, and renaming +// the ui side each turn it red). +describe('[#4738] sync/conflict dual-source retirement', () => { + it('resolves the export surface: one owner per name, across every public entry', async () => { + const ts = (await import('typescript')).default; + const { resolve, relative } = await import('node:path'); + const { dirname } = await import('node:path'); + const { fileURLToPath } = await import('node:url'); + const { readFileSync } = await import('node:fs'); + + const specDir = resolve(dirname(fileURLToPath(import.meta.url)), '../..'); + // Every public entry point, read from package.json's exports map so a + // future entry cannot silently escape the uniqueness pins below. + const pkg = JSON.parse(readFileSync(resolve(specDir, 'package.json'), 'utf8')) as { + exports: Record; + }; + const entries: Record = {}; + for (const sub of Object.keys(pkg.exports)) { + if (sub === '.') entries[sub] = resolve(specDir, 'src/index.ts'); + else if (/^\.\/[a-z-]+$/.test(sub)) entries[sub] = resolve(specDir, `src/${sub.slice(2)}/index.ts`); + // './openapi.json' / './package.json' are not TypeScript entry points. + } + // Anti-vacuity: the enumeration must have found the real surface. + for (const needed of ['./automation', './integration', './ui', './api']) { + expect(Object.keys(entries), `exports map must include ${needed}`).toContain(needed); + } + expect(Object.keys(entries).length).toBeGreaterThan(10); + + const program = ts.createProgram(Object.values(entries), { + module: ts.ModuleKind.ESNext, + moduleResolution: ts.ModuleResolutionKind.Bundler, + skipLibCheck: true, + noEmit: true, + }); + const checker = program.getTypeChecker(); + const unalias = (s: import('typescript').Symbol) => + s.getFlags() & ts.SymbolFlags.Alias ? checker.getAliasedSymbol(s) : s; + + const exportsOf = (sub: string) => { + const sf = program.getSourceFile(entries[sub]); + const moduleSym = sf && checker.getSymbolAtLocation(sf); + // Without this guard a resolution failure would make every assertion + // below pass vacuously — the exact way a gate goes dormant (#4642). + expect(moduleSym, `${sub} module symbol must resolve`).toBeTruthy(); + return checker.getExportsOfModule(moduleSym!); + }; + + const originOf = (sym: import('typescript').Symbol, label: string) => { + const decl = unalias(sym).declarations?.[0]; + expect(decl, `${label} must have a declaration`).toBeTruthy(); + const declFile = decl!.getSourceFile(); + return `${relative(specDir, declFile.fileName)}:${ + declFile.getLineAndCharacterOfPosition(decl!.getStart()).line + 1 + }`; + }; + + /** Every entry that exports `name`, with each occurrence's declaration origin. */ + const holdersOf = (name: string) => { + const out: Array<{ sub: string; origin: string }> = []; + for (const sub of Object.keys(entries)) { + for (const sym of exportsOf(sub).filter((e) => e.getName() === name)) { + out.push({ sub, origin: originOf(sym, `${sub} ${name}`) }); + } + } + return out; + }; + + // 1. The removed side: `./automation` still has a non-trivial surface — + // so the `not.toContain` cannot pass by resolving nothing — and names + // NOTHING from the retired L1 file, while surviving neighbours stand. + const automationNames = exportsOf('./automation').map((e) => e.getName()); + expect(automationNames.length, './automation must export a non-trivial surface').toBeGreaterThan(50); + for (const retired of [ + 'DataSyncConfig', 'DataSyncConfigSchema', + 'ConflictResolution', 'ConflictResolutionSchema', + 'SyncDirection', 'SyncDirectionSchema', + 'SyncMode', 'SyncModeSchema', + 'DataSourceConfig', 'DataSourceConfigSchema', + 'DataDestinationConfig', 'DataDestinationConfigSchema', + 'SyncExecutionStatus', 'SyncExecutionStatusSchema', + 'SyncExecutionResult', 'SyncExecutionResultSchema', + 'Sync', + ]) { + expect(automationNames, `./automation must not export ${retired}`).not.toContain(retired); + } + expect(automationNames).toContain('ETLPipelineSchema'); + expect(automationNames).toContain('StateMachineSchema'); + + // 2. The renamed side: `ConnectorConflictResolution(Schema)` originates in + // integration/connector.zod.ts and is exported by ./integration alone + // (plus nothing else — the rename must not fan out). + for (const name of ['ConnectorConflictResolution', 'ConnectorConflictResolutionSchema']) { + const holders = holdersOf(name); + expect(holders.length, `${name} must be exported (by ./integration)`).toBeGreaterThan(0); + for (const h of holders) { + expect(h.sub, `${name} must only be exported by ./integration`).toBe('./integration'); + expect(h.origin).toMatch(/^src\/integration\/connector\.zod\.ts:\d+$/); + } + } + + // 3. The bare `ConflictResolution(Schema)` now has exactly ONE owner: ./ui, + // declared in ui/offline.zod.ts. Not just "same declaration everywhere" + // — NO other entry may export the bare name at all. A re-export from + // ./integration or ./automation would share the declaration (green to + // the dual-source gate) while telling connector authors the offline + // client/server vocabulary is a connector sync strategy — the C14 + // lesson: a re-export can lie about the domain even when the symbol is + // honest. + for (const name of ['ConflictResolution', 'ConflictResolutionSchema']) { + const holders = holdersOf(name); + expect(holders.map((h) => h.sub), `${name} must be owned by ./ui alone`).toEqual(['./ui']); + expect(holders[0].origin).toMatch(/^src\/ui\/offline\.zod\.ts:\d+$/); + } + + // 4. `DataSyncConfig(Schema)` likewise: ./integration alone, declared in + // integration/connector.zod.ts — it kept its bare name because it is on + // the live `ConnectorSchema.syncConfig` parse path. + for (const name of ['DataSyncConfig', 'DataSyncConfigSchema']) { + const holders = holdersOf(name); + expect(holders.map((h) => h.sub), `${name} must be owned by ./integration alone`).toEqual(['./integration']); + expect(holders[0].origin).toMatch(/^src\/integration\/connector\.zod\.ts:\d+$/); + } + + // 5. The fourth relative is untouched: `ConflictResolutionStrategy` (route + // conflict handling) still exists on ./api under its own distinct name, + // and is a DIFFERENT declaration from ui's ConflictResolution. objectui's + // parity ratchet (offline-nav-performance-spec-parity.test.ts) pins the + // same pair from the consumer side. + const strategyHolders = holdersOf('ConflictResolutionStrategy'); + expect(strategyHolders.length, './api must still export ConflictResolutionStrategy').toBeGreaterThan(0); + expect(strategyHolders.map((h) => h.sub)).toContain('./api'); + const uiOrigin = holdersOf('ConflictResolution')[0].origin; + for (const h of strategyHolders) { + expect(h.origin, 'ConflictResolutionStrategy must not collapse into the ui declaration').not.toBe(uiOrigin); + } + }); + + it('keeps the runtime namespaces consistent with the compiler view', async () => { + const automation = await import('./index'); + const integration = await import('../integration/index'); + const ui = await import('../ui/index'); + + // Removed side — gone at runtime too. + for (const retired of ['DataSyncConfigSchema', 'ConflictResolutionSchema', 'Sync', 'SyncDirectionSchema']) { + expect(retired in automation, `automation must not export ${retired}`).toBe(false); + } + // Anti-vacuity: the namespace we just probed is real and non-trivial. + expect('FlowSchema' in automation).toBe(true); + + // Renamed side — the connector vocabulary, byte-for-byte unchanged. + expect('ConflictResolutionSchema' in integration).toBe(false); + expect('ConnectorConflictResolutionSchema' in integration).toBe(true); + expect(() => integration.ConnectorConflictResolutionSchema.parse('target_wins')).not.toThrow(); + expect(() => integration.ConnectorConflictResolutionSchema.parse('latest_wins')).not.toThrow(); + // The retired automation-side vocabulary was disjoint precisely here: + expect(() => integration.ConnectorConflictResolutionSchema.parse('destination_wins')).toThrow(); + expect(() => integration.ConnectorConflictResolutionSchema.parse('merge')).toThrow(); + + // ui side — untouched, and still the offline client/server vocabulary. + expect('ConflictResolutionSchema' in ui).toBe(true); + expect(() => ui.ConflictResolutionSchema.parse('client_wins')).not.toThrow(); + expect(() => ui.ConflictResolutionSchema.parse('last_write_wins')).not.toThrow(); + expect(() => ui.ConflictResolutionSchema.parse('target_wins')).toThrow(); + expect(() => ui.ConflictResolutionSchema.parse('source_wins')).toThrow(); + }); + + it('still parses authored connector syncConfig through the renamed enum — the live path', async () => { + const { ConnectorSchema } = await import('../integration/connector.zod'); + const connectorWith = (conflictResolution: string) => ({ + name: 'sap_erp', + label: 'SAP ERP', + type: 'saas', + syncConfig: { + strategy: 'incremental', + direction: 'bidirectional', + conflictResolution, + batchSize: 500, + }, + }); + const parsed = ConnectorSchema.parse(connectorWith('target_wins')); + expect(parsed.syncConfig?.conflictResolution).toBe('target_wins'); + // The authored VALUE domain did not move an inch with the TS rename; the + // SAME document differing only in this one value stays illegal (so this + // negative cannot pass for an unrelated reason): + expect(() => ConnectorSchema.parse(connectorWith('destination_wins'))).toThrow(); + }); +}); diff --git a/packages/spec/src/automation/sync.test.ts b/packages/spec/src/automation/sync.test.ts deleted file mode 100644 index e83ce58537..0000000000 --- a/packages/spec/src/automation/sync.test.ts +++ /dev/null @@ -1,320 +0,0 @@ -import { describe, it, expect } from 'vitest'; -import { - SyncDirectionSchema, - SyncModeSchema, - ConflictResolutionSchema, - DataSourceConfigSchema, - DataDestinationConfigSchema, - DataSyncConfigSchema, - SyncExecutionStatusSchema, - SyncExecutionResultSchema, - Sync, -} from './sync.zod'; - -describe('SyncDirectionSchema', () => { - it('should accept all valid directions', () => { - ['push', 'pull', 'bidirectional'].forEach(d => { - expect(() => SyncDirectionSchema.parse(d)).not.toThrow(); - }); - }); - - it('should reject invalid direction', () => { - expect(() => SyncDirectionSchema.parse('both')).toThrow(); - }); -}); - -describe('SyncModeSchema', () => { - it('should accept all valid modes', () => { - ['full', 'incremental', 'realtime'].forEach(m => { - expect(() => SyncModeSchema.parse(m)).not.toThrow(); - }); - }); - - it('should reject invalid mode', () => { - expect(() => SyncModeSchema.parse('batch')).toThrow(); - }); -}); - -describe('ConflictResolutionSchema', () => { - it('should accept all valid strategies', () => { - ['source_wins', 'destination_wins', 'latest_wins', 'manual', 'merge'].forEach(s => { - expect(() => ConflictResolutionSchema.parse(s)).not.toThrow(); - }); - }); - - it('should reject invalid strategy', () => { - expect(() => ConflictResolutionSchema.parse('random')).toThrow(); - }); -}); - -describe('DataSourceConfigSchema', () => { - it('should accept empty object (all optional)', () => { - expect(() => DataSourceConfigSchema.parse({})).not.toThrow(); - }); - - it('should accept full source config', () => { - expect(() => DataSourceConfigSchema.parse({ - object: 'contact', - filters: { status: 'active' }, - fields: ['first_name', 'last_name', 'email'], - connectorInstanceId: 'inst-123', - externalResource: 'Contact', - })).not.toThrow(); - }); -}); - -describe('DataDestinationConfigSchema', () => { - it('should accept minimal destination', () => { - expect(() => DataDestinationConfigSchema.parse({ - operation: 'upsert', - })).not.toThrow(); - }); - - it('should accept simple mapping (record)', () => { - expect(() => DataDestinationConfigSchema.parse({ - object: 'account', - operation: 'upsert', - mapping: { first_name: 'FirstName', last_name: 'LastName' }, - matchKey: ['email'], - })).not.toThrow(); - }); - - it('should accept advanced mapping (array of FieldMapping)', () => { - expect(() => DataDestinationConfigSchema.parse({ - object: 'account', - operation: 'insert', - mapping: [ - { source: 'FirstName', target: 'first_name' }, - { source: 'LastName', target: 'last_name', transform: { type: 'cast', targetType: 'string' } }, - ], - })).not.toThrow(); - }); - - it('should reject invalid operation', () => { - expect(() => DataDestinationConfigSchema.parse({ - operation: 'merge_all', - })).toThrow(); - }); - - it('should reject missing operation', () => { - expect(() => DataDestinationConfigSchema.parse({ - object: 'contact', - })).toThrow(); - }); -}); - -describe('DataSyncConfigSchema', () => { - const minimalSync = { - name: 'contact_sync', - source: {}, - destination: { operation: 'upsert' }, - }; - - it('should accept minimal config with defaults', () => { - const result = DataSyncConfigSchema.parse(minimalSync); - expect(result.direction).toBe('push'); - expect(result.syncMode).toBe('incremental'); - expect(result.conflictResolution).toBe('latest_wins'); - expect(result.batchSize).toBe(100); - expect(result.enabled).toBe(true); - }); - - it('should accept full config', () => { - expect(() => DataSyncConfigSchema.parse({ - name: 'full_sync', - label: 'Full Sync', - description: 'A comprehensive sync', - source: { object: 'contact', fields: ['email'] }, - destination: { - object: 'account', - operation: 'upsert', - mapping: { email: 'Email' }, - matchKey: ['email'], - }, - direction: 'bidirectional', - syncMode: 'realtime', - conflictResolution: 'manual', - schedule: '0 * * * *', - enabled: false, - changeTrackingField: 'updated_at', - batchSize: 500, - retry: { maxAttempts: 5, backoffMs: 60000 }, - validation: { - required: ['email'], - unique: ['email'], - custom: [{ name: 'email_check', condition: 'email != null', message: 'Email required' }], - }, - errorHandling: { - onValidationError: 'fail', - onSyncError: 'skip', - notifyOnError: ['admin@example.com'], - }, - optimization: { - parallelBatches: true, - cacheEnabled: false, - compressionEnabled: true, - }, - audit: { - logLevel: 'debug', - retainLogsForDays: 90, - trackChanges: false, - }, - tags: ['crm', 'critical'], - metadata: { priority: 'high' }, - })).not.toThrow(); - }); - - it('should reject invalid name (not snake_case)', () => { - expect(() => DataSyncConfigSchema.parse({ - ...minimalSync, - name: 'ContactSync', - })).toThrow(); - }); - - it('should reject batchSize out of range', () => { - expect(() => DataSyncConfigSchema.parse({ - ...minimalSync, - batchSize: 0, - })).toThrow(); - expect(() => DataSyncConfigSchema.parse({ - ...minimalSync, - batchSize: 20000, - })).toThrow(); - }); - - it('should apply errorHandling defaults', () => { - const result = DataSyncConfigSchema.parse({ - ...minimalSync, - errorHandling: {}, - }); - expect(result.errorHandling?.onValidationError).toBe('skip'); - expect(result.errorHandling?.onSyncError).toBe('retry'); - }); - - it('should apply audit defaults', () => { - const result = DataSyncConfigSchema.parse({ - ...minimalSync, - audit: {}, - }); - expect(result.audit?.logLevel).toBe('info'); - expect(result.audit?.retainLogsForDays).toBe(30); - expect(result.audit?.trackChanges).toBe(true); - }); - - it('should apply optimization defaults', () => { - const result = DataSyncConfigSchema.parse({ - ...minimalSync, - optimization: {}, - }); - expect(result.optimization?.parallelBatches).toBe(false); - expect(result.optimization?.cacheEnabled).toBe(true); - expect(result.optimization?.compressionEnabled).toBe(false); - }); -}); - -describe('SyncExecutionStatusSchema', () => { - it('should accept all valid statuses', () => { - ['pending', 'running', 'completed', 'partial', 'failed', 'cancelled'].forEach(s => { - expect(() => SyncExecutionStatusSchema.parse(s)).not.toThrow(); - }); - }); - - it('should reject invalid status', () => { - expect(() => SyncExecutionStatusSchema.parse('unknown')).toThrow(); - }); -}); - -describe('SyncExecutionResultSchema', () => { - it('should accept minimal result', () => { - expect(() => SyncExecutionResultSchema.parse({ - id: 'run-001', - syncName: 'contact_sync', - status: 'completed', - startedAt: '2024-01-01T00:00:00Z', - })).not.toThrow(); - }); - - it('should accept full result', () => { - expect(() => SyncExecutionResultSchema.parse({ - id: 'run-002', - syncName: 'contact_sync', - status: 'partial', - startedAt: '2024-01-01T00:00:00Z', - completedAt: '2024-01-01T00:05:00Z', - durationMs: 300000, - stats: { - recordsProcessed: 1000, - recordsInserted: 500, - recordsUpdated: 400, - recordsDeleted: 50, - recordsSkipped: 30, - recordsErrored: 20, - conflictsDetected: 5, - conflictsResolved: 3, - }, - errors: [ - { recordId: 'rec-1', field: 'email', message: 'Invalid email', code: 'VALIDATION' }, - ], - logs: ['Started sync', 'Completed with errors'], - })).not.toThrow(); - }); - - it('should reject missing startedAt', () => { - expect(() => SyncExecutionResultSchema.parse({ - id: 'run-003', - syncName: 'test', - status: 'running', - })).toThrow(); - }); - - it('should reject invalid datetime format', () => { - expect(() => SyncExecutionResultSchema.parse({ - id: 'run-004', - syncName: 'test', - status: 'running', - startedAt: 'yesterday', - })).toThrow(); - }); -}); - -describe('Sync factory', () => { - it('should create object-to-object sync', () => { - const config = Sync.objectSync({ - name: 'contact_to_lead', - sourceObject: 'contact', - destObject: 'lead', - mapping: { first_name: 'FirstName' }, - schedule: '0 * * * *', - }); - expect(config.direction).toBe('push'); - expect(config.syncMode).toBe('incremental'); - expect(config.batchSize).toBe(100); - expect(config.enabled).toBe(true); - expect(() => DataSyncConfigSchema.parse(config)).not.toThrow(); - }); - - it('should create connector sync', () => { - const config = Sync.connectorSync({ - name: 'sf_sync', - sourceObject: 'contact', - connectorInstanceId: 'inst-sf', - externalResource: 'Contact', - mapping: { email: 'Email' }, - }); - expect(config.destination.connectorInstanceId).toBe('inst-sf'); - expect(() => DataSyncConfigSchema.parse(config)).not.toThrow(); - }); - - it('should create bidirectional sync', () => { - const config = Sync.bidirectionalSync({ - name: 'bidir_sync', - object: 'account', - connectorInstanceId: 'inst-hub', - externalResource: 'Company', - mapping: { name: 'Name' }, - }); - expect(config.direction).toBe('bidirectional'); - expect(config.destination.operation).toBe('sync'); - expect(() => DataSyncConfigSchema.parse(config)).not.toThrow(); - }); -}); diff --git a/packages/spec/src/automation/sync.zod.ts b/packages/spec/src/automation/sync.zod.ts deleted file mode 100644 index f75a3ea2b2..0000000000 --- a/packages/spec/src/automation/sync.zod.ts +++ /dev/null @@ -1,536 +0,0 @@ -// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. - -import { z } from 'zod'; -import { FieldMappingSchema } from '../shared/mapping.zod'; -import { ExpressionInputSchema, CronExpressionInputSchema } from '../shared/expression.zod'; - -/** - * Data Sync Protocol - LEVEL 1: Simple Synchronization - * - * Inspired by Salesforce Connect, Segment Sync, and Census Reverse ETL. - * - * **Positioning in 3-Layer Architecture:** - * - **L1: Simple Sync** (THIS FILE) - Business users - Sync Salesforce to Sheets - * - **L2: ETL Pipeline** (automation/etl.zod.ts) - Data engineers - Aggregate 10 sources to warehouse - * - **L3: Enterprise Connector** (integration/connector.zod.ts) - System integrators - Full SAP integration - * - * Data sync provides bidirectional or unidirectional data synchronization - * between ObjectStack and external systems, maintaining data consistency - * across platforms. - * - * **SCOPE: Simple field mappings only. NO complex transformations.** - * For complex transformations (joins, aggregates, custom SQL), use ETL Pipeline (Level 2). - * - * ## When to Use This Layer - * - * **Use Simple Sync when:** - * - Syncing 1:1 fields between two systems - * - Simple field transformations (uppercase, cast, etc.) - * - No complex logic required - * - Business users need to configure integrations - * - * **Examples:** - * - Salesforce Contact ↔ Google Sheets - * - HubSpot Company ↔ CRM Account - * - Shopify Orders → Accounting System - * - * **When to upgrade:** - * - Need multi-source joins → Use {@link file://./etl.zod.ts | ETL Pipeline} - * - Need complex authentication/webhooks → Use {@link file://../integration/connector.zod.ts | Enterprise Connector} - * - Need aggregations or data warehousing → Use {@link file://./etl.zod.ts | ETL Pipeline} - * - * @see {@link file://./etl.zod.ts} for Level 2 (data engineering) - * @see {@link file://../integration/connector.zod.ts} for Level 3 (enterprise integration) - * - * ## Use Cases - * - * 1. **CRM Integration** - * - Sync contacts between ObjectStack and Salesforce - * - Keep opportunity data synchronized - * - Bidirectional updates - * - * 2. **Customer Data Platform (CDP)** - * - Sync user profiles to Segment - * - Enrichment data from Clearbit - * - Marketing automation sync - * - * 3. **Operational Analytics** - * - Sync production data to analytics warehouse - * - Real-time dashboards - * - Business intelligence - * - * @see https://help.salesforce.com/s/articleView?id=sf.platform_connect_about.htm - * @see https://segment.com/docs/connections/sync/ - * @see https://www.getcensus.com/ - * - * @example - * ```typescript - * const contactSync: DataSyncConfig = { - * name: 'salesforce_contact_sync', - * label: 'Salesforce Contact Sync', - * source: { - * object: 'contact', - * filters: { status: 'active' } - * }, - * destination: { - * connector: 'salesforce', - * operation: 'upsert_contact', - * mapping: { - * first_name: 'FirstName', - * last_name: 'LastName', - * email: 'Email' - * } - * }, - * syncMode: 'incremental', - * schedule: '0 * * * *' // Hourly - * } - * ``` - */ - -/** - * Sync Direction - */ -import { lazySchema } from '../shared/lazy-schema'; -export const SyncDirectionSchema = lazySchema(() => z.enum([ - 'push', // ObjectStack -> External (one-way) - 'pull', // External -> ObjectStack (one-way) - 'bidirectional', // Both directions -])); - -export type SyncDirection = z.infer; - -/** - * Sync Mode - */ -export const SyncModeSchema = lazySchema(() => z.enum([ - 'full', // Full refresh every time - 'incremental', // Only sync changed records - 'realtime', // Real-time streaming sync -])); - -export type SyncMode = z.infer; - -/** - * Conflict Resolution Strategy - */ -export const ConflictResolutionSchema = lazySchema(() => z.enum([ - 'source_wins', // Source system always wins - 'destination_wins', // Destination system always wins - 'latest_wins', // Most recently modified wins - 'manual', // Flag for manual resolution - 'merge', // Smart merge (custom logic) -])); - -export type ConflictResolution = z.infer; - -/** - * Field Mapping for Data Sync - * - * Uses the canonical field mapping protocol from shared/mapping.zod.ts - * for simple 1:1 field transformations. - * - * @see {@link FieldMappingSchema} for the base field mapping schema - */ - -/** - * Data Source Configuration - */ -export const DataSourceConfigSchema = lazySchema(() => z.object({ - /** - * Source object name - * For ObjectStack objects - */ - object: z.string().optional().describe('ObjectStack object name'), - - /** - * Filter conditions - * Only sync records matching these filters - */ - filters: z.unknown().optional().describe('Filter conditions'), - - /** - * Fields to include - * If not specified, all fields are synced - */ - fields: z.array(z.string()).optional().describe('Fields to sync'), - - /** - * External connector instance ID - * For external data sources - */ - connectorInstanceId: z.string().optional().describe('Connector instance ID'), - - /** - * External resource identifier - * e.g., Salesforce object name, database table, API endpoint - */ - externalResource: z.string().optional().describe('External resource ID'), -})); - -export type DataSourceConfig = z.infer; - -/** - * Data Destination Configuration - */ -export const DataDestinationConfigSchema = lazySchema(() => z.object({ - /** - * Destination object name - * For ObjectStack objects - */ - object: z.string().optional().describe('ObjectStack object name'), - - /** - * Connector instance ID - * For external destinations - */ - connectorInstanceId: z.string().optional().describe('Connector instance ID'), - - /** - * Operation to perform - */ - operation: z.enum([ - 'insert', // Create new records only - 'update', // Update existing records only - 'upsert', // Insert or update based on key - 'delete', // Delete records - 'sync', // Full synchronization - ]).describe('Sync operation'), - - /** - * Field mappings - * Maps source fields to destination fields - */ - mapping: z.union([ - z.record(z.string(), z.string()), // Simple mapping: { sourceField: 'destField' } - z.array(FieldMappingSchema), // Advanced mapping with transformations - ]).optional().describe('Field mappings'), - - /** - * External resource identifier - */ - externalResource: z.string().optional().describe('External resource ID'), - - /** - * Match key for upsert operations - * Fields to use for matching existing records - */ - matchKey: z.array(z.string()).optional().describe('Match key fields'), -})); - -export type DataDestinationConfig = z.infer; - -/** - * Data Sync Configuration Schema - * - * Complete definition of a data synchronization between systems. - */ -export const DataSyncConfigSchema = lazySchema(() => z.object({ - /** - * Sync configuration name (snake_case) - */ - name: z.string() - .regex(/^[a-z_][a-z0-9_]*$/) - .describe('Sync configuration name (snake_case)'), - - /** - * Human-readable label - */ - label: z.string().optional().describe('Sync display name'), - - /** - * Description - */ - description: z.string().optional().describe('Sync description'), - - /** - * Source configuration - */ - source: DataSourceConfigSchema.describe('Data source'), - - /** - * Destination configuration - */ - destination: DataDestinationConfigSchema.describe('Data destination'), - - /** - * Sync direction - */ - direction: SyncDirectionSchema.default('push').describe('Sync direction'), - - /** - * Sync mode - */ - syncMode: SyncModeSchema.default('incremental').describe('Sync mode'), - - /** - * Conflict resolution strategy - */ - conflictResolution: ConflictResolutionSchema - .default('latest_wins') - .describe('Conflict resolution'), - - /** - * Execution schedule (cron expression) - * For scheduled syncs - * - * @example "0 * * * *" - Hourly - * @example "*\/15 * * * *" - Every 15 minutes - */ - schedule: CronExpressionInputSchema.optional().describe('Cron schedule'), - - /** - * Whether sync is enabled - */ - enabled: z.boolean().default(true).describe('Sync enabled'), - - /** - * Change tracking field - * Field to track when records were last modified - * Used for incremental sync - * - * @example "updated_at", "modified_date" - */ - changeTrackingField: z.string() - .optional() - .describe('Field for change tracking'), - - /** - * Batch size - * Number of records to process per batch - */ - batchSize: z.number().int().min(1).max(10000) - .default(100) - .describe('Batch size for processing'), - - /** - * Retry configuration - */ - retry: z.object({ - maxAttempts: z.number().int().min(0).default(3).describe('Max retries'), - backoffMs: z.number().int().min(0).default(30000).describe('Backoff duration'), - }).optional().describe('Retry configuration'), - - /** - * Pre-sync validation rules - */ - validation: z.object({ - required: z.array(z.string()).optional().describe('Required fields'), - unique: z.array(z.string()).optional().describe('Unique constraint fields'), - custom: z.array(z.object({ - name: z.string(), - condition: ExpressionInputSchema.describe('Validation predicate (CEL).'), - message: z.string().describe('Error message'), - })).optional().describe('Custom validation rules'), - }).optional().describe('Validation rules'), - - /** - * Error handling configuration - */ - errorHandling: z.object({ - onValidationError: z.enum(['skip', 'fail', 'log']).default('skip'), - onSyncError: z.enum(['skip', 'fail', 'retry']).default('retry'), - notifyOnError: z.array(z.string()).optional().describe('Email notifications'), - }).optional().describe('Error handling'), - - /** - * Performance optimization - */ - optimization: z.object({ - parallelBatches: z.boolean().default(false).describe('Process batches in parallel'), - cacheEnabled: z.boolean().default(true).describe('Enable caching'), - compressionEnabled: z.boolean().default(false).describe('Enable compression'), - }).optional().describe('Performance optimization'), - - /** - * Audit and logging - */ - audit: z.object({ - logLevel: z.enum(['none', 'error', 'warn', 'info', 'debug']).default('info'), - retainLogsForDays: z.number().int().min(1).default(30), - trackChanges: z.boolean().default(true).describe('Track all changes'), - }).optional().describe('Audit configuration'), - - /** - * Tags for organization - */ - tags: z.array(z.string()).optional().describe('Sync tags'), - - /** - * Custom metadata - */ - metadata: z.record(z.string(), z.unknown()).optional().describe('Custom metadata'), -})); - -export type DataSyncConfig = z.infer; - -/** - * Sync Execution Status - */ -export const SyncExecutionStatusSchema = lazySchema(() => z.enum([ - 'pending', // Queued - 'running', // Currently executing - 'completed', // Successfully completed - 'partial', // Completed with some errors - 'failed', // Failed - 'cancelled', // Manually cancelled -])); - -export type SyncExecutionStatus = z.infer; - -/** - * Sync Execution Result Schema - * - * Result of a sync execution. - */ -export const SyncExecutionResultSchema = lazySchema(() => z.object({ - /** - * Execution ID - */ - id: z.string().describe('Execution ID'), - - /** - * Sync configuration name - */ - syncName: z.string().describe('Sync name'), - - /** - * Execution status - */ - status: SyncExecutionStatusSchema.describe('Execution status'), - - /** - * Start timestamp - */ - startedAt: z.string().datetime().describe('Start time'), - - /** - * End timestamp - */ - completedAt: z.string().datetime().optional().describe('Completion time'), - - /** - * Duration in milliseconds - */ - durationMs: z.number().optional().describe('Duration in ms'), - - /** - * Statistics - */ - stats: z.object({ - recordsProcessed: z.number().int().default(0).describe('Total records processed'), - recordsInserted: z.number().int().default(0).describe('Records inserted'), - recordsUpdated: z.number().int().default(0).describe('Records updated'), - recordsDeleted: z.number().int().default(0).describe('Records deleted'), - recordsSkipped: z.number().int().default(0).describe('Records skipped'), - recordsErrored: z.number().int().default(0).describe('Records with errors'), - conflictsDetected: z.number().int().default(0).describe('Conflicts detected'), - conflictsResolved: z.number().int().default(0).describe('Conflicts resolved'), - }).optional().describe('Execution statistics'), - - /** - * Errors encountered - */ - errors: z.array(z.object({ - recordId: z.string().optional().describe('Record ID'), - field: z.string().optional().describe('Field name'), - message: z.string().describe('Error message'), - code: z.string().optional().describe('Error code'), - })).optional().describe('Errors'), - - /** - * Execution logs - */ - logs: z.array(z.string()).optional().describe('Execution logs'), -})); - -export type SyncExecutionResult = z.infer; - -/** - * Helper factory for creating sync configurations - */ -export const Sync = { - /** - * Create a simple object-to-object sync - */ - objectSync: (params: { - name: string; - sourceObject: string; - destObject: string; - mapping: Record; - schedule?: string | import("../shared/expression.zod").Expression; - }): DataSyncConfig => ({ - name: params.name, - source: { - object: params.sourceObject, - }, - destination: { - object: params.destObject, - operation: 'upsert', - mapping: params.mapping, - }, - direction: 'push', - syncMode: 'incremental', - conflictResolution: 'latest_wins', - batchSize: 100, - schedule: typeof params.schedule === 'string' ? { dialect: 'cron' as const, source: params.schedule } : params.schedule, - enabled: true, - }), - - /** - * Create a connector sync - */ - connectorSync: (params: { - name: string; - sourceObject: string; - connectorInstanceId: string; - externalResource: string; - mapping: Record; - schedule?: string | import("../shared/expression.zod").Expression; - }): DataSyncConfig => ({ - name: params.name, - source: { - object: params.sourceObject, - }, - destination: { - connectorInstanceId: params.connectorInstanceId, - externalResource: params.externalResource, - operation: 'upsert', - mapping: params.mapping, - }, - direction: 'push', - syncMode: 'incremental', - conflictResolution: 'latest_wins', - batchSize: 100, - schedule: typeof params.schedule === 'string' ? { dialect: 'cron' as const, source: params.schedule } : params.schedule, - enabled: true, - }), - - /** - * Create a bidirectional sync - */ - bidirectionalSync: (params: { - name: string; - object: string; - connectorInstanceId: string; - externalResource: string; - mapping: Record; - schedule?: string | import("../shared/expression.zod").Expression; - }): DataSyncConfig => ({ - name: params.name, - source: { - object: params.object, - }, - destination: { - connectorInstanceId: params.connectorInstanceId, - externalResource: params.externalResource, - operation: 'sync', - mapping: params.mapping, - }, - direction: 'bidirectional', - syncMode: 'incremental', - conflictResolution: 'latest_wins', - batchSize: 100, - schedule: typeof params.schedule === 'string' ? { dialect: 'cron' as const, source: params.schedule } : params.schedule, - enabled: true, - }), -} as const; diff --git a/packages/spec/src/integration/connector.test.ts b/packages/spec/src/integration/connector.test.ts index 2f7c48d12f..be16927e8d 100644 --- a/packages/spec/src/integration/connector.test.ts +++ b/packages/spec/src/integration/connector.test.ts @@ -7,7 +7,7 @@ import { // Data Sync DataSyncConfigSchema, SyncStrategySchema, - ConflictResolutionSchema, + ConnectorConflictResolutionSchema, // Webhook WebhookConfigSchema, @@ -233,6 +233,18 @@ describe('DataSyncConfigSchema', () => { expect(() => DataSyncConfigSchema.parse({ batchSize: 10001 })).toThrow(); expect(() => DataSyncConfigSchema.parse({ batchSize: 500 })).not.toThrow(); }); + + it('resolves conflicts with the CONNECTOR vocabulary, unchanged by the #4738 rename', () => { + // `ConflictResolution` → `ConnectorConflictResolution` renamed the TS + // export only; the authored value domain is byte-for-byte the same. + (['source_wins', 'target_wins', 'latest_wins', 'manual'] as const).forEach((v) => { + expect(() => ConnectorConflictResolutionSchema.parse(v)).not.toThrow(); + }); + // The retired automation-side vocabulary was disjoint precisely where it + // mattered — these values were never part of the connector strategy: + expect(() => ConnectorConflictResolutionSchema.parse('destination_wins')).toThrow(); + expect(() => ConnectorConflictResolutionSchema.parse('merge')).toThrow(); + }); }); // ============================================================================ @@ -815,8 +827,9 @@ describe('[#4684] RateLimitConfig no longer names two declarations', () => { // // The first two are base-and-superset, so "converge them" is a tempting read. // It is wrong in both directions: widening the base to 7 keys pushes connector -// sync semantics onto `automation/sync.zod.ts` and `data/external-lookup.zod.ts` -// which also extend it, and narrowing the connector side to 4 is a retirement of +// sync semantics onto `data/external-lookup.zod.ts` which also extends it +// (`automation/sync.zod.ts` embedded the base too until its retirement in +// #4738), and narrowing the connector side to 4 is a retirement of // three live keys, not a naming fix. ADR-0112 D9a's prefix remedy applies, and // the file next door already demonstrates it: `data/ExternalFieldMappingSchema` // extends the same base and, purely because it carries a prefix, never entered diff --git a/packages/spec/src/integration/connector.zod.ts b/packages/spec/src/integration/connector.zod.ts index c2090f486a..1e65e5dc08 100644 --- a/packages/spec/src/integration/connector.zod.ts +++ b/packages/spec/src/integration/connector.zod.ts @@ -13,10 +13,11 @@ import { FieldMappingSchema as BaseFieldMappingSchema } from '../shared/mapping. * Connectors enable ObjectStack to sync data with SaaS apps, databases, file storage, * and message queues through a unified protocol. * - * **Positioning in 3-Layer Architecture:** - * - **L1: Simple Sync** (automation/sync.zod.ts) - Business users - Sync Salesforce to Sheets - * - **L2: ETL Pipeline** (automation/etl.zod.ts) - Data engineers - Aggregate 10 sources to warehouse - * - **L3: Enterprise Connector** (THIS FILE) - System integrators - Full SAP integration + * **Positioning in the sync/integration layering** (L1 "Simple Sync" was + * retired in #4738 — narrative-only, zero consumers; see + * `packages/spec/docs/SYNC_ARCHITECTURE.md`): + * - **ETL Pipeline** (automation/etl.zod.ts) - Data engineers - Aggregate 10 sources to warehouse + * - **Enterprise Connector** (THIS FILE) - System integrators - Full SAP integration; connector-attached sync via `syncConfig` * * **SCOPE: Most comprehensive integration layer.** * Includes authentication, webhooks, rate limiting, field mapping, bidirectional sync, @@ -64,11 +65,9 @@ import { FieldMappingSchema as BaseFieldMappingSchema } from '../shared/mapping. * - Microsoft Dynamics 365 connector * * **When to downgrade:** - * - Simple field sync → Use {@link file://../automation/sync.zod.ts | Simple Sync} * - Data transformation only → Use {@link file://../automation/etl.zod.ts | ETL Pipeline} - * - * @see {@link file://../automation/sync.zod.ts} for Level 1 (simple sync) - * @see {@link file://../automation/etl.zod.ts} for Level 2 (data engineering) + * + * @see {@link file://../automation/etl.zod.ts} for the ETL Pipeline layer (data engineering) * * ## There is no "Trigger Registry" alternative * @@ -81,7 +80,9 @@ import { FieldMappingSchema as BaseFieldMappingSchema } from '../shared/mapping. * per-provider template cluster). The same defect class as the * `capabilities.readOnly` prescription #4487 corrected: a signpost must land * somewhere enforced. Lightweight cases are served HERE — a connector instance - * with simple `auth` — or by `automation/sync.zod.ts` / `etl.zod.ts` below. + * with simple `auth` — or by `automation/etl.zod.ts` for transformation + * pipelines. (The automation-side L1 "Simple Sync" file was itself retired as + * a dead end of the same class in #4738.) */ // ============================================================================ @@ -163,16 +164,27 @@ export const SyncStrategySchema = lazySchema(() => z.enum([ export type SyncStrategy = z.infer; /** - * Conflict Resolution Strategy + * Connector Conflict Resolution Strategy + * + * Renamed from `ConflictResolution` (#4738, ADR-0112 D9a — the C9/C12 + * prefixing lineage): that bare name was published by three entry points for + * three different declarations (#4411 trap). The connector-sync strategy takes + * the domain prefix; the bare `ConflictResolution` now belongs solely to + * `@objectstack/spec/ui` (offline client/server sync — a different concept + * with a disjoint vocabulary, and the only side with cross-repo consumers). + * The enum VALUES here are unchanged — authored `syncConfig.conflictResolution` + * metadata parses byte-for-byte the same. Note `@objectstack/spec/api` also + * exports `ConflictResolutionStrategy` (route conflicts) — a fourth, distinct + * name; unrelated to this rename. */ -export const ConflictResolutionSchema = lazySchema(() => z.enum([ +export const ConnectorConflictResolutionSchema = lazySchema(() => z.enum([ 'source_wins', // External system data takes precedence 'target_wins', // ObjectStack data takes precedence 'latest_wins', // Most recently modified wins 'manual', // Flag for manual resolution ]).describe('Conflict resolution strategy')); -export type ConflictResolution = z.infer; +export type ConnectorConflictResolution = z.infer; /** * Data Synchronization Configuration @@ -210,7 +222,7 @@ export const DataSyncConfigSchema = lazySchema(() => z.object({ /** * Conflict resolution strategy */ - conflictResolution: ConflictResolutionSchema.optional().default('latest_wins'), + conflictResolution: ConnectorConflictResolutionSchema.optional().default('latest_wins'), /** * Batch size for bulk operations diff --git a/packages/spec/src/shared/mapping.zod.ts b/packages/spec/src/shared/mapping.zod.ts index 5ff9b77b9e..3fa7e1a405 100644 --- a/packages/spec/src/shared/mapping.zod.ts +++ b/packages/spec/src/shared/mapping.zod.ts @@ -6,14 +6,13 @@ import { ExpressionInputSchema } from './expression.zod'; /** * Base Field Mapping Protocol * - * Shared by: ETL, Sync, Connector, External Lookup - * + * Shared by: ETL, Connector, External Lookup + * * This module provides the canonical field mapping schema used across * ObjectStack for data transformation and synchronization. - * + * * **Use Cases:** * - ETL pipelines (data/mapping.zod.ts) - * - Data synchronization (automation/sync.zod.ts) * - Integration connectors (integration/connector.zod.ts) * - External lookups (data/external-lookup.zod.ts) *