Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
ba1769d
[core][python] Add asynchronous table prediction APIs
XiaoHongbo-Hope Nov 16, 2026
a5ddb2f
[core][python] Clarify prediction idempotency
XiaoHongbo-Hope Nov 16, 2026
e995032
[core][python] Make prediction jobs catalog-scoped
XiaoHongbo-Hope Nov 16, 2026
df909e1
[core][python] Align prediction identifiers
XiaoHongbo-Hope Nov 16, 2026
e5cc6e3
[core][python] Simplify prediction lifecycle metadata
XiaoHongbo-Hope Nov 16, 2026
74534e1
[core][python] Model prediction lifecycle after Databricks
XiaoHongbo-Hope Nov 16, 2026
aa12916
[core][python] Align prediction times with Databricks
XiaoHongbo-Hope Nov 16, 2026
4dbc8eb
[core][python] Make prediction output extensible
XiaoHongbo-Hope Nov 16, 2026
f1576bb
[core][python] Remove unversioned prediction model field
XiaoHongbo-Hope Nov 16, 2026
4655252
[core][python] Infer prediction source identity
XiaoHongbo-Hope Nov 16, 2026
a3c65ba
[core][python] Rename prediction prompt to instructions
XiaoHongbo-Hope Nov 16, 2026
b3642cb
[core][python] Standardize prediction result schema
XiaoHongbo-Hope Nov 16, 2026
94f6cc3
[python] Add prediction result waiter
XiaoHongbo-Hope Nov 16, 2026
4a46915
[python] Document prediction waiter usage
XiaoHongbo-Hope Nov 16, 2026
1d5e2bf
[python] Align prediction waiter with long-running operations
XiaoHongbo-Hope Nov 17, 2026
2588191
[rest] Keep prediction API REST-only
XiaoHongbo-Hope Nov 17, 2026
0c6ecc2
[rest] Reuse Paimon options for prediction time travel
XiaoHongbo-Hope Nov 17, 2026
54c9d0e
[rest] Align prediction validation contract
XiaoHongbo-Hope Nov 17, 2026
36452bd
Make prediction idempotency key optional
XiaoHongbo-Hope Nov 17, 2026
fadd422
Use structured model references for predictions
XiaoHongbo-Hope Nov 17, 2026
b64f350
Reuse identifiers for prediction models
XiaoHongbo-Hope Nov 17, 2026
c4b618b
Complete prediction submission validation
XiaoHongbo-Hope Nov 17, 2026
f0e8d22
Align prediction idempotency tokens
XiaoHongbo-Hope Nov 17, 2026
5344355
Fix prediction REST coverage fixture
XiaoHongbo-Hope Nov 18, 2026
1760f69
Clarify prediction lifecycle semantics
XiaoHongbo-Hope Nov 18, 2026
ed98fae
Fix prediction status indentation
XiaoHongbo-Hope Nov 18, 2026
a099fdc
Restrict prediction output to tables
XiaoHongbo-Hope Nov 18, 2026
85aeefd
Harden prediction wait and error handling
XiaoHongbo-Hope Nov 18, 2026
4b62afa
Enforce prediction polling timeout budget
XiaoHongbo-Hope Nov 18, 2026
fe51bad
Enforce prediction wait deadline
XiaoHongbo-Hope Nov 18, 2026
fdc8b53
Simplify prediction wait polling
XiaoHongbo-Hope Nov 18, 2026
2da4c4e
Map missing prediction databases
XiaoHongbo-Hope Nov 18, 2026
5bbf410
Report the missing prediction database
XiaoHongbo-Hope Nov 18, 2026
8799d0f
Reject prediction dot segments
XiaoHongbo-Hope Nov 18, 2026
0969586
[core] Add executable prediction reference integration
XiaoHongbo-Hope Nov 19, 2026
2e0a1e2
[core] Make prediction reference integration race safe
XiaoHongbo-Hope Nov 19, 2026
36bbd5a
[core] Derive prediction identity from primary keys
XiaoHongbo-Hope Nov 19, 2026
a163bec
[core] Validate derived prediction merge keys
XiaoHongbo-Hope Nov 19, 2026
7f2561d
[docs] Clarify prediction get operation
XiaoHongbo-Hope Nov 19, 2026
f785f4e
fix: harden prediction reference execution
XiaoHongbo-Hope Nov 20, 2026
f84bfd9
fix: reject incompatible prediction scan modes
XiaoHongbo-Hope Nov 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/docs/concepts/rest/rest-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ payloads, and error responses are defined in the OpenAPI specification.
| Branches and tags | Manage named histories and retained snapshots. | Table-scoped `branches` and `tags`. |
| Consumers | List and reset streaming consumer progress. | Table-scoped `consumers`. |
| Views and functions | Manage reusable SQL and function definitions. | Database- and catalog-scoped `views` and `functions`. |
| Prediction jobs | Submit, list, get, and cancel asynchronous predictions. | Catalog-scoped `predictions`; the source table is part of the submission. |

In this table, **table-scoped** means
`/v1/{prefix}/databases/{database}/tables/{table}`. Catalog-wide listing and detail-listing
Expand Down
282 changes: 281 additions & 1 deletion docs/static/rest-catalog-open-api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,102 @@ paths:
"404":
$ref: '#/components/responses/DatabaseOrModelNotExistErrorResponse'
"501": {description: Models are not supported}
/v1/{prefix}/predictions:
post:
tags: [prediction]
summary: Submit a prediction job
operationId: submitPrediction
description: >-
Records the source snapshot and resolves the registered model. Execution reads
the recorded snapshot and fails if it is no longer available.
Reusing an idempotencyToken with the same payload returns the existing job;
conflicting reuse returns 409.
parameters:
- {name: prefix, in: path, required: true, schema: {type: string}}
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/SubmitPredictionRequest'
responses:
"202":
description: Prediction accepted
content:
application/json:
schema:
$ref: '#/components/schemas/GetPredictionResponse'
"400":
$ref: '#/components/responses/BadRequestErrorResponse'
"404":
$ref: '#/components/responses/TableOrModelNotExistErrorResponse'
"409":
$ref: '#/components/responses/ResourceAlreadyExistErrorResponse'
"501": {description: Predictions are not supported}
get:
tags: [prediction]
summary: List prediction jobs
operationId: listPredictions
description: Newest first.
parameters:
- {name: prefix, in: path, required: true, schema: {type: string}}
- {name: database, in: query, required: false, schema: {type: string}}
- {name: table, in: query, required: false, schema: {type: string}}
- {name: name, in: query, required: false, schema: {type: string}}
- name: state
in: query
required: false
schema:
$ref: '#/components/schemas/PredictionState'
- {name: maxResults, in: query, required: false, schema: {type: integer, format: int32, minimum: 1}}
- {name: pageToken, in: query, required: false, schema: {type: string}}
responses:
"200":
description: Prediction jobs
content:
application/json:
schema:
$ref: '#/components/schemas/ListPredictionsResponse'
"501": {description: Predictions are not supported}
/v1/{prefix}/predictions/{predictionId}:
get:
tags: [prediction]
summary: Get a prediction job
operationId: getPrediction
parameters:
- {name: prefix, in: path, required: true, schema: {type: string}}
- {name: predictionId, in: path, required: true, schema: {$ref: '#/components/schemas/PredictionId'}}
responses:
"200":
description: Prediction metadata
content:
application/json:
schema:
$ref: '#/components/schemas/GetPredictionResponse'
"404":
$ref: '#/components/responses/PredictionNotExistErrorResponse'
"501": {description: Predictions are not supported}
/v1/{prefix}/predictions/{predictionId}/cancel:
post:
tags: [prediction]
summary: Cancel a prediction job
operationId: cancelPrediction
description: >-
Requests cancellation and returns the latest state. Terminal jobs are
unchanged.
parameters:
- {name: prefix, in: path, required: true, schema: {type: string}}
- {name: predictionId, in: path, required: true, schema: {$ref: '#/components/schemas/PredictionId'}}
responses:
"200":
description: Latest prediction metadata
content:
application/json:
schema:
$ref: '#/components/schemas/GetPredictionResponse'
"404":
$ref: '#/components/responses/PredictionNotExistErrorResponse'
"501": {description: Predictions are not supported}
/v1/config:
get:
tags:
Expand Down Expand Up @@ -2881,6 +2977,32 @@ components:
resourceType: MODEL
resourceName: model
code: 404
TableOrModelNotExistErrorResponse:
description: Database, table, or model not found
content:
application/json:
schema:
allOf:
- $ref: '#/components/schemas/ErrorResponse'
- type: object
required: [ resourceType ]
properties:
resourceType:
type: string
enum: [ "DATABASE", "TABLE", "MODEL" ]
PredictionNotExistErrorResponse:
description: Prediction not found
content:
application/json:
schema:
allOf:
- $ref: '#/components/schemas/ErrorResponse'
- type: object
required: [ resourceType ]
properties:
resourceType:
type: string
enum: [ "PREDICTION" ]
TableNotExistErrorResponse:
description:
Not Found - TableNotExistException, the table does not exist
Expand Down Expand Up @@ -3264,7 +3386,7 @@ components:
type: string
resourceType:
type: [ string, "null" ]
enum: [ "DATABASE", "TABLE", "PARTITION", "COLUMN", "SNAPSHOT", "BRANCH", "TAG", "VIEW", "SEMANTIC_VIEW", "DIALECT", "FUNCTION", "MODEL", "DEFINITION", "UNKNOWN", null ]
enum: [ "DATABASE", "TABLE", "PARTITION", "COLUMN", "SNAPSHOT", "BRANCH", "TAG", "VIEW", "SEMANTIC_VIEW", "DIALECT", "FUNCTION", "MODEL", "PREDICTION", "DEFINITION", "UNKNOWN", null ]
resourceName:
type: [ string, "null" ]
code:
Expand Down Expand Up @@ -4467,6 +4589,12 @@ components:
minLength: 1
pattern: '^(?!\.{1,2}$).+'
description: A non-empty path segment other than `.` or `..`.
PredictionId:
type: string
minLength: 1
maxLength: 64
pattern: '^(?!\.{1,2}$).+'
description: A prediction identifier other than `.` or `..`.
ModelField:
type: object
additionalProperties: false
Expand Down Expand Up @@ -4548,6 +4676,158 @@ components:
$ref: '#/components/schemas/GetModelResponse'
nextPageToken:
type: [ string, "null" ]
PredictionState:
type: string
enum: [PENDING, RUNNING, TERMINATING, TERMINATED]
PredictionTerminationCode:
type: string
enum: [SUCCESS, SUCCESS_WITH_FAILURES, FAILED, CANCELED]
PredictionTerminationDetails:
type: object
additionalProperties: false
required: [code]
properties:
code:
$ref: '#/components/schemas/PredictionTerminationCode'
message:
type: string
PredictionStatus:
type: object
additionalProperties: false
required: [state]
properties:
state:
$ref: '#/components/schemas/PredictionState'
terminationDetails:
$ref: '#/components/schemas/PredictionTerminationDetails'
oneOf:
- properties:
state: {enum: [PENDING, RUNNING]}
not:
required: [terminationDetails]
- properties:
state: {const: TERMINATING}
- properties:
state: {const: TERMINATED}
required: [terminationDetails]
description: >-
terminationDetails is forbidden for PENDING and RUNNING, optional for
TERMINATING, and required for TERMINATED.
PredictionSource:
type: object
additionalProperties: false
required: [database, table]
properties:
database: {type: string, minLength: 1, maxLength: 128}
table: {type: string, minLength: 1, maxLength: 128}
PredictionOutput:
type: object
additionalProperties: false
required: [table]
properties:
table:
$ref: '#/components/schemas/PredictionSource'
description: >-
Managed result table containing source identity columns, a response string,
and an error string.
SubmitPredictionRequest:
type: object
additionalProperties: false
required: [source, model, inputs, instructions]
properties:
source:
$ref: '#/components/schemas/PredictionSource'
name:
type: string
minLength: 1
maxLength: 128
description: Optional non-unique label.
idempotencyToken:
type: string
minLength: 1
maxLength: 64
description: >-
Optional token reused for retries of one logical submission. SDKs generate
one by default; requests without a token are independent submissions.
model:
allOf:
- $ref: '#/components/schemas/Identifier'
required: [database, object]
properties:
database: {type: string, minLength: 1, maxLength: 128}
object: {type: string, minLength: 1, maxLength: 128}
inputs:
type: object
minProperties: 1
propertyNames: {type: string, minLength: 1, maxLength: 128}
additionalProperties:
type: string
minLength: 1
maxLength: 1024
description: Model input-to-column mappings.
mergeKeyColumns:
type: array
uniqueItems: true
items: {type: string, minLength: 1, maxLength: 128}
description: >-
Stable source columns used when table primary keys or row IDs are unavailable.
Omit to let the server derive identity; without stable identity the prediction
runs as a full refresh.
instructions:
type: string
minLength: 1
maxLength: 1048576
description: Natural-language guidance for the prediction.
where:
type: string
minLength: 1
maxLength: 1048576
options:
type: object
propertyNames: {type: string, minLength: 1, maxLength: 128}
additionalProperties:
type: string
minLength: 1
maxLength: 1024
description: >-
Paimon dynamic options. Use scan.snapshot-id or scan.tag-name for time
travel; the two options are mutually exclusive.
GetPredictionResponse:
type: object
additionalProperties: false
required: [predictionId, source, status, snapshotId, model]
properties:
predictionId: {$ref: '#/components/schemas/PredictionId'}
source:
$ref: '#/components/schemas/PredictionSource'
name: {type: string, minLength: 1, maxLength: 128}
status:
$ref: '#/components/schemas/PredictionStatus'
snapshotId: {type: integer, format: int64, minimum: 1}
model:
$ref: '#/components/schemas/Identifier'
output:
$ref: '#/components/schemas/PredictionOutput'
startTime:
type: integer
format: int64
minimum: 0
description: Start time as Unix epoch milliseconds in UTC.
endTime:
type: integer
format: int64
minimum: 0
description: End time as Unix epoch milliseconds in UTC.
ListPredictionsResponse:
type: object
required: [predictions]
properties:
predictions:
type: array
items:
$ref: '#/components/schemas/GetPredictionResponse'
nextPageToken:
type: [ string, "null" ]
ConsumerInfo:
type: object
properties:
Expand Down
Loading
Loading