Conversation
penghuo
force-pushed
the
feat/query-job-lifecycle-mvp
branch
3 times, most recently
from
September 28, 2026 17:41
c9f299a to
b80dc75
Compare
…s only Introduces org.opensearch.sql.job — a five-state query lifecycle (PENDING, RUNNING, SUCCEEDED, FAILED, CANCELLED) — in response to review of opensearch-project#5809. Design-only PR: adds the neutral SPI, the state machine, the registry, the OpenSearch-side lifecycle service, and their tests. No existing API is touched. PPL / SQL / transport wiring lands in a follow-up. Addresses @dai-chen's three review comments: 1. Lifecycle layer is language-neutral, Livy-style. The job package does not import PPL; each engine implements the QueryRunner SPI. 2. Execution boundary is engine-neutral. QueryRunner returns a neutral QueryResult (schema/rows/cursor/warnings/tookMillis); the same shape fits SQL and AE. 3. Execution state is decoupled from retention. State stays RUNNING regardless of whether a caller is still waiting. Service responsibilities split into QueryJobService (public interface), QueryJobStore (in-memory registry SPI), and QueryJob (state machine active object). SecurityAdapter SPI keeps OpenSearch types out of core. Module layout: core/job/ neutral interface + value types + reference in-memory store + engine SPI opensearch/job/ OpenSearchQueryJobService — symmetric with the existing OpenSearchQueryManager Public surface (all in core/job/): QueryJobService submit(runner, submitter) / get / cancel QueryJob state machine active object (5-state) QueryJobId opaque, node-routable, URL-safe QueryJobState PENDING / RUNNING / SUCCEEDED / FAILED / CANCELLED QueryJobStatus immutable snapshot with invariant checks QueryJobStore registry SPI + InMemoryQueryJobStore QueryRunner engine SPI: run() / cancel() QueryResult neutral final result QueryFailure neutral failure descriptor Principal caller identity for owner authorization SecurityAdapter SPI: runtime context -> Principal Verification (rerun against the reduced scope): ./gradlew :core:test :opensearch:test -> 39 job tests, 0 failures MVP scope intentionally excludes wait_for_completion_timeout, keep_alive, retention, admission control, and REST wiring. Every deferred feature lands behind an existing interface without touching QueryJob. Design doc: docs/dev/query-job-mvp-design.md (design/query-job-mvp branch). Signed-off-by: Peng Huo <penghuo@amazon.com>
penghuo
force-pushed
the
feat/query-job-lifecycle-mvp
branch
from
September 28, 2026 17:47
b80dc75 to
990c47b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces the core/job package that carries a query through a five-state lifecycle (PENDING, RUNNING, SUCCEEDED, FAILED, CANCELLED) with no retention or wait-timeout coupling. Sync- and async-query transports share the same QueryJob; only the caller decides whether to block on completion().
Addresses review of opensearch-project#5809:
MVP scope: no wait_for_completion_timeout, no keep_alive, no retention, no admission control, no REST wiring. Every deferred feature lands behind one of the existing interfaces without touching QueryJob.
Design doc: docs/dev/query-job-mvp-design.md (design/query-job-mvp branch).
Description
[Describe what this change achieves]
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.