Repository navigation
refactor(table): introduce a common table hierarchy - #527
Merged
Merged
Conversation
SteNicholas
reviewed
Nov 6, 2026
Member
There was a problem hiding this comment.
@lucasfang, I left two comments for this introduction. PTAL.
zjw1111
reviewed
Nov 12, 2026
zjw1111
left a comment
Collaborator
There was a problem hiding this comment.
- Tests lists build and test commands, pass counts, and pre-commit results for
0d2ddf52alongside the coverage summary. The PR template uses this section for the UT and IT cases changed by the PR and the behavior they cover. - API and Format omits the change to system-table
FullName(), which now drops the database name.Uuid()falls back to that value when there is no catalog UUID, so identically named system tables in different databases can return the same fallback ID. - Documentation has two stale statements. The system-table guide says
sys.tables,sys.partitions, andsys.all_table_optionshave no public read path. Both catalogs now expose them through a shared catalog handle. The catalog guide says builders reject the identifier spellingtbl$branch_MAIN. The branch resolver now accepts it as an alias formain.
zjw1111
reviewed
Nov 12, 2026
zjw1111
left a comment
Collaborator
There was a problem hiding this comment.
A few issues on the loaded-table paths, schema reads and code style. See the inline comments.
zjw1111
reviewed
Nov 12, 2026
Preserve query authorization and branch-aware catalog credentials in the table hierarchy. Retain both sets of regression tests and cover authorization through loaded-table virtual operators.
zjw1111
reviewed
Nov 13, 2026
zjw1111
left a comment
Collaborator
There was a problem hiding this comment.
The inline comments cover query authorization, format-table identity, fallback reads, and system-table behavior.
zjw1111
approved these changes
Nov 13, 2026
zjw1111
left a comment
Collaborator
There was a problem hiding this comment.
Approved. Please address the remaining issues in the follow-up PR.
- Query authorization still accepts a context from another REST catalog when the table name and URI match. Two storage endpoints can expose the same name and URI, so the scan and reader can still apply B's rules to A's data.
- An existing
$filesor$manifestshandle can still return raw statistics after REST enables query auth and before the filesystem schema catches up. The read ignores the catalog supplied in the context. - A query-auth reader for an already loaded
devtable rejects a validdevsplit when the read context omits the branch. Split validation uses the context's defaultmain. - Two loads of the same REST format table can create different token filesystem wrappers. Comparing their pointers rejects a valid typed write or commit context.
- Static
$rocan still lose options from the on-disk schema whenSetTableSchema()is supplied. It can also lose storage settings because the underlying table keeps the filesystem resolved from the context alone. - Update the REST format-table example to show
GetTable()->NewXxx(), and update the PR's Tests section to describe the added or changed tests and their coverage.
This was referenced Nov 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #526
The current
Table::NewScan/NewRead/NewWrite/NewCommitmethods route back through context-based factories, so calling an operator on a loaded table does not bind it to that table's schema and storage identity. Table-kind dispatch is also concentrated intable.cpp, with format tables exposed through a separate handle.Introduce a common table hierarchy so
Catalog::GetTable()returns the appropriate managed, format or system table, and operator creation dispatches through that implementation.FileStoreTablewith append-only and primary-key implementations, reusing the FileStore factories from refactor(table): extract file store factories #516 as merged into main. FileStore continues to derive its branch from options.FormatTableandSystemTableimplement the common Table interface. System tables expose their own schemas and reject writes and commits.TableFactoryand the virtual operator methods. Preserve main's precise file-index evaluation behavior through both entrypoints.This is the hierarchy step of #459, following #516. Execution-context separation and general Table Copy/time-travel APIs remain separate follow-ups.
Tests
Full build and all 36 CTest targets passed on commit
0d2ddf52:git diff apache/main HEAD --checkpassed.Regression coverage includes catalog type dispatch, loaded schema/path/filesystem binding, explicit branch conflicts and aliases, builder retries after failed validation, deferred catalog lifetime, URI-map merging for main/fallback reads, changelog sequence numbers, system-table snapshot selection, and precise-index entrypoint parity.
API and Format
Tablebecomes abstract, with a virtual destructor and virtual operator creation methods. Callers that directly constructed Table should load it through the catalog.Table::LatestSchema()andFormatTable::LatestSchema()toGetSchema(), without compatibility aliases. The accessor returns the bound schema without metadata I/O; reload through the catalog to refresh it.CurrentBranch()from the common Table interface; managed-table branch handling belongs to FileStoreTable.FormatTablederives from Table.Catalog::GetFormatTable()remains available for typed callers.Create()signatures remain unchanged. Catalogs must be owned bystd::shared_ptrwhen loading global system tables, which retain the catalog for deferred reads.Documentation
Update the catalog and format-table guides to describe common catalog dispatch, bound-table operator semantics, schema access, branch validation and global-system-table ownership.
Generative AI tooling
Generated-by: OpenAI Codex (GPT-6)