Skip to content

[python] Infer multimodal vector columns by query dimension - #9553

Merged
JingsongLi merged 3 commits into
apache:masterfrom
wangzhigang1999:fix/pypaimon-vector-column-dimension-inference
Sep 3, 2026
Merged

[python] Infer multimodal vector columns by query dimension#9553
JingsongLi merged 3 commits into
apache:masterfrom
wangzhigang1999:fix/pypaimon-vector-column-dimension-inference

Conversation

@wangzhigang1999

Copy link
Copy Markdown
Contributor

Purpose

Closes #9552.

MultimodalTable.search() and search_vectors() currently infer a vector column only when the schema contains one fixed-size vector field. Callers still need to pass column when multiple vector fields have different dimensions, even though the query dimension can identify one field.

This PR resolves vector columns by matching the query dimension to FixedSizeListType.list_size. It also validates explicitly selected columns and rejects mixed dimensions in search_vectors(). Multiple columns with the same dimension remain ambiguous and require column.

Tests

  • pypaimon/tests/multimodal_table_test.py: 80 passed
  • Flake8 on the changed Python files
  • Python compilation check on the changed Python files
  • git diff --check

Copilot AI lite review requested due to automatic review settings September 2, 2026 14:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Improve multimodal vector search ergonomics by resolving the target vector column using the query vector’s dimension, and by tightening validation/error reporting for ambiguous or invalid vector-column selections.

Changes:

  • Infer vector column by matching query dimension to FixedSizeListType.list_size when column is not provided.
  • Validate explicitly provided column (exists, fixed-size vector type, matching dimension).
  • Enforce consistent dimensions across all queries in search_vectors() and raise clearer errors.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
paimon-python/pypaimon/multimodal/table.py Implements _resolve_vector_column() and updates search() / search_vectors() to use dimension-based resolution + stronger validation.
paimon-python/pypaimon/tests/multimodal_table_test.py Adds coverage for dimension-based inference, ambiguous matches, and new validation error paths.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread paimon-python/pypaimon/multimodal/table.py Outdated
Comment thread paimon-python/pypaimon/multimodal/table.py Outdated
Comment thread paimon-python/pypaimon/multimodal/table.py
@JingsongLi

Copy link
Copy Markdown
Contributor

+1

@JingsongLi
JingsongLi merged commit 2788fe5 into apache:master Sep 3, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Infer multimodal vector columns by query dimension

3 participants