[doc](lance) Document OSS support for the Lance catalog - #4094
Open
FANNG1 wants to merge 2 commits into
Open
Conversation
apache/doris#67157 lets a Lance catalog reach Alibaba Cloud OSS, but the page still listed only local, file:// and s3:// warehouses. Add an OSS section beside the existing S3 one, note the qualified oss://bucket.oss-<region>.aliyuncs.com/path form and the bucket requirement, and name the OSS properties a REST catalog can use as defaults. Also state that namespace-vended credentials take precedence over the catalog's own, and that Doris accepts both the oss_ prefixed and the bare OSS-native spellings of the vended options - the shape a real Lance REST namespace hands back. Both the English and zh-CN copies are updated.
Alibaba Cloud OSS uses virtual-hosted addressing and the public endpoints no longer accept path-style requests, so there is no configuration this note usefully applies to and none that could be exercised to confirm it. Documenting the switch would only invite readers to try a setting whose behaviour is unverified.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documents the Alibaba Cloud OSS support added by apache/doris#67157 (issue apache/doris#67131). The Lance catalog page currently tells users that a warehouse can only be a local path,
file://ors3://, so OSS looks unsupported.Requested by @zhangstar333 in apache/doris#67157.
Changes
Both
versioned_docs/version-4.x/.../lance-catalog.mdxand the zh-CN copy:warehousenow listsoss://among the accepted forms.CREATE CATALOGexample, the qualifiedoss://bucket.oss-<region>.aliyuncs.com/pathform Doris reduces to the bucket, the bucket requirement, and a note onoss.session_token.oss_endpoint/oss_access_key_id/oss_secret_access_key/oss_region/oss_security_tokenor the bare OSS-nativeendpoint/access_key_id/access_key_secret/region/security_token, and that vending one option under both spellings with different values is an error.Verification
The examples were run against a real OSS bucket on a Doris cluster built from apache/doris#67157, not written from the code alone:
fs.oss.support, which the doc does not mention — creates the catalog and scans correctly;oss://bucket.<endpoint>/pathwarehouse also scans, andoss:/pathis rejected atCREATE CATALOG;lance-restservice, which vends the bare OSS-native spellings;Scope
Every statement on the page was exercised against a live bucket, including
oss.session_token: an STS triple obtained throughAssumeRolescans the table, and the same temporary key pair without the token fails to initialize the catalog with OSS's ownInvalidAccessKeyId/ "The Security Token may be lost to specify that it is a STS Access Id". That negative control is what shows the token is carried through to the BE rather than the scan succeeding by some other route.A path-style addressing note was written and then removed: Alibaba Cloud OSS uses virtual-hosted addressing and its public endpoints no longer accept path-style requests, so there was no configuration to confirm it against.