feat(storage): add multi-realm RGW support and phase-resume fixes - #22
Open
htohru-ibm wants to merge 1 commit into
Open
feat(storage): add multi-realm RGW support and phase-resume fixes#22htohru-ibm wants to merge 1 commit into
htohru-ibm wants to merge 1 commit into
Conversation
Key changes
-----------
- Add --rgw-realm option for multi-realm Ceph clusters (native mode).
Required when the cluster has named realms instead of 'default'.
- Rewrite detect_rgw_zone() for reliable zone resolution:
- Use 'radosgw-admin zone get --rgw-realm=<realm>' when --rgw-realm
is given (default_info in zone list contains a UUID, not the name).
- Redirect debug/warning to stderr so the zone name captured via $()
is not polluted.
- Extract zone name from the 'zones' array in the fallback path.
- Resolve RGW_HOST from short hostname to IP address:
'ceph orch ps' returns a short hostname that is typically not resolvable
from an external tenant cluster. Use 'ceph orch host ls -f json' + jq
to map the hostname to its public IP address automatically.
If resolution fails the short hostname is kept as-is.
- Auto-detect RGW protocol (http/https) from 'ceph config dump' by
checking the rgw_frontends config for ssl_port / ssl_certificate.
- Rename ceph-external-cluster-details-exporter.py references to
create-external-cluster-resources.py (current Rook/ODF script name).
- Restore RGW credentials and MAIN_RGW_ENDPOINT when resuming from
phase 6, 7, or 8 (previously caused failures on --phase N reruns).
Falls back to 'ceph orch ps --daemon-type rgw' detection if the
credentials file does not yet contain the RGW_ENDPOINT entry.
- Add MAIN_RGW_ENDPOINT_OVERRIDE env var to allow explicit endpoint
override without modifying the script.
- Add --connect-timeout/--max-time to backing-bucket curl call for
reliable timeout behaviour on slow/unresponsive endpoints.
- Delete state file in cleanup_artifacts() so the script can be
re-run from scratch after --delete.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enhancements to
scripts/setup-storage.shto support multi-realm Cephclusters and fix several phase-resume failures.
Changes
Multi-realm RGW support (
--rgw-realm)--rgw-realm REALMoption for native-mode clusters that have namedrealms (i.e. not the built-in
defaultrealm).detect_rgw_zone()for reliable zone name resolution:--rgw-realmis provided, useradosgw-admin zone get --rgw-realm=<realm>to obtain the zone name.zone list'sdefault_infofield contains a UUID, not the zone name, so it cannotbe used for this purpose.
captured via
$()is not polluted.zonesarrayinstead of
default_info.Resolve RGW host short name to IP address
ceph orch psreturns a short hostname that is typically not resolvablefrom an external tenant cluster.
RGW_HOSTfrom the orchestrator, resolve it to thenode's public IP address using
ceph orch host ls -f json+jq.If resolution fails, the short hostname is kept as-is.
Auto-detect RGW protocol (http/https)
ceph config dumpforssl_port/ssl_certificatein thergw_frontendsconfig and setRGW_PROTOCOL=httpsautomatically whenSSL is detected.
Script name correction
ceph-external-cluster-details-exporter.pywith
create-external-cluster-resources.py, which is the current nameused by Rook/ODF.
Phase-resume credential restoration (phases 6, 7, 8)
--phase N,RGW_ACCESS_KEY,RGW_SECRET_KEY, andMAIN_RGW_ENDPOINTcould be unset because phase 1was skipped. These are now restored from the credentials file written by
phase 5. If
MAIN_RGW_ENDPOINTis missing from the file, it isre-detected via
ceph orch ps --daemon-type rgw.Other
MAIN_RGW_ENDPOINT_OVERRIDEenv var to allow explicit endpointoverride without modifying the script.
--connect-timeout 10 --max-time 20to the backing-bucketcurlcall for reliable timeout behaviour on slow or unresponsive endpoints.
cleanup_artifacts()so the script can bere-run from scratch after
--delete.Testing
Tested against a native Ceph cluster with:
--rgw-realm realm-ocp1) — correct zone resolution