feat: v4 multi-cloud estate, global omni-search, and topology UX refinements - #206
Open
JLCode-tech wants to merge 59 commits into
Open
JLCode-tech wants to merge 59 commits into
JLCode-tech wants to merge 59 commits into
Conversation
…n set
A credential template accepted any string as `provider`. Only a handful of
literals are ever matched when credentials are resolved (`aws`, `ibm`, `gcp`,
`azure`, `ssh`), so a natural misspelling like `provider="ibmcloud"` — which
matches every adjacent field name (`ibmcloud_api_key`, `ibmcloud_resource_group`)
— was stored happily, read back looking healthy, and then matched NO branch in
the resolver. The template silently contributed nothing, the deploy fell through
to global `.env` credentials that weren't there, and the failure surfaced far
away as an opaque Terraform "BearerToken property is required" error.
Root cause: `provider` was never validated at create/update, and the canonical
set it must belong to was implicit, scattered across the resolver's `if
template.provider == ...` branches.
Fix:
- Add `SUPPORTED_PROVIDERS = {aws, gcp, azure, ibm, ssh}` as the single source
of truth in credential_template_service.py, documented against each consumer
that injects/resolves credentials for that provider.
- Validate `provider` in `create_template` and `update_template` (service layer,
covers every caller) -> BadRequestError with an enumerated message.
- Add matching Pydantic validation on the create/update route models so the API
boundary returns a clean 422 naming the bad value and the supported set,
before the service is reached. Update-time validation only fires when the
caller is actually changing `provider`.
Tests lock: the exact `ibmcloud` misspelling is rejected at both the service
(400) and route (422) layers; every canonical provider is still accepted; an
update can't switch a template onto a no-op provider; and a resolver-contract
test documents that a pre-existing `ibmcloud` row injects nothing (the behavior
the validation now prevents from being created). Mutation-tested: reverting the
guards reds 5 tests.
Closes #191
Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4
Self-review M3: the SUPPORTED_PROVIDERS comment overstated azure/gcp as general 'credential resolution' — they inject only a post-provision kubeconfig token (AKS via engine_router, GKE via get_gcp_service_account_info), not terraform-env credentials. Only aws/ibm inject into the terraform env, so #191's 'looks healthy, injects nothing' class fully closes for aws/ibm; azure/gcp are still validated but that terraform-env class never applied to them. Comment-only; no behavior change. Claude-Session: https://claude.ai/code/session_01UpRYiFserdBE5ESHn759N4
JLCode-tech
force-pushed
the
feat/v4-multi-cloud-ux-enhancements
branch
from
September 7, 2026 00:05
9f75326 to
42ee999
Compare
…ak + wire the enum into the contract
bonnyr-f5 round-2 (REVISE). Findings 1+2 share one fix; +minor 3.
Findings 1 & 2 - replace the provider membership check in the route
@model_validator with a Literal field type on both the create and update
models:
CredentialTemplateBase.provider: Literal["aws","azure","gcp","ibm","ssh"]
CredentialTemplateUpdate.provider: Literal[...] | None = None
A model-level ValueError made Pydantic attach the ENTIRE request body to the
422 (no RequestValidationError handler in main.py), leaking the plaintext
credential to the MCP client / FE toast. A Literal yields a field-scoped error
(no body echo) and maps to an OpenAPI enum (INV-3). Mirrors the auth_type
idiom at routes/f5_devices.py:73/82. The service-level validate_provider /
SUPPORTED_PROVIDERS stays as defense for direct callers; the two lists agree
exactly. Regenerated backend/openapi.json (provider now type:string + enum on
both models) and frontend-v2 api-generated.ts (provider is the union).
Finding 3 - type TEMPLATE_PROVIDER_OPTIONS against the generated provider
union (as const satisfies) so an invalid/divergent entry is a compile error.
ssh stays intentionally omitted from the create UI but is now permitted by the
type.
Minor 3 - PUT {"provider": null} returned HTTP 500: model_dump(exclude_unset=
True) includes {"provider": None}, and the generic assignment loop wrote None
into the nullable=False column. Guard now checks presence, dropping an explicit
null so the provider is left unchanged; fixed the inaccurate comment.
Finding 6 - de-vacuumed the misspelled-provider assertion (assert on azure/ssh,
not the "ibm" substring of the offender) and turned it into the body-leak
reproduction (asserts the plaintext secret is absent from the error). Added a
PUT-null regression test. Mutation-checked both new assertions.
Verified: 109 passed (affected suite); ruff clean; generate-openapi.py --check
OK; FE types regen idempotent; tsc --noEmit clean.
Claude-Session: https://claude.ai/code/session_01UCsZXDxBsWV2s4kT47DwDW
This was referenced Sep 8, 2026
jgruberf5
pushed a commit
that referenced
this pull request
Sep 8, 2026
…core Self-review findings on feat/v4-multi-cloud-ux-enhancements (tractable minors + a real coverage gap; no blockers). MINOR 1 — llm_observability_service.stats fleet branch: per-cluster `models` is already a COUNT of distinct models, so max() silently understated the fleet whenever clusters ran disjoint model sets. Switch to sum() as an upper bound of "models in use across the fleet" (matching the generic "Models" stat-tile), with a comment stating the true union is not computable from these inputs. Pin the semantics with a fleet `stats` test (previously untested). Coverage gap — routes/k8s/search._scan_cluster_for_query (the ~260-line live-scan core) was FULLY MOCKED in both integration tests. Add unit tests that mock the k8s client RESPONSES (not the scan function) and exercise Ingress/HTTPRoute/VirtualServer/Egress/Gateway/Service parsing, LB-IP harvesting, CRD-absent tolerance, and the ThreadPoolExecutor timeout harvest (partial results, no hang) + cross-cluster dedup. Add a request-based negative-authZ test asserting the route requires require_viewer (was only asserted by inspection). Did NOT touch credentials_service.py (reconciled with #208 at merge). Claude-Session: https://claude.ai/code/session_01UCsZXDxBsWV2s4kT47DwDW
Collaborator
Self-review (cold, adversarial) + fixes appliedIndependent cold audit of the highest-risk surfaces. No blockers. The top-flagged risk — an omni-search cross-tenant leak — was investigated and refuted: Findings (minor) — fixed @
Merge-coordination note: the credential-fallback + dead-Azure-resolver findings in Verified: 41 target tests pass, ruff clean, |
This was referenced Sep 11, 2026
Collaborator
Review —
|
- Convert AWS/IBM region validators from hardcoded-list rejection to pattern-based acceptance so new or private regions are selectable. - Add Azure and GCP region validators using the same pattern-based approach; wire them into project, credential-template, and cluster schemas/routes. - Update frontend region selectors (AWS, Cloud, SystemDefaults) to free-form inputs with datalist suggestions instead of restrictive dropdowns. - Add KubernetesCluster.account_id and discovery_status columns plus fleet-health response fields for cloud context. - Update unit tests for validators, project schemas, k8s schemas, and frontend selectors.
…ctor - Extract shared is_operator_live_connected() helper and use it in the operator list, fleet health, and BNK health context. - Reuse services.scanner.nodes.parse_node() in BNK fetch instead of duplicating the zone/instance-type label fallback logic. - Add an optional label prop to CloudRegionSelector and reuse it in SystemDefaults to remove four near-identical region input blocks.
- Add connectivity and integration sections to BnkHealthResponse. - Reuse the cluster's persisted status for connectivity and the shared operator live-connection helper for integration. - Display ConnectivityBadge and IntegrationBadge in the dashboard banner. - Add backend unit tests and frontend dashboard tests for the new fields.
Add /detect-credentials endpoint that discovers existing Kubernetes clusters from a project's credential template for AWS, IBM Cloud, Azure, and GCP. Each provider lists accessible clusters, builds a kubeconfig from the template credentials, and registers the cluster in BNK-Forge. - New ClusterDiscoveryService orchestrates detection and registration. - Provider helpers: EKS, ROKS, AKS, GKE. - Frontend auto-detect switched to api.detectClustersFromCredentials(). - Backend + frontend tests updated; openapi.json and api-generated.ts regenerated.
- Move BNK Resources tab from System page to Fleet page - Make GET /api/system/bnk-consumption viewer-accessible - Move MCP Server from standalone sidebar page to System page tab - Move Benchmarks sidebar item from OPERATE to OBSERVE section - Update affected tests and regenerate OpenAPI types
- Add services/bnk/traffic_stats.py with analyze_traffic_stats() and fetch_tmm_traffic_stats() wrapping existing TMM debug helpers. - Add Pydantic schemas for listener/egress/firewall-rule traffic stats. - Wire trafficStats into the unified /f5bnk/data response. - Surface hit/connection badges on F5BNKTopologyViewer listener/egress nodes. - Add hits column to F5BNKPolicyViewer firewall-rule tables. - Add total-connections summary chips in TrafficFlowOverview. - Regenerate openapi.json and TypeScript generated types. - Add backend unit tests and frontend component/hook tests.
- Enrich BNK topology with gateway/listener/route accepted/programmed conditions - Add policy resolved/programmed status to topology and policy associations - Add response models for gateway topology and policy associations endpoints - Surface inline status badges in topology, traffic flow, and policy views - Visualize cross-namespace ReferenceGrants in topology and traffic flow - Extract shared ConditionsList component for Gateway/HTTPRoute/Service details - Add lightweight Service detail fallback and register it in resource registry - Regenerate OpenAPI spec and TypeScript generated types
…urce with settings Module Library sync failed for official-bnk-forge-modules because the clone used source.branch and then tried git checkout <git_ref>. A shallow branch clone does not fetch tags, so checking out a tag ref (v2.2.0) failed with 'pathspec did not match any file(s) known to git'. Use source.git_ref (falling back to branch) directly in git clone --branch, which accepts branch and tag names and already checks out the requested ref. Also reconcile the canonical official module source with the current module_library.git_* settings before a direct source sync, so a stale branch/git_ref on the source row does not override the configured ref. Validated: /api/module-sources/3/sync now succeeds, discovers 24 pack modules, and updates the source row to branch=git_ref=release/2.2.
…figview probes when no VS rows
…d CNE available state - Update has_condition() and get_condition_message() to inspect direct conditions arrays on parent_status dicts as well as standard K8s status.conditions. - Add get_policy_operational_status() to evaluate status.ancestors and status.descendants condition refs for BNKNetPolicy and BNKSecPolicy in BNK 2.3. - Update _build_cne_instance() to recognize Available/Reconciled condition states and populate default phase when healthy. - Update _match_routes_to_listener() to check parent_status condition acceptance.
Stop per-request ThreadPoolExecutors from spawning 20 workers each, which exploded backend PID count to 100+ under concurrent BNK page loads. Use module-level shared executors with small caps for BNK CRD fetches and TMM configview probes. Add Redis-backed short-term caches for: - EKS/GCP bearer tokens (10 min TTL) - fetch_all_bnk_data results (30 s TTL) - TMM traffic stats + configview uuid mappings (30 s / 5 min TTL) - CWC license status (30 s) and report (60 s) Each cache supports force=true to bypass when the UI explicitly refreshes. License activation invalidates the cached status/report so the new state is reflected immediately.
Add account_id, discovery_status, connectivity_status, integration_status,
zones, access_method, and node_count to the KubernetesCluster model, cluster
response schemas, serializers, and detail endpoints. Populate account_id from
credential-template discovery paths (AWS account, Azure subscription, GCP
project) and persist version/node_count/zones/last_synced_at from the scanner.
Includes migration v2_157 and a new GET /api/projects/{project_id}/connectivity
route backed by probe_project_clusters.
…O routes, doc SSO/terraform split F1: normalize naive azure_sso_token_expiry to UTC before comparing in _test_azure_template — matches get_sso_status / credential_refresh_service guards; fixes TypeError on SQLite/dev naive round-trip. Adds SQLite regression test (mutation-verified: fails with the exact TypeError without the guard). F2: remove the unwired standalone Azure SSO routes (/azure/sso/initiate, /azure/sso/poll, /azure/subscriptions), their request models, and the unused client methods (initiateAzureSSO/pollAzureSSO/listAzureSubscriptions). The frontend uses the server-side template flow (authenticate-sso/poll-sso, returns only has_credentials); these paths leaked long-lived access/refresh tokens in the response body. Regenerated openapi.json + api-generated.ts. F4: document at the terraform credential-injection site that SSO Azure templates deliberately inject no credential (SSO is validation/console; terraform provisioning uses the service-principal secret). Claude-Session: https://claude.ai/code/session_01UCsZXDxBsWV2s4kT47DwDW
- Rebase onto work/v4-localhost (incorporating staging and #205) - Renumber Alembic migration to v2_158 with down_revision v2_157 - Restrict azure_auth_method typing with Literal["service_principal", "sso"] - Persist refreshed Azure SSO tokens on test connection by committing DB session - Ensure SSO action menu and config completeness checks gate strictly on azure_auth_method == 'sso' - Add regression component tests for refresh token DB persistence and config validation
…topology views - Add global Omni-Search endpoint (/api/v1/k8s/search) with client hook and Hero search bar on Dashboard - Add MultiCloudEstate overview card with provider breakdowns and interactive filtering - Streamline F5 BNK page to direct vertical topology views (Object Topology, Traffic Flow Pipeline, Policy Matrix) - Consolidate CRD exploration into a unified Kubernetes Explorer Hub and streamline CNF infrastructure view - Polish Benchmarks page setup headers and remove redundant stepper progress banner - Clean up Fleet Management provider filter chips and guidance - Optimize LLM Observability filters and debounce hooks with full test coverage
…de log aggregation
…licate filter logic
…core Self-review findings on feat/v4-multi-cloud-ux-enhancements (tractable minors + a real coverage gap; no blockers). MINOR 1 — llm_observability_service.stats fleet branch: per-cluster `models` is already a COUNT of distinct models, so max() silently understated the fleet whenever clusters ran disjoint model sets. Switch to sum() as an upper bound of "models in use across the fleet" (matching the generic "Models" stat-tile), with a comment stating the true union is not computable from these inputs. Pin the semantics with a fleet `stats` test (previously untested). Coverage gap — routes/k8s/search._scan_cluster_for_query (the ~260-line live-scan core) was FULLY MOCKED in both integration tests. Add unit tests that mock the k8s client RESPONSES (not the scan function) and exercise Ingress/HTTPRoute/VirtualServer/Egress/Gateway/Service parsing, LB-IP harvesting, CRD-absent tolerance, and the ThreadPoolExecutor timeout harvest (partial results, no hang) + cross-cluster dedup. Add a request-based negative-authZ test asserting the route requires require_viewer (was only asserted by inspection). Did NOT touch credentials_service.py (reconciled with #208 at merge). Claude-Session: https://claude.ai/code/session_01UCsZXDxBsWV2s4kT47DwDW
JLCode-tech
force-pushed
the
feat/v4-multi-cloud-ux-enhancements
branch
from
September 11, 2026 06:44
3cccc07 to
7294fe9
Compare
Collaborator
Author
Re-Review Requested: All Review Feedback Addressed & Rebased onto #207We have rebased
@bonnyr-f5 — ready for final re-review! |
Collaborator
Author
Re-Review Request: Schema Stack Alignment & CI 100% GreenAll review action items from the
Branch is fully ready for re-review and merge. |
Brings the multi-cloud UX branch up to what runs on the localhost test line. - system: cloud provider and region badges on the BNK consumption table (schema, aggregation, route, panel and unit tests), with the fleet collection switched to as_completed under a single fleet timeout - BnkResourcesPanel: overview / BNK resources / fleets tabs with provider filtering, plus a component test - MultiCloudEstate: estate layout refinements - cloud-providers: Azure badge styling and provider normalisation helpers - clusters route: project-scoped delete alias with an optional project_id, covered by an integration test - bnk/fetch: 64 fetch workers, per-future timeouts, no namespace sweep in pod discovery, and connect/read timeouts on node listing Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…and project-scoped cluster delete Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
D-020 forbids raw Tailwind palette colours; restores the token classes the badge had before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eached the cluster Every fetcher swallows its exception and returns an empty default, so an unreachable or expired-token cluster yields a fully shaped empty scan. _persist_cluster_metadata wrote a fresh last_synced_at and connectivity_status="connected" over it, hiding the outage behind a current sync time (#194). Gate both on the fetch having returned a server version; the other metadata fields are still written from what is available. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
JLCode-tech
pushed a commit
that referenced
this pull request
Sep 16, 2026
JLCode-tech
pushed a commit
that referenced
this pull request
Sep 16, 2026
Staging removed the same verify=False fallback for GCP; corporate TLS interception is handled by the CA bundle built from /app/certs (SSL_CERT_FILE, REQUESTS_CA_BUNDLE), which localhost already uses. Restores the file to the PR #206/#207 version. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This pull request brings together multi-cloud Azure / GCP / AWS cluster and auth support, global Omni-Search, unified CRD exploration, direct F5 BNK topology views, multi-cluster LLM Observability analytics, and UX polish across the v4 platform.
Key Highlights
1. Multi-Cloud & Hybrid Estate
2. Global Omni-Search
/api/k8s/search) scanning clusters, projects, stacks, blueprints, and workloads.HeroOmniSearchwith keyboard shortcuts (Cmd+K/Ctrl+K), type categorizations, and instant deep links.3. LLM Observability & Multi-Cluster AI Gateway
4. F5 BNK Topology & Diagnostics Polish
5. CNF & Kubernetes CRD Hub Consolidation
K8sCrdExplorerPanelon the Kubernetes page.6. Fleet & Benchmarks Polish
Verification
LlmDashboard.test.tsxandLlmLogs.test.tsx).test_llm_observability_service.pyand routes).