feat(lightspeed): replace RAG init container with OKP deployment - #3370
feat(lightspeed): replace RAG init container with OKP deployment #3370maysunfaisal wants to merge 4 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3370 +/- ##
==========================================
+ Coverage 65.31% 65.74% +0.42%
==========================================
Files 38 43 +5
Lines 2465 2633 +168
==========================================
+ Hits 1610 1731 +121
- Misses 705 747 +42
- Partials 150 155 +5
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
51db81c to
3bbec01
Compare
| selectorLabels := okpSelectorLabels(backstage.Name) | ||
|
|
||
| // OKP Deployment | ||
| deployment := &appsv1.Deployment{ |
There was a problem hiding this comment.
@maysunfaisal Though I'll mostly let the @redhat-developer/rhdh-install team comment on the operator source changes I'll make one comment on this part.
It makes sense to have controller parsing for OpenShift only parts, I wondering if it would be better to have the OKP deployment specification either in a OKP flavour or a separate OKP deployment specification within the lightspeed (intelligent-assistant) flavour that this controller source parses?
I'm considering the different non-lightspeed flavours users and customers may use as well as any automated patching that may be used here (i.e. patching the OKP image).
fyi @gazarenkov @rm3l
There was a problem hiding this comment.
Valid concern @michael-valdron
Midstream digest-pinning scans YAML manifests, not Go source, so it would miss the OKP image ref in okp.go. A couple of things worth noting:
- OKP is already lightspeed-only at runtime — applyOkpResources gates on IsFlavourEnabled("lightspeed") && IsOpenshift(), so non-lightspeed users get zero OKP resources today. That said, having it described declaratively in the flavour is cleaner.
- The move is non-trivial:
- the operator's config kinds (deployment, service, route) are singletons tied to the main RHDH app, so OKP would need new registered kinds + factories. The existing .ocp/.k8s suffix mechanism also doesn't handle "create-or-skip-entirely" gating, so some Go code will still be needed for the
OpenShift-only conditional logic.
- the operator's config kinds (deployment, service, route) are singletons tied to the main RHDH app, so OKP would need new registered kinds + factories. The existing .ocp/.k8s suffix mechanism also doesn't handle "create-or-skip-entirely" gating, so some Go code will still be needed for the
I'm going to explore options on how best to approach this. Would appreciate any suggestions or guidance from @gazarenkov / @rm3l on the preferred pattern for flavour-owned sidecar resources.
There was a problem hiding this comment.
The OKP Deployment/Service/Route are now defined in the lightspeed flavour YAML (config/profile/rhdh/default-config/flavours/lightspeed/okp-*.yaml) and applied via the normal model path, rather than constructed imperatively in okp.go.
Each is a separately registered key, still gated to OpenShift. The only bit kept in Go is injecting OKP_SERVICE_URL into the lightspeed-core sidecar, since that value is computed at runtime (CR name/namespace + ingress domain) and can't be a static YAML env. This also means the OKP image string now rides in the bundle ConfigMap, so midstream image pinning can see it.
|
Corresponding rhdh-chart PR redhat-developer/rhdh-chart#500 |
Move the OKP (Offline Knowledge Portal) Deployment, Service, and Route out of imperative Go construction in internal/controller/okp.go and into the lightspeed flavour as YAML manifests, so OKP is described like every other flavour resource and its image string rides in the bundle ConfigMap (visible to midstream image pinning). Addresses PR redhat-developer#3370 review comment: "should OKP be in the flavour YAML rather than Go?". Each OKP object is a separately registered model key (OkpDeploymentKey, OkpServiceKey, OkpRouteKey) with its own MergeFunc, and stays gated to OpenShift in addToModel. Flavour gating is implicit (files live only in flavours/lightspeed/). The runtime behavior is unchanged: same objects, same OpenShift-only gating, same OKP_SERVICE_URL injection into the lightspeed-core sidecar before apply (no dual ReplicaSet). Remove applyOkpResources; OKP now flows through the normal applyObjects(GetRuntimeObjects()) SSA path. Keep prepareOkpEnvVar and prepareOkpConfig as controller wiring. Regenerate bundle manifests and dist/rhdh/install.yaml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Move the OKP (Offline Knowledge Portal) Deployment, Service, and Route out of imperative Go construction in internal/controller/okp.go and into the lightspeed flavour as YAML manifests, so OKP is described like every other flavour resource and its image string rides in the bundle ConfigMap (visible to midstream image pinning). Addresses PR redhat-developer#3370 review comment: "should OKP be in the flavour YAML rather than Go?". Each OKP object is a separately registered model key (OkpDeploymentKey, OkpServiceKey, OkpRouteKey) with its own MergeFunc, and stays gated to OpenShift in addToModel. Flavour gating is implicit (files live only in flavours/lightspeed/). The runtime behavior is unchanged: same objects, same OpenShift-only gating, same OKP_SERVICE_URL injection into the lightspeed-core sidecar before apply (no dual ReplicaSet). Remove applyOkpResources; OKP now flows through the normal applyObjects(GetRuntimeObjects()) SSA path. Keep prepareOkpEnvVar and prepareOkpConfig as controller wiring. Regenerate bundle manifests and dist/rhdh/install.yaml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
15302a5 to
ecd3199
Compare
|
/build-images |
|
❌ Image build failed See workflow run for details: https://github.com/redhat-developer/rhdh-operator/actions/runs/32705736214 Triggered by @rm3l |
|
/build-images |
|
❌ Image build failed See workflow run for details: https://github.com/redhat-developer/rhdh-operator/actions/runs/32705962118 Triggered by @rm3l |
Move the OKP (Offline Knowledge Portal) Deployment, Service, and Route out of imperative Go construction in internal/controller/okp.go and into the lightspeed flavour as YAML manifests, so OKP is described like every other flavour resource and its image string rides in the bundle ConfigMap (visible to midstream image pinning). Addresses PR redhat-developer#3370 review comment: "should OKP be in the flavour YAML rather than Go?". Each OKP object is a separately registered model key (OkpDeploymentKey, OkpServiceKey, OkpRouteKey) with its own MergeFunc, and stays gated to OpenShift in addToModel. Flavour gating is implicit (files live only in flavours/lightspeed/). The runtime behavior is unchanged: same objects, same OpenShift-only gating, same OKP_SERVICE_URL injection into the lightspeed-core sidecar before apply (no dual ReplicaSet). Remove applyOkpResources; OKP now flows through the normal applyObjects(GetRuntimeObjects()) SSA path. Keep prepareOkpEnvVar and prepareOkpConfig as controller wiring. Regenerate bundle manifests and dist/rhdh/install.yaml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
71c206f to
9726670
Compare
|
/build-images |
|
✅ PR images built successfully! Images are available for testing:
Also available with PR number tag:
Triggered by @maysunfaisal |
Jdubrick
left a comment
Jdubrick
left a comment
There was a problem hiding this comment.
Just a general question as well: will OKP get cleaned up if someone disables the lightspeed flavour?
|
/cc @gazarenkov |
|
/build-images |
|
✅ PR images built successfully! Images are available for testing:
Also available with PR number tag:
Triggered by @maysunfaisal |
gazarenkov
left a comment
gazarenkov
left a comment
There was a problem hiding this comment.
To add the additional resources needed for your plugin please use Plugin dependencies feature instead.
You can see how it implemented in orchestrator flavour as well (it uses sonataflow dependencies).
Thanks @gazarenkov, looked into plugin-deps for OKP. Our manifests are already declarative YAML, not imperative Go: The Go wrappers (
Orchestrator's sonataflow deps are standalone resources, no cross-container wiring or platform gating needed. OKP needs both. Open to discussion if there's a path to extend plugin-deps for these cases! |
…DP-16103] Remove the init-rag-data init container and FAISS vector store from the lightspeed flavour. Add OKP (Offline Knowledge Portal) as a Deployment, Service, and Route managed by the operator, gated on OpenShift; vanilla K8s swaps in the lightspeed-stack-no-okp.yaml config to avoid LCORE crashing without an OKP backend. Inject OKP_SERVICE_URL into the lightspeed-core sidecar before applying objects to avoid a dual ReplicaSet rollout. Align the LCORE sidecar with the OGX image and pin the OKP image. Sync vendored Lightspeed configs from lightspeed-configs@main. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the OKP (Offline Knowledge Portal) Deployment, Service, and Route out of imperative Go construction in internal/controller/okp.go and into the lightspeed flavour as YAML manifests, so OKP is described like every other flavour resource and its image string rides in the bundle ConfigMap (visible to midstream image pinning). Addresses PR redhat-developer#3370 review comment: "should OKP be in the flavour YAML rather than Go?". Each OKP object is a separately registered model key (OkpDeploymentKey, OkpServiceKey, OkpRouteKey) with its own MergeFunc, and stays gated to OpenShift in addToModel. Flavour gating is implicit (files live only in flavours/lightspeed/). The runtime behavior is unchanged: same objects, same OpenShift-only gating, same OKP_SERVICE_URL injection into the lightspeed-core sidecar before apply (no dual ReplicaSet). Remove applyOkpResources; OKP now flows through the normal applyObjects(GetRuntimeObjects()) SSA path. Keep prepareOkpEnvVar and prepareOkpConfig as controller wiring. Regenerate bundle manifests and dist/rhdh/install.yaml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ext, docs [RHIDP-16103] Switch IA plugin references from oci://...inherit to ref:// pattern (per rm3l review, rhdh#5204). Add OTEL_SDK_DISABLED=true to LCORE env to prevent crash when OTEL collector unreachable. Add securityContext to LCORE and OKP containers matching the operator RHDH deployment precedent. Update docs: remove inherit note, add accuracy caveat for vanilla K8s without OKP knowledge base. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…RHIDP-16103] Bundle and dist regeneration after rebasing onto main (intelligent-assistant rebranding). Fix .npmrc integration test init container count (2→1) since OKP replaces the RAG init container. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ec0b6e3 to
b8ae9d6
Compare
|
|
Thanks for explanations, @maysunfaisal I think I understand your point and it tells me that we probably would need a bit improve the framework to:
But before going to it, I would like to understand why it is important in this concrete case. Please correct me if I am wrong:
|
Thanks @gazarenkov, great questions. Let me address each (with Agentic help as it was used for rhdh-operator and rhdh-chart): Why the Route? OKP isn't just a backend for LCORE — it also serves the RAG reference pages that users click in the chat UI. When the Intelligent Assistant/Lightspeed chatbot returns grounded answers, the response includes clickable A ClusterIP Service is only reachable from within the cluster. The browser is outside — so without a Route (or Ingress), the reference links are dead. Why OpenShift-only gating? Two reasons:
On the chart side redhat-developer/rhdh-chart#500, we do support vanilla K8s as an opt-in — users set One OKP per RHDH instance — temporary Yes, currently each RHDH install provisions its own OKP Deployment. We've raised this with the OKP team and they have an epic for standalone installation: RHOKP-1632 with child stories for Helm (RHOKP-1690) and Operator (RHOKP-1691) deployment. Once those land, OKP can be independently installed and shared. We have had discussions about this in meeting with the RHOKP team and how it is not a great UX. |
|
@maysunfaisal Route Curious how UX of K8s vs Openshift end-users differs? What if we provide Route based automation for Openshift users only and for K8s instruct that for this feature they have to configure the networking (ingress, loadbalancer etc) and provide it's URL manually? I see why we need this conditional deployment (at least for Route), let me think how to implement it better with plugin-deps (it should not be that complex I think) OKP instances I understand, deploying it independently makes sense in prod (however self-created instance is useful for quick start scenario). This way the only thing the plugin needed is URL (or 2 URLs: internal and external), no additional resources to create. So, as the per-RHDH is not the point we can consider per-namespace (which in most cases the same), right? |
|
Route Yes, that's exactly how it works today in this PR. On OpenShift, the operator creates a Route automatically. On vanilla K8s, the operator does not create any networking resource because there is no Ingress support; the user would need to configure Ingress/LoadBalancer themselves and provide the URL. The Helm chart takes a similar approach where it auto-detects OpenShift for Route creation, and offers opt-in Ingress on K8s. Regarding enhancing plugin-deps with platform gating and templated env injection, happy to refactor to use that once it's available, but I'd prefer not to block this PR on that future work unless the turn around time is 1 day or so. We are currently blocking on other 2.1 Feature work for Intelligent Assistant (this repo as @Jdubrick is putting some changes in + midstream Gitlab rhdh repo for pinning OKP digest image and removing the old RHDH RAG from Lightspeed) due to these PRs and we have one Sprint before 2.1 FF. OKP instances Per-CR is intentional as it keeps lifecycle simple since each CR owns its OKP and cleans it up on delete (at the cost of fist time quick start and resource consumption). Per-namespace is possible without any bundle or midstream impact (the OKP resources stay in the flavour ConfigMap either way), but it adds reconciliation complexity: multiple Backstage CRs would race on the same OKP Deployment, and we'd need to coordinate who owns it and when to delete it. If the goal is a shared, independently-managed OKP, the OKP team is working on that under RHOKP-1632, they also have OKP as MCP for Q4 on their agenda, so I am not trying to over-engineer how OKP is deployed right now, for what may seem to be temporary. (This is also consistent with redhat-developer/rhdh-chart#500 where we have one RHDH/LCORE deployment and OKP deployment per helm release). What are your thoughts? |
|
@maysunfaisal Based on our conversation I've identified the following framework enhancements we could implement:
The only question I have for the time: where do we need to inject the Route-related URL to be used in frontend? My understanding it should be backstage container but as I can see you inject it to Lcore container. Thoughts?
I am sorry to not to know about this approach earlier, the problem is that it introduces plugin-specific logic directly into the operator model, which isn't the direction we want to take, it is hardly supportable way. The operator should remain plugin-agnostic and provide generic mechanisms that any plugin can leverage. Thanks |
The ENV injection is happening to the LCORE container here in my changes https://github.com/redhat-developer/rhdh-operator/pull/3370/changes#diff-f16bc2e3b21ea94fe1e0cb23af92be4335ce36540bcef4c07d420c97067aeb6dR26 (internal/controller/okp.go - prepareOkpEnvVar()) This is relatively easy on rhdh-chart due to helm templating magic but i had to do this for Go operator. The OKP deployment is separate from the RHDH/LCORE deployment; so we had to use the above approach to get the OKP Route endpoint and set it to So, the RHDH plugin Intelligent Assistant frontend just renders what LCORE returns - OKP links pointing to RHDH docs in OKP Service. User is able to click them on the browser and see the RHDH docs from OKP service. LCORE needs the OKP Route URL to construct the citation links on LCORE server-side.
I understand the sentiment but Intelligent Assistant (IA, formerly Lightspeed) is now included by default OOTB as a flavor with upstream RHDH. Previously, Lightspeed used initContainers and volume mounts directly to RHDH deployment to set up RAG but that is now taken away and replaced by a standalone OKP Deployment/Svc/Route. From my POV, I was just plugging out and plugging in how RAG is set up. OKP resources are just dependencies of a default config Lightspeed/IA. Maybe standalone OKP resources and platform gating is a deal breaker? But let me know how you feel about this and also about your progress on https://redhat.atlassian.net/browse/RHDHPLAN-1763 because I may have to react differently based on that decision like whether I can make OKP changes to upstream and midstream before FF 🤔 |
6 participants
Description
Replace the Lightspeed RAG init container / FAISS vector store with an OKP (Offline Knowledge Portal) backend and swap the Lightspeed dynamic plugins to the Intelligent Assistant plugins.
OKP deployment
Deployment,Service, andRoute(Solr8983+ httpd8080) — newinternal/controller/okp.go, wired inbackstage_controller.go.lightspeed-stack-no-okp.yamlso LCORE does not crash without an OKP backend (operator has no Ingress support — RHDHPLAN-423/RHIDP-2176).OKP_SERVICE_URLinto thelightspeed-coresidecar before applying objects, avoiding a dual-ReplicaSet rollout.Plugin swap (Intelligent Assistant)
oci://quay.io/rhdh/red-hat-developer-hub-backstage-plugin-intelligent-assistant(-backend):{{inherit}},enabled: true; FE carries the fullpluginConfig(dynamicRoutes/mountPoints).quay.io/rhdh(notregistry.access.redhat.com) to match the operator'sCATALOG_INDEX_IMAGE+ RHDH image, so{{inherit}}resolves at the container level (install-dynamic-plugins).Config & images
HF_HOME+--synthesized-config-outputfor read-only rootfs, and pin the OKP image toregistry.redhat.io/offline-knowledge-portal/rhokp-rhel9:1.2.10-1786628394.lightspeed-configs@main(hack/sync-lightspeed-configs.sh).dist/rhdh/install.yaml+bundle/rhdh/manifests/rhdh-flavour-lightspeed-config_v1_configmap.yaml.Which issue(s) does this PR fix or relate to
https://redhat.atlassian.net/browse/RHIDP-16103
PR acceptance criteria
How to test changes / Special notes to the reviewer
On OpenShift:
make deploy IMG=<operator-image> PROFILE=rhdhOPENAI_API_KEY) — seeexamples/lightspeed.yaml.BackstageCR with thelightspeedflavour enabled and the secret wired into thelightspeed-corecontainer.developer-hubpod 2/2 Running; OKPDeployment/Service/Route1/1 Running.OKP_SERVICE_URL+HF_HOMEinjected into thelightspeed-coresidecar; LCOREGET /v1/modelsreturns 200.{{inherit}}(digests match the RHDH image's catalog).On vanilla K8s: OKP is skipped,
lightspeed-stack-no-okp.yamlis used, and LCORE starts without crashing.Building Container Images for Testing
Need to test container images from this PR?
For Maintainers: To trigger a test image build, review the code and comment
/build-images.This always builds the HEAD of the PR branch.
For Contributors: Ask a maintainer to run
/build-images.Images will be built and pushed to Quay with links posted in comments.