Add idle servable management preview docs

[atobiszei]

Documentation to:
#4486

Preview:
https://openvino-doc.iotg.sclab.intel.com/atobisze_idle_docs/model-server/ovms_docs_idle_servable_management.html

[Copilot]

🟡 Changes recommended

The new documentation contains a few correctness/clarity issues (notably around group swap timing/behavior and some link/style consistency) that should be addressed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds documentation for the new Idle Servable Management preview feature and surfaces it across existing docs, including a new metric for graph loaded/idle state.

Changes:

• Adds a new preview doc page describing idle servable management, grouping, behavior, and metrics.
• Cross-links the feature from generative use-case prep and model repository docs (classic + graph).
• Documents the new ovms_graph_loaded metric and exposes the feature in the main features index.

File summaries

File	Description
docs/prepare_generative_use_cases.md	Adds a note pointing readers to idle servable management and --cache_dir.
docs/models_repository_graph.md	Adds a note about unloading/reloading graph groups via idle management and group_name.
docs/models_repository_classic.md	Adds a note about unloading/reloading model groups via idle management and group_name.
docs/metrics.md	Adds ovms_graph_loaded gauge to the default metrics table.
docs/idle_servable_management.md	New preview documentation page for idle servable management.
docs/features.md	Adds idle servable management to the features list and overview section.

Review details

Suppressed comments (3)

docs/idle_servable_management.md:25

• These sentences have a few grammatical issues (missing articles) and are a bit ambiguous; tightening the wording improves readability and reduces the chance of misinterpreting grouping semantics.

A group is unit OVMS loads and unloads. Add `group_name` to classic model configuration or MediaPipe graph configuration. All servables in same group load and unload together. Use groups for servables which must be available together, for example, graph and its dependent models.

Without `group_name`, every classic model and MediaPipe graph is its own group. Group named `permanent` stays loaded and is never unloaded.

OVMS keeps one non-permanent group active. A request for servable from another group waits for current group to finish its requests, then loads requested group. First request after unload includes wake-up latency. 

docs/idle_servable_management.md:30

• This example is JSON; using a json code fence gives correct syntax highlighting, and capitalizing “JSON” matches common documentation style.

Example config json with grouping


```text

docs/idle_servable_management.md:58

• The note says not to depend on a bounded switch time, but the implementation currently has a bounded drain wait (30s) and then fails the swap with an error. The docs should mention the timeout/failure behavior to set correct expectations.

> **Important:** Concurrent requests targeting loaded and unloaded groups have no scheduling policy in this preview. While requests run on active group, incoming traffic continues to that group. Do not depend on fair routing or a bounded switch time between groups.

• Files reviewed: 6/6 changed files
• Comments generated: 5
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[rasapala]

Fix

[atobiszei]

This is false positive - we have a lot of places where we don;t use that prefix.

[rasapala]

Unload inactive model and graph groups, then load them with the next inference request. Reduce and reuse the CPU and GPU memory needed to serve multiple large models.

[rasapala]

One suggestion for consideration.