Skip to content

Add collectResourcesDetailed plugin - #58

Open
prabhkodes wants to merge 1 commit into
polyaxon:masterfrom
prabhkodes:feat/collect-resources-detailed
Open

prabhkodes wants to merge 1 commit into
polyaxon:masterfrom
prabhkodes:feat/collect-resources-detailed

Conversation

@prabhkodes

@prabhkodes prabhkodes commented Oct 27, 2026 •

Copy link
Copy Markdown

Refs polyaxon/traceml#31

Summary

This PR adds the collectResourcesDetailed plugin.

plugins:
  collectResourcesDetailed: true

When set, the runner adds POLYAXON_COLLECT_RESOURCES_DETAILED=true to the main container. TraceML then logs its detailed resource metrics (polyaxon/traceml#32).

Changes

  • New env key and getter, next to collectResources.
  • New V1Plugins field. Default is False. Off under no_api.
  • Converter adds the env var for the k8s and docker runners.

Runs without the plugin do not change.

Tests

  • One test for each runner. The default adds no env var. The plugin adds one.
  • Converter and polyaxonfile suites: 610 passed. ruff: clean.

@prabhkodes

Copy link
Copy Markdown
Author

How to use

In a polyaxonfile:

plugins:
  collectResourcesDetailed: true

In Python:

from polyaxon.schemas import V1Plugins

plugins = V1Plugins(collect_resources_detailed=True)

Leave it unset to keep the default: TraceML logs only the five core container metrics. Set it to log the full set of 22. Needs TraceML with polyaxon/traceml#32.

TraceML logs a small set of container-aware resources by default
(cgroup memory percent and OOM kills, cpu throttling, /dev/shm usage,
children stuck in disk sleep) and a detailed breakdown when
POLYAXON_COLLECT_RESOURCES_DETAILED is set. Expose that switch as a
plugin so it can be set from the component spec:

    plugins:
      collectResourcesDetailed: true

Disabled by default, forced off with no_api like collectResources,
and mapped to the env var by the runner converter for both the k8s
and docker runners.
@prabhkodes
prabhkodes force-pushed the feat/collect-resources-detailed branch 2 times, most recently from 721d3ec to 1ba02f8 Compare October 27, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant