Skip to content

[SDK] Support MeterConfigurator updates and example - #4313

Open
adityabavadekar wants to merge 4 commits into
open-telemetry:mainfrom
adityabavadekar:add_meter_configurator_update
Open

[SDK] Support MeterConfigurator updates and example#4313
adityabavadekar wants to merge 4 commits into
open-telemetry:mainfrom
adityabavadekar:add_meter_configurator_update

Conversation

@adityabavadekar

Copy link
Copy Markdown

Fixes #4256

Adds the ability to update a MeterConfigurator (and the MeterConfig of
existing meters) at runtime. Following the pattern used in #4065 and #4224.

This implements the optional requirement to allow updating the
MeterProvider's MeterConfigurator:
https://opentelemetry.io/docs/specs/otel/metrics/sdk/#meterconfigurator

This function is called when a Meter is first created, and for each
outstanding Meter when a MeterProvider's MeterConfigurator is updated (if
updating is supported).

Changes

  • Adds UpdateMeterConfigurator to MeterProvider to support setting new
    meter configs at runtime. The update holds the same lock as GetMeter, so a
    meter is never returned with a config that is out of date with the provider
    configurator.
  • Moves the Meter enabled flag to an atomic so updating it does not block
    instrument creation or Collect.
  • Adds MeterContext::SetMeterConfigurator.
  • Adds tests and an example.

Note: Instruments created from a disabled meter remain no-op after the meter is re-enabled, and measurements recorded while disabled still appear in the cumulative total. Both behaviors are documented in the example README.

For significant contributions please make sure you have completed the following items:

  • CHANGELOG.md updated for non-trivial changes
  • Unit tests have been added
  • Changes in public API reviewed

Adds MeterProvider::UpdateMeterConfigurator. It sets a new MeterConfig on
all meters that already exist, and new meters get the new configurator too.

The update takes the same lock as GetMeter, so GetMeter can never hand back
a meter with an old config. The meter enabled flag is now an atomic, so the
update does not block instrument creation or Collect.
@adityabavadekar
adityabavadekar requested a review from a team as a code owner July 27, 2026 09:59
@linux-foundation-easycla

linux-foundation-easycla Bot commented Jul 27, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: adityabavadekar / name: Aditya Bavadekar (0209a3d)

@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.59459% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.18%. Comparing base (7a38ec8) to head (19936ab).

Files with missing lines Patch % Lines
sdk/src/metrics/meter.cc 90.48% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4313      +/-   ##
==========================================
+ Coverage   81.14%   81.18%   +0.05%     
==========================================
  Files         446      446              
  Lines       18922    18941      +19     
==========================================
+ Hits        15353    15376      +23     
+ Misses       3569     3565       -4     
Files with missing lines Coverage Δ
...etry/sdk/instrumentationscope/scope_configurator.h 100.00% <ø> (ø)
sdk/include/opentelemetry/sdk/metrics/meter.h 62.50% <100.00%> (+5.36%) ⬆️
sdk/src/metrics/meter_context.cc 93.19% <100.00%> (+0.42%) ⬆️
sdk/src/metrics/meter_context_factory.cc 50.00% <ø> (ø)
sdk/src/metrics/meter_provider.cc 88.71% <100.00%> (+2.18%) ⬆️
sdk/src/metrics/meter_provider_factory.cc 100.00% <ø> (ø)
sdk/src/metrics/meter.cc 82.56% <90.48%> (+1.20%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Fixes include lists reported by include-what-you-use. Adds a bazel target for
the example, and tests for the null configurator and invalid context branches.
@adityabavadekar
adityabavadekar force-pushed the add_meter_configurator_update branch from eeaf61d to ab71694 Compare July 28, 2026 11:55

@dbarker dbarker left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feature and tests! Please see some initial comments below.

// This example shows how to use MeterProvider::UpdateMeterConfigurator to enable and disable
// meters by instrumentation scope at runtime. Updating the MeterConfigurator affects all existing
// and future meters provided by the MeterProvider. It is safe to call concurrently with
// MeterProvider::GetMeter and with instrument creation and recording on existing meters.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

existing meters -> existing instruments

// and stop collection. Code that must tolerate being started with a disabled Meter has to create
// its instruments again after the Meter is enabled.

#include <stdint.h>

@dbarker dbarker Jul 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick - use the c++ header <cstdint> and std:: namespace for integer types.


// A long export interval is used so that this example only exports when it calls ForceFlush.
metrics_sdk::PeriodicExportingMetricReaderOptions options;
options.export_interval_millis = std::chrono::milliseconds(60000);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider reducing this (maybe 10 s). I think force flush may block for as long as the export interval plus the export.

void UpdateMeterConfig(MeterConfig config) noexcept;

/** Returns whether this meter is enabled by its current MeterConfig. */
bool IsMeterEnabled() const noexcept { return meter_enabled_.load(std::memory_order_relaxed); }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider renaming to IsEnabled()

MeterConfig meter_config_;
// MeterConfig state is stored in an atomic variable so that instrument creation and Collect()
// never block on a concurrent MeterProvider::UpdateMeterConfigurator.
std::atomic<bool> meter_enabled_{true};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As implemented the dynamic enable/disable has some quirks:

  • If the meter is disabled when the instrument is created then the instrument is a noop for the life of the process regardless if the meter is later enabled.
  • If an instrument is created from an enabled meter and then that meter is disabled, the instrument still records into storage as if it is enabled. The recording cost does not go away.

This is a bit out of line with the spec:

If a Meter is disabled, it MUST behave equivalently to No-op Meter.
The value of enabled MUST be used to resolve whether an instrument is Enabled. See Instrument Enabled for details.

The instrument needs to also have knowledge of the meter's enabled state. This is also a requirement if we support the sync instrument Enabled API method. I think we should support this method :).

To support the ability to enable and disable instruments consider making this a std::shared_ptr<std::atomic<bool>>. This could then be passed on to each instrument (Synchronous base class), SyncMetricStorage and BoundEntry to allow each instrument to disable (behave as a noop instrument) and enable based on the MeterConfig applied at any time.

I think it would okay to not implement full sharing of this meter_enabled flag and adding the sync instrument Enabled API method in this PR, but there should be a follow up to do so.

// investigate. Its metrics are collected and exported again.
// Stage 4: The investigation completes and the external_library meter is disabled again.
//
// IMPORTANT: a Meter that is disabled when an instrument is created returns a no-op instrument,

@dbarker dbarker Jul 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation here. I think we should move towards meters/instruments that can be enabled and disabled dynamically independent of the MeterConfig on startup. More in this comment .


provider->UpdateMeterConfigurator(EnableAll());
counter->Add(1);
EXPECT_EQ(CollectScopeNames(reader), (std::set<std::string>{"scope.toggle"}));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion only proves that the scope is present; it does not prove that the collected value is correct. All six new UpdateMeterConfigurator* tests use CollectScopeNames(), which discards the metrics and points:

disable -> Add(1)  // should be ignored
enable  -> Add(1)
expected sum: 1
current sum:  2

Both sums produce {"scope.toggle"}, so this test stays green while the disabled measurement is retained and later exported. Please inspect the counter's SumPointData (or add a value-returning collection helper) and assert that the sum is 1 here.

Comment thread sdk/src/metrics/meter.cc
opentelemetry::nostd::string_view unit) noexcept
{
if (!meter_config_.IsEnabled())
if (!IsMeterEnabled())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch turns the Meter's current disabled state into a permanent object choice: kNoopMeter returns a NoopCounter whose Add() methods are empty. Re-enabling the Meter cannot replace a counter already held by library code:

static auto counter = meter->CreateUInt64Counter("requests");  // Meter is disabled
// The host later enables this Meter.
counter->Add(1);  // Still a NoopCounter.

This defeats the runtime-enable-for-debugging use case in #4256, especially for third-party instrumentation. The example avoids the failure only by creating every instrument while enabled; an application cannot force a dependency to recreate its instruments.

Please return an instrument whose recording behavior observes the Meter's shared enabled state (or an equivalent revivable proxy), and add a regression test: create while disabled, enable, Add(1), collect, and assert value 1.


Metrics are exported to stdout via the `OStreamMetricExporter`. The example
uses a long export interval and calls `ForceFlush` at the end of each
stage, so each stage exports exactly once.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The long interval does not make ForceFlush() the only export trigger. OnInitialized() starts the worker, and DoBackgroundWork() calls CollectAndExportOnce() before its first wait. That startup collection races with Stage 1's recording and ForceFlush():

documented: 4 stage-triggered export batches
observed:   5 batches, with the extra batch's placement varying

Therefore the checked-in exact transcript is not deterministic. The CTest target only runs the executable, and Bazel uses a plain cc_test, so neither validates stdout.

const std::lock_guard<std::mutex> guard(lock_);
context_->SetMeterConfigurator(std::move(meter_configurator));

for (auto &meter : context_->GetMeters())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for (auto &meter : context_->GetMeters())GetMeters() is explicitly non-thread-safe, so this loop is safe only under a cross-class invariant: every production mutation of meters_ currently reaches AddMeter() or RemoveMeter() through a MeterProvider method that holds lock_. That invariant is not stated here, and it is not structurally enforced because MeterContext exposes both mutators publicly.

T1: provider lock_ -> GetMeters() -> iterate borrowed span
T2: meter_lock_    -> AddMeter()  -> meters_ may reallocate

lock_ does not exclude a direct/future T2 caller. The analogous tracer and logger loops explicitly document their ownership invariant, and their collections are provider-private.

Please either enforce this invariant by using ForEachMeter() or making the mutators provider-only, or at minimum add the equivalent ownership comment above this loop so future callers do not introduce a race.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MeterProvider support for updating the MeterConfigurator

3 participants