feat(llo/v31): skip observation/aggregation for recently-reported channels

[cawthorne]

Summary

Adds an offchain config field DefaultMinObservationIntervalNanoseconds that skips observation and aggregation for channels that are not yet due to report — the same idea as the existing DefaultMinReportIntervalNanoseconds report-skip, but applied earlier in the pipeline so the work is never done rather than done and discarded.

Motivation

DefaultMinReportIntervalNanoseconds skips report generation when a channel's last report was too recent. But the channel's streams are still observed (a data-source fetch, via the blob pump) and aggregated every round, even when the channel is guaranteed not to report.

Design

Per-channel observation schedule

Each channel carries an observation schedule in the hot state: the timestamp at which it next becomes due. It advances by the interval from its own previous slot, and only on a round that actually reported.

Advancing from the reporting round instead would make the interval a fixed delay, so any latency in a cycle would be added to every later cycle and the cadence would creep. Advancing from the schedule makes latency a one-time phase offset: a channel configured to report every T keeps reporting every T.

This is deliberately not validAfter. That watermark is a report boundary, emitted in the report and defining the window (validAfter, observationTimestamp] that consecutive reports must tile exactly; anchoring it to a schedule would leave gaps between reports.

Withholding, and the lead it produces

Stream values are gathered asynchronously and arrive a round after a channel's streams enter the pump's input, so the first due round after a skip window has no aggregates. Reporting anyway would emit nil values and advance validAfter over a round that carried nothing, so instead the channel withholds its report. Neither the watermark nor the schedule advances, so it stays due and reports once the values land.

That delay then becomes the offset between schedule and watermark — and that offset is exactly the lead the pump needs. A channel becomes due for observation far enough ahead of when it may report that its values are always gathered in time. The lead matches whatever the data source actually needs and is not configured anywhere.

What the skip does not apply to

history_backfill channels are exempt: their watermark is a history timestamp
rather than a report time, so a report cadence means nothing for them.

Channels reading History(...) are not exempt, deliberately. The skip makes a
channel's report cadence the sampling rate for its windows. That lowers their
resolution but does not make them wrong — records carry their own
observedAtNanoseconds and TWAP integrates over real time — and it cannot stall
silently, because an unreadable window leaves the channel unreportable, which also
stops its schedule advancing, so it reverts to observing every round until the
window is satisfied. The degenerate case costs reports, not correctness.

What it does change is how a window must be sized, now documented on the config
field and beside the warmup section in the calculated engine:

• TWAP buckets its window by the second and keeps the newest record per bucket,
  so an interval at or below one second changes nothing about achievable quality
  and makes depth go further, since depth stops being spent on records that
  collapse into the same bucket. Above a second, the observed bucket count falls
  to about window/interval and interior gaps become interval-1 buckets.
• Every other window function (Avg, Median, EMA, SMA, WMA, Delta,
  PctChange, Spread, Variance, Stddev, Last) reads the record series
  directly with no bucketing, so its wall-clock meaning tracks the sampling rate
  at any interval.

This is not a new class of concern: checkTWAP already rejects
minSamples > depth at config time, and because M counts distinct seconds, any
working TWAP config is already sized against the sampling rate. A node logs once
per channel when a history-reading channel is first skipped, carrying the
interval.

Carry-forward

A pair belonging only to a skipped channel keeps its carried value. Carry-forward
is the last-known-good store behind TimestampedStreamValue monotonicity and the
fallback for a transient aggregation failure, so a channel must not come out of a
skip window worse off than it went in. The pass is driven from the live
definitions rather than the carry map, so a pair whose last channel was removed is
still reclaimed; and it is restricted to pairs the round never visited, so the
deliberate drops in the main loop keep their decision. Carried values never reach
history — a window records a gap rather than a repeat, so a carried value is not
weighted twice and a stalled feed stays visible to TWAP's gap checks.

Promotion

Promotion replaces validAfter wholesale from the predecessor's retirement report so the handover is gapless, and the observation schedule is cleared to match. A slot inherited from staging could otherwise leave a channel not due on the promotion round, and a channel that is not aggregated has no values to report — reopening the gap, up to a full interval wide, in the one path built to avoid it. An unset schedule means due, so every channel aggregates immediately and rebuilds its slot from its first report.

Channels that have never reported

A channel with no schedule entry is due. A newly effective channel therefore aggregates from its first round and builds its initial aggregates and history windows immediately. It is still not reportable that round, since validAfter equals the observation timestamp — pre-existing behaviour.

Changes

Config (llo/protocol/)

• llo_offchain_config.proto / .pb.go: new field 4 defaultMinObservationIntervalNanoseconds
• offchain_config.go: wired through Decode/Encode; validation — must be 0 for protocol v0; for v1+, 0 disables the skip and it must not exceed DefaultMinReportIntervalNanoseconds. Setting it equal is fine.
• plugin_codecs.proto / .pb.go: LLOHotStateProto gains observationDueNanoseconds (field 5), sorted ascending by channelID like its neighbours for deterministic serialization

Plugin (llo/dev/v31/)

• plugin.go: isObservationDue is a schedule lookup; nextObservationDue owns the fixed-rate advance and skips missed slots rather than firing every round to catch up after a channel has been unable to report. Observation applies the one-round-lagging advance the hot state has not yet recorded, and publishes the pump's input every round so a later cycle can never gather a stale stream set.
• statetransition.go: computes the schedule alongside validAfter, and derives the due set once so aggregation and calculated-stream evaluation cannot disagree about which channels a round covers.
• reports.go: a due channel with a missing stream aggregate is not reportable while the interval is configured.
• kv.go: schedule persisted in the hot state; readValidAfterOnly reads what Observation needs without decoding carry-forward aggregates.

Behaviour when disabled

With the interval unset (0, every deployment today) nothing changes: every channel is due, the withholding rule does not apply, and the observed set is identical — including history_backfill channels, which are exempt from the schedule entirely since their watermark is a history timestamp rather than a report time.

Trade-off

The data source is gathered only on the rounds a channel is actually due rather than on every round, so the saving scales with how much slower a channel's report cadence is than the round rate. The cost is that a channel's report lands one gather later than the moment it becomes due — a constant phase offset per channel, not a growing one, and not something operators need to compensate for in config.

Tests

• Test_ObservationIntervalSkip_observableStreams — due/not-due filtering, shared streams, unscheduled channels, disabled mode
• Test_ObservationIntervalSkip_ObservationAdvancesLaggingSchedule — Observation and StateTransition agree despite the hot state lagging a round
• Test_nextObservationDue — fixed-rate advance, no drift over 100 late cycles, catch-up after an outage
• Test_ObservationIntervalSkip_WithholdsReportUntilValuesArrive — no nil reports; rule is gated on the interval
• Test_ObservationIntervalSkip_CarryForwardSurvivesSkip — a skipped channel's carried value survives, is not published as this round's aggregate, and is still reclaimed once no live channel declares the pair
• Test_History_SurvivesObservationSkip — a history window gains one record per aggregated cycle, none on skipped rounds, and stays readable across a skip window
• Test_ObservationIntervalSkip_aggregate, Test_ObservationIntervalSkip_FullRound — unit and end-to-end skip/report cycle

ok  github.com/smartcontractkit/chainlink-data-streams/llo/dev/v31
ok  github.com/smartcontractkit/chainlink-data-streams/llo/dev/v31/llotest
ok  github.com/smartcontractkit/chainlink-data-streams/llo/protocol
ok  github.com/smartcontractkit/chainlink-data-streams/llo/protocol/calculated

Notes for review

• LLOHotStateProto is replicated state. The new field is additive, but writing it changes the serialized record, so rollout must be gated on the agreed offchain config like the rest of the feature.
• plugin_codecs.pb.go was regenerated with protoc-gen-go v1.36.11 / protoc v7.35.0 against the committed v1.36.12 / v7.35.1, so the version header moves. Worth regenerating with the pinned toolchain before merge.