Skip to content

On-change subscriptions over NETCONF and RESTCONF, RESTCONF event streams - #1814

Open
mattiaswal wants to merge 10 commits into
mainfrom
restconf-streams
Open

mattiaswal wants to merge 10 commits into
mainfrom
restconf-streams

Conversation

@mattiaswal

@mattiaswal mattiaswal commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Description

A RESTCONF client can now subscribe to notifications and read them as a stream of server-sent events, and on-change subscriptions to the operational datastore report changes to interfaces, their addresses, LLDP neighbors and the alarm list. On-change works over NETCONF as well.

Event streams (nginx)

rousette refuses establish-subscription without a Forward header, and nginx had no location for /streams/. Both are added. A stream is a single request to the rate limit, so open streams are capped at 16 per client address instead. The read timeout is 120 s, two of rousette's 55 s keep-alives.

On-change (yangerd, statd-push, sysrepo)

Operational data is served by statd answering each GET from yangerd. That gives sysrepo nothing to compare, so an on-change subscription never fired. For the keys listed in src/statd/push.c, the data is now also written to the operational datastore when yangerd reports a change, and sysrepo sends the difference to the subscribers. A GET is still answered by statd as before.

  • yangerd sends a watching client the keys that change, without the values it computes per request, so counters do not show up as changes.
  • A sysrepo patch makes changes to the alarm list reportable when it holds a link-down alarm, whose resource is the path of an interface.

Why a daemon of its own, statd-push

The write cannot run in statd's event loop:

  • sr_apply_changes() is synchronous, and it needs the lock of the module it writes.
  • A client GET holds that lock while it waits for statd's operational callback.
  • That callback is dispatched from the same event loop that would be sitting in sr_apply_changes(). Each side waits for the other until one of them times out, 5 s.
  • The wait for the module lock is not bounded by the timeout passed to sr_apply_changes(). The operational write path passes 0, which is the fixed SR_MOD_LOCK_TIMEOUT of 5 s, so a short try and a retry from a timer does not avoid the stall.

statd already writes the mDNS neighbors from its loop this way and retries when the datastore is busy. ietf-interfaces is read far more often than infix-services, and a stalled GET or a late event on each collision is not acceptable there.

So the write needs a second execution context. statd-push is a small libev daemon with its own sysrepo connection, started by Finit with the same condition as statd. It follows a yangerd watch and writes each changed key. When a write times out it opens a new watch, on which yangerd sends every key again.

Tests

  • services/restconf/streams: subscribe without a Forward header, keep a silent stream open for 150 s, then receive a netconf-config-change.
  • misc/on_change: runs on either transport, on a veth pair so that disabling one end takes the carrier from the other. The link going down and up is reported within a second. The link-down alarm is reported within 15 s, after its 10 s hold-off, and cleared within two.
  • infamy gets subscribe() on both transports. NETCONF uses a session per subscription.

The last commit is unrelated: two Wi-Fi topology images that make test-spec rewrote on every run.

Checklist

Tick relevant boxes, this PR is-a or has-a:

  • Bugfix
    • Regression tests
    • ChangeLog updates (for next release)
  • Feature
    • YANG model change => revision updated?
    • Regression tests added?
    • ChangeLog updates (for next release)
    • Documentation added?
  • Test changes
    • Checked in changed Readme.adoc (make test-spec)
    • Added new test to group Readme.adoc and yaml file
  • Code style update (formatting, renaming)
  • Refactoring (please detail in commit messages)
  • Build related changes
  • Documentation content changes
    • ChangeLog updated (for major changes)
  • Other (please describe):

@mattiaswal
mattiaswal marked this pull request as ready for review November 21, 2026 22:00
@mattiaswal mattiaswal changed the title Restconf streams RESTCONF notifications and on-change subscriptions Nov 21, 2026
@mattiaswal mattiaswal changed the title RESTCONF notifications and on-change subscriptions On-change subscriptions over NETCONF and RESTCONF, RESTCONF event streams Nov 21, 2026
@mattiaswal
mattiaswal force-pushed the restconf-streams branch 3 times, most recently from 641affe to 8765fb7 Compare November 22, 2026 10:35
rousette refuses establish-subscription without a Forward header and
nginx had no location for /streams/.  The 120 s read timeout is two of
rousette's 55 s keep-alives.  A stream is one request to the rate
limit, so cap open streams per address instead.

Signed-off-by: Mattias Walström <lazzer@gmail.com>
The 150 s without events is longer than both the default and the
configured nginx read timeout.

Signed-off-by: Mattias Walström <lazzer@gmail.com>
Signed-off-by: Mattias Walström <lazzer@gmail.com>
statd pushes a key when it changes, and wants it without the values
computed per request, so counters do not show up as changes.

Signed-off-by: Mattias Walström <lazzer@gmail.com>
An on-change subscription never fired for data from yangerd, sysrepo
only compares what is written to the datastore.  Pushing from statd's
main loop deadlocks for 5 s with any client GET of the same module,
which holds the module lock while it waits for that loop, hence a
daemon of its own, statd-push.

Signed-off-by: Mattias Walström <lazzer@gmail.com>
The resource of a link-down alarm is the path of the interface, which
sysrepo could not put in a change notification, so a subscriber of the
alarm list heard nothing once such an alarm was in the list.

Signed-off-by: Mattias Walström <lazzer@gmail.com>
NETCONF delivers notifications on the session that subscribed, so each
subscription gets a session of its own.  netopeer2 resolves the module
names in an XPath filter as XML prefixes, hence the declarations.

Signed-off-by: Mattias Walström <lazzer@gmail.com>
Signed-off-by: Mattias Walström <lazzer@gmail.com>
Signed-off-by: Mattias Walström <lazzer@gmail.com>
make test-spec rewrote them on every run, they were checked in without
the cleanup it does.

Signed-off-by: Mattias Walström <lazzer@gmail.com>
@mattiaswal mattiaswal added the ci:main Build default defconfig, not minimal label Nov 22, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:main Build default defconfig, not minimal

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant