Hot-reload never reaches the engine: single-file `config.toml` bind mount pins the inode, so removed/rotated secrets keep working

[rvalitov]

Summary

Every command that changes users or limits reports success while the running
engine keeps the old config. secret add prints "Config reloaded (hot-reload,
no restart)", the config file on disk genuinely contains the change, and the
engine never sees it. Only mtproxymax restart applies it.

This is not cosmetic, and it fails in both directions:

• A new user cannot connect — the engine has never heard of them.
• A removed or rotated secret keeps working. Cutting off a leaked key is
  exactly the operation that silently does n

Environment

• MTProxyMax v1.4.0-LTS
• telemt 3.5.6 (3693d1e, ghcr.io/samnet-dev/mtproxymax-telemt)
• Alpine / OpenRC LXC, Docker, single contai
• Reproduced on a plain single-instance install

Reproduction

The engine container gets the config as a single-file bind mount
(mtproxymax.sh:9241):

-v "${CONFIG_DIR}/config.toml:/etc/telem

A single-file mount pins one inode. Compare he
container is actually reading, around a config change:

# 1. Both inodes match after a container start
stat -c '%i' /opt/mtproxymax/mtproxy/config.toml
PID=$(docker inspect -f '{{.State.Pid}}' mtp
stat -c '%i' "/proc/$PID/root/etc/telemt.toml"

# 2. Change something
mtproxymax secret add zzreloadtest      # prints "Config reloaded (hot-reload, no restart)"

# 3. Re-check: the host inode moved, the con
stat -c '%i' /opt/mtproxymax/mtproxy/config.toml
stat -c '%i' "/proc/$PID/root/etc/telemt.toml"

# 4. The kernel says the mounted inode is gone
mount | grep telemt                      # -> /opt/mtproxymax/mtproxy/config.toml//deleted

Observed on an affected install:

Step	Host inode	Container inode
after restart	542	542	in sync
first config generation	1125	542	detached
second generation	1125	542	stays detached

Once detached, no SIGHUP, no inotify event aeliver
the new bytes — they are not visible inside the container at all. Only
recreating the container re-mounts the current file, which is why restart
appears to be the only thing that works.

The engine side is not at fault

Worth stating up front, since the obvious first suspicion is that telemt does
not support hot reload:

• src/config/hot_reload/fields.rs swaps aled,
  user_ad_tags, user_max_tcp_conns, user_expirations, user_data_quota,
  per-user rate limits) as hot fields.
• src/config/hot_reload/watcher.rs has three reload triggers: a directory
  inotify watcher, a PollWatcher with poll_interval(3s) and
  compare_contents(true), and a SIGHUP handler via
  SignalKind::hangup().
• The image is ENTRYPOINT ["telemt"], so telemt is PID 1 and
  docker kill -s SIGHUP does reach it.

The signal path is fine. The bytes never arrive.

Correct diagnostic (the metrics caveat)

Don't use the octet/connection families to decide whether a user exists:

• telemt_user_octets_* and telemt_user_connections_* are rendered from a
  runtime traffic registry, not from the config
  (src/metrics/render/users.rs: `for entry. A
  configured user with no traffic emits no series, and a user removed from the
  config keeps its series. Absence there proves nothing.
• Use telemt_user_unique_ips_current instem
  config.access.users, so a configured-but-idle user does appear:

curl -s localhost:9090/metrics | grep 'telemt_user_unique_ips_current'

Scope

reload_proxy_config is the single reload path for secrets, limits, ad-tags
and templates — 46 call sites — so everything that changes users or limits is
affected:

secret add / remove / rotate / rotate --all / enable / disable /
rename / clone / import / extend / archive / unarchive, batch
add/remove, secret set-limits, `template a
ad-tag changes.

Separately, all three paths that build a secondary-instance config
(_start_all_instances, reload_proxy_config, instance_add) do
mv "${CONFIG_DIR}/config.toml" "$inst_config". That is rename(2): it gives
the instance config a new inode (so instance**
unlinks the primary config.toml, which detaches a running primary regardless
of the mount question. A third defect, reload_proxy_config printing
"Config reloaded (hot-reload, no restart)" ual
line is 2>/dev/null || true, is what makes all of this silent.

Proposed fix

Mount the config directory instead of the file, and point the engine at the
file inside it. A directory mount tracks directory entries rather than pinning
one inode, so replacing config.toml is picked up immediately:

-v "${CONFIG_DIR}:/etc/telemt:ro"
"$(get_docker_image)" /etc/telemt/config.toml

Plus: write instance configs straight to their own file instead of moving them
through config.toml, verify the engine can actually see the bytes just
written and restart to apply if it cannot, aen the
signal was really delivered.