Skip to content

[FEATURE] Add NAM_SINGLE_THREADED option with a threading shim#307

Open
elcamino wants to merge 1 commit into
sdatkinson:mainfrom
elcamino:single-threaded-option
Open

[FEATURE] Add NAM_SINGLE_THREADED option with a threading shim#307
elcamino wants to merge 1 commit into
sdatkinson:mainfrom
elcamino:single-threaded-option

Conversation

@elcamino

Copy link
Copy Markdown

Problem

Since the slimmable-container and version-registry work (#256, #260, #267, #285), the core no longer compiles for bare-metal targets (Arm Cortex-M, newlib). Two independent breakages:

  1. std::mutex (member of ContainerModel, static in get_dsp.cpp) does not exist on newlib without gthreads; libstdc++ only defines it when _GLIBCXX_HAS_GTHREADS is set. Arm GNU Toolchain 15.2:
error: 'mutex' in namespace 'std' does not name a type
  1. thread_local bool gPrewarmOnResetDefault (dsp.cpp) emits a reference to the TLS runtime, which bare metal does not provide:
undefined reference to `__aeabi_read_tp'

Before those changes the core built fine on these targets, so embedded users are effectively pinned to pre-0.7.x cores and locked out of the SlimmableContainer/A2 features.

Change

A new NAM/threading.h routes the three primitives through nam::Mutex, nam::LockGuard, and NAM_THREAD_LOCAL. By default these alias std::mutex, std::lock_guard, and thread_local — zero change for every existing build. With NAM_SINGLE_THREADED defined, locking collapses to no-ops and NAM_THREAD_LOCAL to a plain static: on a single-threaded MCU there is nothing to synchronize, and the embedder takes responsibility for external synchronization if it does run threads. std::atomic is deliberately untouched (newlib supports it, and the lock-free container index should stay atomic).

Validation

  • Default and inline-GEMM builds: run_tests green (same matrix as CI)
  • Full tools build with -DNAM_SINGLE_THREADED on desktop: compiles and run_tests green (the shim is semantically neutral in a single-threaded process)
  • In production use in a Teensy 4.1 (i.MX RT1062) firmware

Route the engine's std::mutex/std::lock_guard/thread_local uses through
a new NAM/threading.h. By default the shim aliases the std primitives
and nothing changes. With NAM_SINGLE_THREADED defined, locking collapses
to no-ops and thread_local to a plain static, for bare-metal targets
whose C library has no thread support (newlib without gthreads provides
no std::mutex, and thread_local needs a TLS runtime such as
__aeabi_read_tp that bare metal lacks). std::atomic is untouched; the
embedder guarantees single-threaded use or provides its own external
synchronization around model loading and slim switching.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant