From c82331ff1178244b97c1730a353ce920c4a33889 Mon Sep 17 00:00:00 2001 From: Cyril Achard Date: Fri, 21 Aug 2026 15:15:08 +0200 Subject: [PATCH 1/2] Add v2.0.0 and v2.0.0rc1 changelogs Adds two new changelog entries under `changelog/`: a detailed `v2_0_0.md` covering major 2.0.0 features, improvements, and fixes (sync/triggering, multi-camera recording, timestamps, mono handling, processors, lifecycle, and diagnostics), plus `v2_0_0rc1.md` documenting the 2.0 RC1 rewrite highlights, release notes, contributor credits, and PR-based change log. --- changelog/v2_0_0.md | 159 +++++++++++++++++++++++++++++++++++++++++ changelog/v2_0_0rc1.md | 55 ++++++++++++++ 2 files changed, 214 insertions(+) create mode 100644 changelog/v2_0_0.md create mode 100644 changelog/v2_0_0rc1.md diff --git a/changelog/v2_0_0.md b/changelog/v2_0_0.md new file mode 100644 index 00000000..e063a8c3 --- /dev/null +++ b/changelog/v2_0_0.md @@ -0,0 +1,159 @@ +# DeepLabCut-Live-GUI v2.0.0 + +This release expands hardware synchronization, high-throughput multi-camera recording, frame timestamp reporting, monochrome capture performance, and custom processor integration. +It also improves data safety, camera lifecycle handling, diagnostics, and general GUI usability. + +## Camera triggering and synchronization + +### Added + +- Added hardware trigger configuration for compatible **GenTL** and **Basler** cameras +- Added trigger roles for common synchronization setups: + - **Off / Free-run** for continuous acquisition + - **External trigger** for cameras driven by an external hardware signal + - **Follower** for cameras synchronized to a master camera or shared trigger source + - **Master output** for free-running cameras that emit a synchronization signal +- Added configuration fields for trigger selector, input source, signal activation, and frame read timeout +- Includes a **Strict mode** to reject unsupported trigger configurations instead of applying them best-effort + +### Changed + +- Updated the camera configuration dialog with a dedicated trigger settings GUI section +- Updated active camera labels to show trigger roles +- Changed multi-camera startup ordering so external and follower cameras begin opening before free-running cameras and trigger-generating masters + +## Multi-camera performance and recording + +### Added + +- Added a dedicated recording path for each active camera + - Rather than being coupled upstream of inference and UI display, recording is now a fully independent, prioritized pipeline + - This helps ensure that recording is not affected by inference or display slowdowns, and that each camera can be recorded at its full frame rate +- Added additional recorder throughput, write latency, and buffer diagnostics. +- Added an optional **Use faster encoding parameters** setting for supported FFmpeg software codecs. + - At the cost of file size, this can help support very high frame rates and resolutions on multi-camera setups + +### Changed + +- Decoupled the GUI display rate from camera capture, pose inference, and recording rates + - The GUI is now capped at 30 FPS for display, while cameras can run at their full frame rate for inference and recording. + - Inference still ignores any stale frames. +- Added support for better using each camera’s reported runtime frame rate when configuring its recorder +- Moved recording dispatch and frame transformations closer to the camera worker to reduce the impact on GUI responsiveness + +### Fixed + +- Several logic and lifecycle fixes, which should help improve overall error handling and stability + +## Frame timestamps and synchronization data + +### Added + +- Added optional hardware frame timestamp reporting for supported cameras +- Added Basler camera timestamp-counter reporting with raw tick values and tick-frequency metadata when available +- Added updated timestamp JSON schema with: + - Per-frame software timestamps + - Optional per-frame hardware timestamp values, supplied by cameras and SDK + - Timestamp source, timebase, unit, and conversion metadata + - Explicit frame indexes for written frames + +### Changed + +- Changed timestamp sidecars to describe software and hardware timestamp sources separately +- Clarified the distinction between Unix wall-clock timestamps and device-specific camera-clock values +- Changed timestamp recording so entries are added only for frames successfully written by the encoder + - This should help the reliability of timestamp data for downstream analysis + +### Fixed + +- Improved timestamp consistency when frames are dropped before encoding + +## Monochrome camera performance improvements + +### Added + +- Added a **Preserve mono frames** option for supported backends. + - This avoids performance bottlenecks from unnecessary conversion of mono frames to color +- Added camera output format settings and runtime format reporting +- Added native monochrome frame support for compatible GenTL & Basler cameras +- Added camera probing for detected pixel format, emitted frame format, and recommended mono handling. + +### Changed + +- Updated camera labels and diagnostics to indicate preserved mono output +- Additional reporting when the camera’s pixel format differs from the frame format emitted to the application + +## GUI and usability + +### Added + +- Added detected camera resolution, frame rate, pixel format, and output format to the camera configuration dialog +- Added better recording path previews with session and run folder placeholders. +- Added additional persistent preferences for recording names, processor selection, inference camera, and encoding options +- Added **Help → View documentation** to open the DeepLabCut-Live-GUI documentation + +### Changed + +- Improved camera configuration, reset, discovery, and preview workflows +- Improved camera and recorder status summaries +- Improved recording session naming and container-aware filename handling + +### Fixed + +- Prevented recording and preview controls from becoming permanently disabled after failed stop operations +- Improved cleanup when closing camera dialogs or the main application window +- Improved handling of unavailable cameras loaded from a saved configuration +- Improved configuration persistence and restoration across application sessions + +## Custom processor workflows + +### Added + +- Added processor hooks for recording start and recording stop events + - This allows custom data processing, saving, or cleanup to be performed in sync with the recording + - NOTE: Stopping inference while recording is active may leave processor data unsaved or unpaired with the recording, depending on the processor implementation. +- Added recording context communication with session, run folder, filename, and output file information +- Added support for processor plugins that must be constructed in the inference worker +- Added clearer processor status and recording state information in the GUI + +### Changed + +- Improved custom processor discovery, selection and settings persistence + +### Fixed + +- Improved processor cleanup and finalization when recording stop + +## Camera and preview lifecycle + +### Added + +- Added better checks for cameras that are starting, running, or stopping to avoid unresponsive GUI states +- Improved cancelling preview while cameras are still opening + +### Changed + +- Updated preview controls so **Stop Preview** remains available during camera startup +- Preserved user-configured camera order for the tiled display + +### Fixed + +- Improved partial multi-camera startup behavior when some cameras open successfully and others fail +- Separated stable internal camera identity from human-readable display labels +- Prevented a camera from reporting a successful start after startup was cancelled +- Improved cleanup of partially initialized backends and worker threads +- Improved detection and rejection of duplicate camera configurations +- Improved recovery when cameras disconnect, repeatedly return empty frames, or fail during startup +- Improved controller shutdown finalization + +## Reliability and diagnostics + +### Added + +- Added focused timing diagnostics for camera workers, controller dispatch, inference, and recording + +### Fixed + +- Improved handling of stale camera identities and GenTL producer paths +- Improved best-effort cleanup without allowing cleanup errors to hide the original failure +- Improved consistency between camera backend return values and downstream capture, inference, and recording code diff --git a/changelog/v2_0_0rc1.md b/changelog/v2_0_0rc1.md new file mode 100644 index 00000000..dc8cdd1d --- /dev/null +++ b/changelog/v2_0_0rc1.md @@ -0,0 +1,55 @@ +Version 2.0 is a major rewrite of the application from scratch, delivering a redesigned GUI powered by the latest DLC-Live pose estimation workflows. + +## New in 2.0 + +- New GUI rebuilt from the ground up using PySide6 +- PyTorch support for DeepLabCut Live inference +- New camera backends for broader hardware support and more flexible acquisition setups + +## Highlights + +Improved experience for live preview, inference, and recording +Better support for multi-camera experimental workflows +Modernized architecture to support future extensions and backend improvements + +## Notes + +This is a major update and may differ from previous versions in setup, supported workflows, and backend behavior. +Please review the [updated documentation](https://deeplabcut.github.io/DeepLabCut/docs/dlc-live/dlc-live-gui/index.html) for installation, camera configuration, and model compatibility. + +## Credits +* @arturoptophys PySide6 rewrite, new preview, controls & camera backends +* @C-Achard Production polish, UI/UX improvements, camera backends improvements, documentation +* @deruyter92 Code review, testing and feedback +* @MMathisLab Project lead + +--- + +## What's Changed +* Support Basler camera a2A2590-60umBAS (and other Basler cameras, untested) by @antortjim in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/11 +* Basler camera support by @MMathisLab in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/12 +* Relax python version upper bound by @jeylau in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/20 +* Upgrade Aravis backend features and testing by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/42 +* Improving testing coverage and pre-commit checks before release by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/43 +* Refactor camera config dialog and reduce code duplication by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/46 +* Final UI tweaks by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/47 +* Improve camera config loading UX and improve Basler backend by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/45 +* Upgrade Gentl backend by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/44 +* README update for PySide6 version by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/50 +* Add standardized linting workflow and config by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/53 +* Upgrade Gentl backend cti discovery by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/51 +* Pre-release cleanup for PySide6 GUI by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/54 +* Update dlclive requirement & add deploy workflow by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/48 +* Deploy workflow candidate for 2.0.0rc1 by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/55 +* Upgrade PySide6 multi-camera GUI by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/38 +* Remove .coveragerc (duplicated in pyproject.toml) by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/61 +* Trigger CI on main/master pushes and tags by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/62 +* Add docs link to README by @C-Achard in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/63 + +## New Contributors +* @antortjim made their first contribution in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/11 +* @MMathisLab made their first contribution in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/12 +* @jeylau made their first contribution in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/20 +* @C-Achard made their first contribution in https://github.com/DeepLabCut/DeepLabCut-live-GUI/pull/42 + +**Full Changelog**: https://github.com/DeepLabCut/DeepLabCut-live-GUI/compare/v1.0...v2.0.0rc1 From d12b6514f92f5c966fa8ca95df978e63ad4b916b Mon Sep 17 00:00:00 2001 From: Cyril Achard Date: Fri, 21 Aug 2026 15:21:35 +0200 Subject: [PATCH 2/2] Refine v2.0.0 changelog structure Reworked the v2.0.0 release notes for clearer organization and wording across trigger sync, timestamps, recording, monochrome support, processor workflows, lifecycle handling, GUI updates, and diagnostics. --- changelog/v2_0_0.md | 182 +++++++++++++++++++++++++------------------- 1 file changed, 105 insertions(+), 77 deletions(-) diff --git a/changelog/v2_0_0.md b/changelog/v2_0_0.md index e063a8c3..b26c7a45 100644 --- a/changelog/v2_0_0.md +++ b/changelog/v2_0_0.md @@ -1,159 +1,187 @@ # DeepLabCut-Live-GUI v2.0.0 -This release expands hardware synchronization, high-throughput multi-camera recording, frame timestamp reporting, monochrome capture performance, and custom processor integration. -It also improves data safety, camera lifecycle handling, diagnostics, and general GUI usability. +This release expands hardware synchronization, high-throughput multi-camera recording, frame timestamp reporting, monochrome capture, and custom processor integration. +It also improves data safety, lifecycle management, diagnostics, and general usability. ## Camera triggering and synchronization ### Added -- Added hardware trigger configuration for compatible **GenTL** and **Basler** cameras -- Added trigger roles for common synchronization setups: +- Hardware trigger configuration for compatible **GenTL** and **Basler** cameras +- Trigger roles for common synchronization setups: - **Off / Free-run** for continuous acquisition - **External trigger** for cameras driven by an external hardware signal - **Follower** for cameras synchronized to a master camera or shared trigger source - **Master output** for free-running cameras that emit a synchronization signal -- Added configuration fields for trigger selector, input source, signal activation, and frame read timeout -- Includes a **Strict mode** to reject unsupported trigger configurations instead of applying them best-effort +- Configuration for trigger selector, input source, activation condition, and frame read timeout +- **Strict mode** to reject unsupported trigger configurations instead of applying them best-effort ### Changed -- Updated the camera configuration dialog with a dedicated trigger settings GUI section -- Updated active camera labels to show trigger roles -- Changed multi-camera startup ordering so external and follower cameras begin opening before free-running cameras and trigger-generating masters - -## Multi-camera performance and recording - -### Added - -- Added a dedicated recording path for each active camera - - Rather than being coupled upstream of inference and UI display, recording is now a fully independent, prioritized pipeline - - This helps ensure that recording is not affected by inference or display slowdowns, and that each camera can be recorded at its full frame rate -- Added additional recorder throughput, write latency, and buffer diagnostics. -- Added an optional **Use faster encoding parameters** setting for supported FFmpeg software codecs. - - At the cost of file size, this can help support very high frame rates and resolutions on multi-camera setups - -### Changed - -- Decoupled the GUI display rate from camera capture, pose inference, and recording rates - - The GUI is now capped at 30 FPS for display, while cameras can run at their full frame rate for inference and recording. - - Inference still ignores any stale frames. -- Added support for better using each camera’s reported runtime frame rate when configuring its recorder -- Moved recording dispatch and frame transformations closer to the camera worker to reduce the impact on GUI responsiveness +- Added a dedicated trigger settings dialog to the camera configuration workflow +- Active camera labels now show configured trigger roles +- Multi-camera startup now prioritizes external and follower cameras before free-running cameras and trigger-generating masters ### Fixed -- Several logic and lifecycle fixes, which should help improve overall error handling and stability +- Improved handling of unsupported or incomplete trigger configurations +- Improved expected timeout handling while cameras wait for hardware trigger pulses +- Improved trigger cleanup when cameras stop or close + ## Frame timestamps and synchronization data ### Added -- Added optional hardware frame timestamp reporting for supported cameras -- Added Basler camera timestamp-counter reporting with raw tick values and tick-frequency metadata when available -- Added updated timestamp JSON schema with: +- Optional hardware frame timestamp reporting for supported cameras +- Basler camera timestamp-counter reporting with raw tick values and tick-frequency metadata when available +- An updated timestamp JSON schema containing: - Per-frame software timestamps - - Optional per-frame hardware timestamp values, supplied by cameras and SDK + - Optional per-frame hardware timestamp values supplied by cameras or SDKs - Timestamp source, timebase, unit, and conversion metadata - - Explicit frame indexes for written frames + - Explicit indexes for written frames ### Changed -- Changed timestamp sidecars to describe software and hardware timestamp sources separately -- Clarified the distinction between Unix wall-clock timestamps and device-specific camera-clock values -- Changed timestamp recording so entries are added only for frames successfully written by the encoder - - This should help the reliability of timestamp data for downstream analysis +- Software and hardware timestamp sources are now described separately +- Timestamp entries are added only after frames are successfully written by the encoder +- Documentation now distinguishes Unix wall-clock timestamps from device-specific camera clock values ### Fixed - Improved timestamp consistency when frames are dropped before encoding +- Preserved raw hardware timestamp values alongside converted values +- Improved fallback behavior when hardware timestamps or conversion metadata are unavailable -## Monochrome camera performance improvements +## Multi-camera performance and recording ### Added -- Added a **Preserve mono frames** option for supported backends. - - This avoids performance bottlenecks from unnecessary conversion of mono frames to color -- Added camera output format settings and runtime format reporting -- Added native monochrome frame support for compatible GenTL & Basler cameras -- Added camera probing for detected pixel format, emitted frame format, and recommended mono handling. +- An optional **Use faster encoding parameters** setting for supported FFmpeg software codecs + - This can improve encoding throughput for high-resolution or high-frame-rate recordings, at the cost of larger files +- A dedicated full-rate recording path for each active camera + - Recording is independent of the inference and display pipelines + - Display or inference slowdowns no longer directly limit recording throughput +- Recorder diagnostics for throughput, write latency, queue depth, and dropped frames ### Changed -- Updated camera labels and diagnostics to indicate preserved mono output -- Additional reporting when the camera’s pixel format differs from the frame format emitted to the application +- Decoupled GUI display from camera capture, pose inference, and recording rates + - The GUI display is capped at 30 FPS while cameras can continue running at their full rate + - Inference continues to discard stale frames +- Improved use of each camera's reported runtime frame rate when configuring its recorder +- Moved recording dispatch and frame transformations closer to camera workers to reduce work on the GUI thread +- Reorganized recording logic into dedicated services -## GUI and usability +### Fixed + +- Prevented asynchronously queued frames from being modified through reused camera buffers or NumPy views +- Improved frame-size mismatch and encoder error handling +- Improved recording shutdown, retry, cleanup, and UI recovery behavior +- Prevented recording controls from remaining disabled after a failed stop operation + +## Monochrome camera performance improvements ### Added -- Added detected camera resolution, frame rate, pixel format, and output format to the camera configuration dialog -- Added better recording path previews with session and run folder placeholders. -- Added additional persistent preferences for recording names, processor selection, inference camera, and encoding options -- Added **Help → View documentation** to open the DeepLabCut-Live-GUI documentation +- A **Preserve mono frames** option for supported backends + - This avoids unnecessary conversion of monochrome frames to three-channel color +- Native monochrome frame support for compatible GenTL and Basler cameras +- Camera output-format settings and runtime format reporting +- Camera probing for detected pixel format, emitted frame format, and recommended mono handling ### Changed -- Improved camera configuration, reset, discovery, and preview workflows -- Improved camera and recorder status summaries -- Improved recording session naming and container-aware filename handling +- Camera labels and diagnostics now indicate preserved mono output +- Improved reporting when the camera pixel format differs from the frame format emitted to the application +- Grayscale conversion remains available in the recorder when required by the selected encoding path ### Fixed -- Prevented recording and preview controls from becoming permanently disabled after failed stop operations -- Improved cleanup when closing camera dialogs or the main application window -- Improved handling of unavailable cameras loaded from a saved configuration -- Improved configuration persistence and restoration across application sessions +- Improved handling of monochrome and higher-bit-depth frames before display and encoding ## Custom processor workflows ### Added -- Added processor hooks for recording start and recording stop events - - This allows custom data processing, saving, or cleanup to be performed in sync with the recording - - NOTE: Stopping inference while recording is active may leave processor data unsaved or unpaired with the recording, depending on the processor implementation. -- Added recording context communication with session, run folder, filename, and output file information -- Added support for processor plugins that must be constructed in the inference worker -- Added clearer processor status and recording state information in the GUI +- Processor hooks for recording start and stop events + - These allow custom processing, saving, and cleanup to remain synchronized with recording +- Recording context containing session, run folder, filename, and output file information +- Support for processor plugins that must be constructed in the inference worker +- Clearer processor status and recording state information in the GUI ### Changed -- Improved custom processor discovery, selection and settings persistence +- Improved custom processor discovery, selection, enablement, and settings persistence +- Processor-controlled recording now uses the standard recording lifecycle +- Stopping inference while recording now provides a clearer warning that processor data may remain unsaved or unpaired, depending on the processor implementation ### Fixed -- Improved processor cleanup and finalization when recording stop +- Improved processor cleanup and finalization when recording stops +- Prevented duplicate recording start notifications +- Improved handling of processors that do not implement optional saving methods ## Camera and preview lifecycle ### Added -- Added better checks for cameras that are starting, running, or stopping to avoid unresponsive GUI states -- Improved cancelling preview while cameras are still opening +- Explicit handling for cameras that are starting, running, or stopping +- Support for cancelling preview while cameras are still opening +- Runtime reporting of actual camera frame rate, resolution, pixel format, and output format ### Changed -- Updated preview controls so **Stop Preview** remains available during camera startup -- Preserved user-configured camera order for the tiled display +- **Stop Preview** remains available during camera startup +- User-configured camera order is preserved in the tiled display while trigger startup ordering is handled separately +- Stable internal camera identities are now separated from human-readable display labels ### Fixed -- Improved partial multi-camera startup behavior when some cameras open successfully and others fail -- Separated stable internal camera identity from human-readable display labels -- Prevented a camera from reporting a successful start after startup was cancelled +- Improved partial multi-camera startup when some cameras open successfully and others fail +- Prevented cameras from reporting a successful start after startup was cancelled - Improved cleanup of partially initialized backends and worker threads - Improved detection and rejection of duplicate camera configurations -- Improved recovery when cameras disconnect, repeatedly return empty frames, or fail during startup +- Improved recovery from camera disconnections, repeated empty frames, and startup failures - Improved controller shutdown finalization +## GUI and usability + +### Added + +- **Help → View documentation** action for opening the DeepLabCut-Live-GUI documentation +- Detected camera resolution, frame rate, pixel format, and output format in the camera configuration dialog +- Improved recording path previews with session and run-folder placeholders +- Persistent preferences for recording names, processor selection, inference camera, and encoding options + +### Changed + +- Improved camera discovery, configuration, reset, and preview workflows +- Improved camera and recorder status summaries +- Improved recording session naming and container-aware filename handling +- Improved control availability during camera startup, inference, recording, and shutdown + +### Fixed + +- Improved asynchronous error reporting so background failures are visible in the GUI +- Improved cleanup when closing camera dialogs or the main application window +- Improved handling of unavailable cameras restored from saved configurations +- Improved configuration persistence and restoration across application sessions + ## Reliability and diagnostics ### Added -- Added focused timing diagnostics for camera workers, controller dispatch, inference, and recording +- Timing diagnostics for camera workers, controller dispatch, inference, and recording +- Expanded automated coverage for camera backends, trigger modes, recording, timestamps, processor hooks, and GUI lifecycle behavior + +### Changed + +- Improved logging for camera discovery, runtime information, recording state, and processor lifecycle events +- Expanded backend contract tests for camera reads, trigger waits, timestamp propagation, and cleanup ### Fixed - Improved handling of stale camera identities and GenTL producer paths -- Improved best-effort cleanup without allowing cleanup errors to hide the original failure -- Improved consistency between camera backend return values and downstream capture, inference, and recording code +- Improved best-effort cleanup without hiding the original failure +- Improved consistency between camera backend results and downstream capture, inference, and recording code