Separate recording logic from GUI#92
Draft
C-Achard wants to merge 14 commits into
Draft
Conversation
C-Achard
added
enhancement
C-Achard
force-pushed
the
cy/processor-fixes
branch
from
52f9b92 to
a800c08
Compare
Centralized recording container configuration by introducing shared allowed/default container constants and using them across settings, UI, and utilities. The recording UI now populates container options from config and keeps the filename extension aligned with the selected container when switching between known video formats, so saved settings and path previews stay consistent.
Introduce a new `DLC_DO_LOG_TIMING` config flag and wire `WorkerTimingStats` into `DLCLiveMainWindow` for DLC enqueue and pose-ready callback timing. The pose callback now logs camera-to-GUI latency in debug mode, marks display state dirty instead of forcing an immediate redraw, and emits periodic timing stats via `maybe_log()`.
Update `_enqueue_frame` to keep enqueueing the newest frame by removing one queued item when `put_nowait` hits `queue.Full`, instead of dropping the incoming frame. This makes processing more real-time under load and keeps enqueue/drop stats consistent, including safe `task_done()` handling.
Demoted multiple verbose runtime messages from INFO to DEBUG in Basler and GenTL backends. This keeps normal logs cleaner by moving routine configuration/readback details (FPS setup, converter mode, exposure/gain settings, trigger configuration, and startup/close diagnostics) out of INFO-level output while preserving the diagnostics when DEBUG is enabled.
Adds fine-grained `WorkerTimingStats` instrumentation across enqueue, initialization, inference, and emit paths, with error/frame accounting and optional timing logging. It also makes worker startup/stop behavior safer by deferring queue creation until RUNNING, blocking enqueue during STARTING, normalizing input frames before inference, and adding richer debug diagnostics (CUDA/runner state and thread stack dumps on stuck shutdown).
Comment out the camera-to-GUI pose latency calculation and debug log in `pose_ready_callback`. This removes the try/except-wrapped timing log path while leaving pose handling and display update behavior unchanged.
Reorders preview teardown so inference is stopped before stopping the multi-camera controller. This avoids stopping the controller while inference is still active and keeps shutdown state cleanup consistent.
Adds a `preserve_mono` option to the GenTL camera backend so Mono pixel formats can remain 2D Mono8 output instead of always converting to BGR. The backend now reports preserve-mono capability, exposes recommended/actual output format based on camera format, and persists detected pixel/output format metadata in settings. It also logs first-frame format details to make runtime format behavior easier to inspect.
Move recording shutdown off the UI thread and finalize it via a Qt signal so stop actions no longer block the interface. RecordingManager now uses a lock-protected recorder map plus a bounded background frame-dispatch queue/thread to decouple frame intake from disk writes, drop frames under sustained backpressure, and cleanly stop/tear down recorders. The recording-with-overlays UI path is also disabled in this change.
Extracts `SingleCameraWorker` into a new `services/camera_controller.py` module and moves per-frame rotation/cropping plus recording-sink writes into the worker thread. `MultiCameraController` now injects and updates a shared recording sink on workers, and recording enable/disable is propagated directly to active workers. The previous recording-frame emission path and transform handling in the multi-camera slot are removed/commented out to avoid duplicate processing and keep the controller focused on frame aggregation. Also relocates `recording_manager.py` from `gui` to `services` with a file rename.
Switch `RecordingManager` import to the new `services` package path and update recording flow to use an explicit recording sink callback. Recording now sets `multi_camera_controller.set_recording_sink(self._rec_manager.write_frame)` when starting and clears it on stop, replacing the previous direct `recording_frame_ready` signal hookup.
Adjust test fixtures and GUI recording manager tests to import `RecordingManager` and related module symbols from `dlclivegui.services.recording_manager` instead of the old `dlclivegui.gui.recording_manager` path.
C-Achard
force-pushed
the
cy/separate-recording
branch
from
c48e082 to
688cf18
Compare
Update the single-camera pipeline to use structured capture results from backend reads, extracting `frame`, `software_timestamp`, and `timestamp_metadata` and forwarding metadata through frame emission and recording writes. Recording queue handling now expects the metadata field as well. Preview transform helpers were also switched to reuse `SingleCameraWorker` crop/rotation methods instead of `MultiCameraController`.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR continues the “separate recording logic from GUI” refactor by moving/rewiring recording-related responsibilities into dlclivegui/services, shifting multi-camera frame-to-recording flow to a direct sink callback, and adding background dispatch/stop behaviors to reduce GUI-thread load.
Changes:
- Update tests and GUI wiring to use
dlclivegui.services.recording_manager.RecordingManagerinstead of the GUI module. - Introduce a
SingleCameraWorkerservice module and route recording frames via a configurable “recording sink” rather than a GUI Qt signal path. - Add/adjust performance/timing instrumentation and input normalization in DLC processing; improve container handling defaults; extend camera backends to optionally preserve mono output and reduce log verbosity.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/gui/test_rec_manager.py | Switch test imports/patching to the new services-based RecordingManager module. |
| tests/conftest.py | Update monkeypatch targets for RecordingManager/VideoRecorder/build_run_dir to services paths. |
| dlclivegui/utils/utils.py | Use DEFAULT_RECORDING_CONTAINER for filename splitting; add helper to format a thread’s stack for debugging. |
| dlclivegui/utils/stats.py | Track per-measurement elapsed time in timing context manager. |
| dlclivegui/services/video_recorder.py | Ensure -input_framerate is passed as a float. |
| dlclivegui/services/recording_manager.py | Add locking + background dispatcher queue for frame writes; make stop/start and stats access thread-safe. |
| dlclivegui/services/multi_camera_controller.py | Move SingleCameraWorker out; add recording sink plumbing; adjust recording enable behavior and disable in-controller transforms/emission path. |
| dlclivegui/services/dlc_processor.py | Add timing instrumentation, queue drop strategy, thread-stack diagnostics, and frame normalization; refactor inference call path. |
| dlclivegui/services/camera_controller.py | New module containing SingleCameraWorker, including per-worker transforms and recording sink invocation. |
| dlclivegui/gui/main_window.py | Wire RecordingManager from services; add container/filename alignment logic; switch recording to sink-based path; async stop recording. |
| dlclivegui/gui/camera_config/preview.py | Use SingleCameraWorker transform helpers now that MultiCameraController ones were removed. |
| dlclivegui/config.py | Add allowed/default recording container constants; adjust RecordingSettings defaults; change DLC timing flag default. |
| dlclivegui/cameras/backends/gentl_backend.py | Add optional mono-preservation path + richer runtime format reporting; reduce some INFO logs to DEBUG. |
| dlclivegui/cameras/backends/basler_backend.py | Reduce verbose INFO logs to DEBUG; add/align mono-preservation logging. |
Comments suppressed due to low confidence (2)
dlclivegui/services/recording_manager.py:144
_stop_dispatcher()clears_dispatch_stopand nulls_dispatch_thread/_frame_queueeven when the dispatcher thread is still alive. Ifjoin()times out (or sentinel can't be enqueued), clearing the stop event can let the still-running thread continue indefinitely, potentially writing frames after stop and leaking the thread.
dlclivegui/services/recording_manager.py:114- The dispatcher queue size is hard-coded to 4096 frames, which can consume a very large amount of RAM with high-resolution frames before backpressure/drop behavior kicks in. Consider reducing this default or making it configurable so recording can't OOM the process under load.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| class SingleCameraWorker(QObject): | ||
| """Worker for a single camera in multi-camera mode.""" | ||
|
|
||
| frame_captured = Signal(str, object, float) # camera_id, frame, timestamp |
|
|
||
| with timing.measure("Multi.slot.total"): | ||
| settings = self._settings.get(camera_id) | ||
| self._settings.get(camera_id) |
| SINGLE_CAMERA_WORKER_DO_LOG_TIMING: bool = False | ||
| MULTI_CAMERA_WORKER_DO_LOG_TIMING: bool = False | ||
| REC_DO_LOG_TIMING: bool = False | ||
| DLC_DO_LOG_TIMING: bool = True |
Comment on lines
+587
to
+594
| with self._timing.measure("DLC.process_frame"): | ||
| processed_frame = self._dlc.process_frame(frame) | ||
|
|
||
| with self._timing.measure("DLC.runner.get_pose"): | ||
| self._dlc.pose = self._dlc.runner.get_pose(processed_frame) | ||
|
|
||
| with self._timing.measure("DLC.post_process_pose"): | ||
| raw_pose: Any = self._dlc._post_process_pose(processed_frame, frame_time=timestamp) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.