OME support

.gitignore

@@ -1,33 +1,35 @@
-*
-!src/
-!src/**
-!ScopeOneCore/
-!ScopeOneCore/**
-ScopeOneCore/install/**
-ScopeOneCore/build/**
-ScopeOneCore/external/**
-ScopeOneCore/python/**
-!scripts/
-!scripts/**
+/*
+!/src/
+!/src/**
+!/ScopeOneCore/
+!/ScopeOneCore/**
+/ScopeOneCore/install/
+/ScopeOneCore/build/
+/ScopeOneCore/external/*
+!/ScopeOneCore/external/ScopeWriter/
+!/ScopeOneCore/external/ScopeWriter/**
+/ScopeOneCore/python/*
+!/scripts/
+!/scripts/**
 
-!.github/
-!.github/**
+!/.github/
+!/.github/**
 
-!resources/
-!resources/**
+!/resources/
+!/resources/**
 
-!config/
-config/*
-!config/MMConfig_demo.cfg
-!config/dual_hamamatsu_897D.cfg
-!config/MMConfig_demo_dual.cfg
+!/config/
+/config/*
+!/config/MMConfig_demo.cfg
+!/config/dual_hamamatsu_897D.cfg
+!/config/MMConfig_demo_dual.cfg
 
-!CMakeLists.txt
-!README.md
-!VERSION
+!/CMakeLists.txt
+!/README.md
+!/VERSION
 
-!.gitignore
+!/.gitignore
 
-build/**
-ref/**
+/build/
+/ref/
 **/.DS_Store

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "ScopeOneCore/external/ScopeWriter"]
+	path = ScopeOneCore/external/ScopeWriter
+	url = https://github.com/Experimental-Microscopy-Lab/ScopeWriter.git

README.md

@@ -30,7 +30,7 @@ Download the latest release package from the [Releases](https://github.com/Exper
 
 **Device Adapter Setup:**
 
-ScopeOne loads Micro-Manager configuration files (.cfg) directly. We recommend installing [Micro-Manager 2.0](https://download.micro-manager.org/nightly/2.0/Windows/) to access the full device adapter library. The release package includes only a minimal set of device adapter DLLs. To add support for additional devices, simply copy the required DLLs(`mmgr_dal_xxx.dll`) from your Micro-Manager installation directory (typically `C:\Program Files\Micro-Manager-2.0`) to the root folder containing `ScopeOne.exe`. Besides, we kindly remind you first ensure your devices are working properly in Micro-Manager before using them in ScopeOne, as device compatibility issues are often related to the device adapter itself.
+ScopeOne loads Micro-Manager configuration files (.cfg) directly and includes the basic adapters required for its demo configurations. To access additional hardware adapters, install [Micro-Manager 2.0](https://download.micro-manager.org/nightly/2.0/Windows/). ScopeOne automatically detects the standard `C:\Program Files\Micro-Manager-2.0` installation; custom locations can be selected under Settings. Bundled adapters take precedence over external adapters. We recommend confirming that hardware works in the selected Micro-Manager installation before using it in ScopeOne.
 
 **Dual-camera Setup:**
 
@@ -44,23 +44,32 @@ There is an example dual-camera .cfg file in the config folder, just change the
 - [Visual Studio 2022](https://visualstudio.microsoft.com/vs/) (MSVC v143 toolset)
 - [Qt](https://www.qt.io/development/download-qt-installer-oss) 6.9.1 (msvc2022_64)
 - OpenCV 4.12.0
-- libtiff 4.7.1,
-- zlib 1.3.1
 - mmCoreAndDevices
 
-Extract the bundled third-party dependencies into `ScopeOneCore/external` under this repository. Expected third-party dependencies path layout:
+Clone ScopeOne and initialize all submodules with:
+
+```powershell
+git clone --recurse-submodules https://github.com/Experimental-Microscopy-Lab/ScopeOne.git
+```
+
+For an existing checkout, initialize the submodules with:
+
+```powershell
+git submodule update --init --recursive
+```
+
+The expected layout is:
 
 ```text
 ScopeOne/
   ScopeOneCore/
     external/
       mmCoreAndDevices/
       opencv-4.12.0/
-      tiff-4.7.1/
-      zlib-1.3.1/
+      ScopeWriter/
 ```
 
-<!-- Setting up these dependencies can be time-consuming. To simplify this, we provide a pre-packaged development source archive that includes OpenCV, MMCore, libtiff and zlib except Qt, VS and CMake. Download the development package from [Releases](https://github.com/Experimental-Microscopy-Lab/ScopeOne/releases). -->
+ScopeWriter contains its filesystem Zarr V3 writer and carries libtiff, zlib, zstd and crc32c under its own `third_party` directory. It builds these dependencies from source without downloading packages during CMake configuration.
 
 **Windows Build Steps:**
 
@@ -183,7 +192,7 @@ cp ScopeOneCore/external/mmCoreAndDevices/DeviceAdapters/DemoCamera/.libs/libmmg
 
 ## 🤖 Automation and AI Agents
 
-The desktop app exposes a language-neutral local control API and shared-memory frame channel. An AI agent does not run inside ScopeOne or depend on Python. A tool adapter can discover supported operation groups with the `capabilities` request, read a structured observation with `state_snapshot`, and invoke the exposed camera, stage, mosaic, processing, image analysis, experiment, recording, layer, and markup operations. Requests may carry an ID that is echoed by the app for correlation.
+The desktop app exposes a language-neutral local control API and shared-memory frame channel. An AI agent does not run inside ScopeOne or depend on Python. A tool adapter can discover supported operation groups with the `capabilities` request, read a structured observation with `state_snapshot`, and invoke the exposed camera, stage, mosaic, processing, image analysis, experiment, recording, layer, and markup operations. Requests may carry an ID that is echoed by the app for correlation. Recording save operations use `ome-tiff` by default and also accept `ome-zarr`, `tiff` and `binary`.
 
 The API reports which operations mutate hardware, write files, remove state, or may run for a long time. An agent adapter should request user confirmation before those operations and verify the result with the returned read-back value, experiment status, or a new state snapshot. The Python package is one optional client implementation. See the [Python client and Local API protocol guide](ScopeOneCore/python/scopeone/README.md) for protocol details and runnable examples.
 

ScopeOneCore/.gitignore

@@ -1,20 +1,20 @@
-!include/
-!internal/
+!/include/
+!/internal/
 !/src/
 
-!CMakeLists.txt
-!README.md
+!/CMakeLists.txt
+!/README.md
 
-!python/
-!python/scopeone/
-!python/scopeone/.gitignore
-!python/scopeone/README.md
-!python/scopeone/pyproject.toml
-!python/scopeone/examples/
-!python/scopeone/examples/**
-!python/scopeone/src/
-!python/scopeone/src/scopeone/
-!python/scopeone/src/scopeone/**/*.py
-!python/scopeone/src/scopeone/*.py
+!/python/
+!/python/scopeone/
+!/python/scopeone/.gitignore
+!/python/scopeone/README.md
+!/python/scopeone/pyproject.toml
+!/python/scopeone/examples/
+!/python/scopeone/examples/**
+!/python/scopeone/src/
+!/python/scopeone/src/scopeone/
+!/python/scopeone/src/scopeone/**/*.py
+!/python/scopeone/src/scopeone/*.py
 
 !/.gitignore

ScopeOneCore/CMakeLists.txt

@@ -1,19 +1,16 @@
-cmake_minimum_required(VERSION 3.16)
+cmake_minimum_required(VERSION 3.23)
 set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS "${CMAKE_CURRENT_LIST_DIR}/../VERSION")
 file(STRINGS "${CMAKE_CURRENT_LIST_DIR}/../VERSION" SCOPEONE_VERSION LIMIT_COUNT 1)
 project(ScopeOneCore VERSION "${SCOPEONE_VERSION}" LANGUAGES CXX)
 
 set(CMAKE_CXX_STANDARD 20)
-set(CMAKE_AUTOMOC ON)
-set(CMAKE_AUTORCC ON)
-set(CMAKE_AUTOUIC ON)
 
 include(GNUInstallDirs)
 include(CMakePackageConfigHelpers)
 
 set(CMAKE_INSTALL_PREFIX "${CMAKE_CURRENT_SOURCE_DIR}/install")
 if (WIN32)
-    set(CMAKE_PREFIX_PATH "C:/Qt/6.11.0/msvc2022_64")
+    list(PREPEND CMAKE_PREFIX_PATH "C:/Qt/6.11.0/msvc2022_64")
 endif ()
 
 set(MMCORE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/external/mmCoreAndDevices")
@@ -22,21 +19,19 @@ set(MMDEVICE_INCLUDE_DIR "${MMCORE_DIR}/MMDevice")
 
 find_package(Qt6 COMPONENTS Core Gui Network Concurrent REQUIRED)
 
+set(SCOPEWRITER_BUILD_TESTS OFF)
+add_subdirectory(external/ScopeWriter)
+
+set(CMAKE_AUTOMOC ON)
+set(CMAKE_AUTORCC ON)
+set(CMAKE_AUTOUIC ON)
+
 if (WIN32)
     set(MMCORE_BIN_DIR "${MMCORE_DIR}/build/Release/x64")
 
     set(OPENCV_DIR "${CMAKE_CURRENT_SOURCE_DIR}/external/opencv-4.12.0")
     set(OpenCV_DIR "${OPENCV_DIR}/build")
     set(OPENCV_BIN_DIR "${OPENCV_DIR}/build/x64/vc16/bin")
-    set(ZLIB_DIR "${CMAKE_CURRENT_SOURCE_DIR}/external/zlib-1.3.1")
-    set(ZLIB_BIN_DIR "${ZLIB_DIR}/build/Release")
-    set(ZLIB_INCLUDE_DIR "${ZLIB_DIR}" "${ZLIB_DIR}/build")
-    set(ZLIB_LIB "${ZLIB_BIN_DIR}/zlib.lib")
-
-    set(TIFF_DIR "${CMAKE_CURRENT_SOURCE_DIR}/external/tiff-4.7.1")
-    set(TIFF_BIN_DIR "${TIFF_DIR}/build/libtiff/Release")
-    set(TIFF_INCLUDE_DIRS "${TIFF_DIR}/libtiff;${TIFF_DIR}/build/libtiff")
-    set(TIFF_LIB "${TIFF_BIN_DIR}/tiff.lib")
 
     find_package(OpenCV REQUIRED)
 
@@ -45,13 +40,9 @@ if (WIN32)
             "${OPENCV_BIN_DIR}/opencv_videoio_ffmpeg*_64.dll"
     )
     file(GLOB MMCORE_RUNTIME_DLLS "${MMCORE_BIN_DIR}/*.dll")
-    set(TIFF_RUNTIME_DLL "${TIFF_BIN_DIR}/tiff.dll")
-    set(ZLIB_RUNTIME_DLL "${ZLIB_BIN_DIR}/zlib.dll")
 
     set(SCOPEONE_RUNTIME_DLLS
             ${OPENCV_RUNTIME_DLLS}
-            "${TIFF_RUNTIME_DLL}"
-            "${ZLIB_RUNTIME_DLL}"
             ${MMCORE_RUNTIME_DLLS}
     )
 
@@ -62,17 +53,10 @@ if (WIN32)
         endif ()
     endforeach ()
 else ()
-    # Linux/Unix: use system OpenCV/TIFF/ZLIB, and link the MMCore static libraries
+    # Linux/Unix: use system OpenCV and link the MMCore static libraries
     # built from the sibling micro-manager checkout (symlinked into
     # external/mmCoreAndDevices). No runtime DLLs to stage.
     find_package(OpenCV REQUIRED COMPONENTS core imgproc)
-    find_package(TIFF REQUIRED)
-    find_package(ZLIB REQUIRED)
-
-    set(TIFF_LIB TIFF::TIFF)
-    set(TIFF_INCLUDE_DIRS "")
-    set(ZLIB_LIB ZLIB::ZLIB)
-    set(ZLIB_INCLUDE_DIR "")
     set(MMCORE_BIN_DIR "")
 
     add_library(MMCore STATIC IMPORTED GLOBAL)
@@ -153,12 +137,10 @@ target_link_libraries(ScopeOneCore
         Qt::Network
         Qt::Concurrent
         ${OpenCV_LIBS}
-        ${TIFF_LIB}
-        ${ZLIB_LIB}
+        ScopeWriter::ScopeWriter
 )
 target_compile_definitions(ScopeOneCore PRIVATE
         MMDEVICE_CLIENT_BUILD
-        SCOPEONE_HAVE_TIFF
         SCOPEONE_CORE_EXPORTS
         SCOPEONE_CORE_VERSION_STRING="${PROJECT_VERSION}"
 )
@@ -171,8 +153,6 @@ target_include_directories(ScopeOneCore
         ${MMCORE_INCLUDE_DIR}
         ${MMDEVICE_INCLUDE_DIR}
         ${OpenCV_INCLUDE_DIRS}
-        ${TIFF_INCLUDE_DIRS}
-        ${ZLIB_INCLUDE_DIR}
 )
 target_precompile_headers(ScopeOneCore PRIVATE
         <QtGlobal>
@@ -205,6 +185,11 @@ target_precompile_headers(ScopeOneCore PRIVATE
 target_link_directories(ScopeOneCore PRIVATE ${MMCORE_BIN_DIR})
 
 scopeone_add_runtime_copy_commands(ScopeOneCore $<TARGET_FILE_DIR:ScopeOneCore>)
+add_custom_command(TARGET ScopeOneCore POST_BUILD
+        COMMAND ${CMAKE_COMMAND} -E copy_if_different
+        $<TARGET_FILE:ScopeWriter>
+        $<TARGET_FILE_DIR:ScopeOneCore>
+)
 
 add_executable(ScopeOne_Agent
         src/AgentMain.cpp

ScopeOneCore/README.md

@@ -109,8 +109,8 @@ External code should enter through these headers and `scopeone::core::ScopeOneCo
 
 ## Processing Data Flow
 
-`ImageFrame` is the frame model used by preview, processing, recording, gallery and the local API. Use `processFrameThrough(...)` to stop at one pipeline stage and `processFrameFrom(...)` to continue from a later module after an edited frame is written back. Saved TIFF and binary recording outputs are read back asynchronously through `ScopeOneCore::requestRecordingSessionFrame(...)`. Live preview processing and synchronous API processing use separate runtime pipeline state so offline frame edits do not change live module buffers.
+`ImageFrame` is the frame model used by preview, processing, recording, gallery and the local API. Use `processFrameThrough(...)` to stop at one pipeline stage and `processFrameFrom(...)` to continue from a later module after an edited frame is written back. Saved OME-TIFF, OME-Zarr, TIFF and binary recording outputs are read back asynchronously through `ScopeOneCore::requestRecordingSessionFrame(...)`. Live preview processing and synchronous API processing use separate runtime pipeline state so offline frame edits do not change live module buffers.
 
 Raw live frames, processed live frames, static tool/gallery frames, external API frames and session frame sources are routed through the core frame graph. UI preview widgets keep only a render cache, and callers should use `ScopeOneCore` frame facade methods instead of reading camera managers, recording sessions or preview cache state directly.
 
-`ExperimentPlan` is the single recording and MDA input contract. `ExperimentDocument` adds actual event results, software and device provenance, output files, stable image layers, pixel-to-sensor transforms and markups. Schema version 1 documents are validated strictly and can be round-tripped with `experimentDocumentToJson(...)`, `saveExperimentDocument(...)` and `loadExperimentDocument(...)`; image payloads remain in TIFF, binary files or shared memory.
+`ExperimentPlan` is the single recording and MDA input contract. `ExperimentDocument` adds actual event results, software and device provenance, output files, stable image layers, pixel-to-sensor transforms and markups. Schema version 2 documents include the sample-plane `pixelSizeUm` calibration, are validated strictly, and can be round-tripped with `experimentDocumentToJson(...)`, `saveExperimentDocument(...)` and `loadExperimentDocument(...)`; OME-TIFF is the default disk format. All OME-TIFF, OME-Zarr, plain TIFF and binary frame writing is delegated to the reusable ScopeWriter library. ScopeOne retains acquisition orchestration, experiment documents, output naming and reading.

ScopeOneCore/include/scopeone/ExperimentDocument.h

@@ -17,13 +17,15 @@
 
 namespace scopeone::core
 {
-    inline constexpr int kExperimentDocumentSchemaVersion = 1;
+    inline constexpr int kExperimentDocumentSchemaVersion = 2;
     inline constexpr int kProcessingModuleSchemaVersion = 1;
 
     enum class RecordingFormat
     {
         Tiff = 0,
-        Binary = 1
+        Binary = 1,
+        OmeTiff = 2,
+        OmeZarr = 3
     };
 
     enum class RecordingAxis
@@ -98,7 +100,7 @@ namespace scopeone::core
         int schemaVersion{kExperimentDocumentSchemaVersion};
         QString experimentId;
         QStringList cameraIds;
-        RecordingFormat format{RecordingFormat::Tiff};
+        RecordingFormat format{RecordingFormat::OmeTiff};
         bool streamToDisk{true};
         bool enableCompression{false};
         int compressionLevel{6};
@@ -108,6 +110,7 @@ namespace scopeone::core
         double burstIntervalMs{0.0};
         double mdaIntervalMs{0.0};
         double exposureMs{0.0};
+        double pixelSizeUm{0.0};
         std::vector<RecordingAxis> order{RecordingAxis::Time, RecordingAxis::Z, RecordingAxis::XY};
         std::vector<QPointF> positions;
         std::vector<double> zPositions;