Dev to Main

docs/trials_table_mapping.md

@@ -42,21 +42,6 @@ Columns are grouped by the raw source they map from.
 | `block_beta`, `block_duration`, `block_min`, `block_max` | `block_length` |
 | `delay_beta`, `delay_duration`, `delay_min`, `delay_max` | `quiescent_duration_key` (scalar distribution, so no beta/min/max) |
 
-### From `task_logic_input` (under `task_parameters`)
-
-| Trials column | Source field |
-| --- | --- |
-| `reward_size_left` | `task_parameters.reward_size.left_value_volume` — the reward volume (uL) at the left port. |
-| `reward_size_right` | `task_parameters.reward_size.right_value_volume` — the reward volume (uL) at the right port. |
-
-> **Note:** `reward_size` is read from the task parameters, not the trial
-> generator, so it is populated even when no summarising generator is resolved.
-> The acquisition system can in principle vary reward size per trial, but the
-> current data format only exposes a single session-level value, so these
-> columns are constant across trials. They are **required** (non-nullable): a
-> missing `TaskLogic` stream raises rather than silently producing null reward
-> sizes when there are trials to build.
-
 ### From `TrialMetrics.json` (`SoftwareEvents` stream)
 
 | Trials column | Mapping |
@@ -81,14 +66,16 @@ Columns are grouped by the raw source they map from.
 | `response_duration` | `response_deadline_duration`. |
 | `reward_consumption_duration` | `Trial -> reward_consumption_duration`. |
 | `reward_probabilityL` / `reward_probabilityR` | The **block** probability from `Trial -> metadata -> p_reward_left` / `p_reward_right`. The top-level `trial.p_reward_left` / `p_reward_right` is the per-trial probability, not the block probability, so it is not used here. `None` when the trial or its metadata is missing. |
+| `reward_size_left` | `Trial -> reward_size.left` — the reward volume (uL) at the left port. Defaults to `2.0` when not set on the trial. `None` when the trial is missing. |
+| `reward_size_right` | `Trial -> reward_size.right` — the reward volume (uL) at the right port. Defaults to `2.0` when not set on the trial. `None` when the trial is missing. |
 | `rewarded_historyL` / `rewarded_historyR` | Filter `is_rewarded == True`, then on `is_right_choice`. |
 
 ### From `TrialGeneratorSpec.json` (`SoftwareEvents` stream)
 
 | Trials column | Mapping |
 | --- | --- |
 | `base_reward_probability_sum` | If `type == "CoupledTrialGenerator"`, look at `reward_probability_parameters`. |
-| `min_reward_each_block` | Present when `type == "CoupledTrialGenerator"`; otherwise `None`. |
+| `min_reward_each_block` | Present when `type == "CoupledWarmupTrialGenerator"` (has `min_block_reward`); otherwise `None`. |
 
 ### From `QuiescentPeriod.json` (`SoftwareEvents` stream)
 
@@ -123,11 +110,11 @@ seconds, far too coarse for the ~tens-of-ms valve pulse.
 | --- | --- |
 | `goCue_start_time` | `PlaySoundOrFrequency` `WRITE` message. |
 
-### From `InitialManipulatorPosition` (software event)
+### From `HarpManipulator` `AccumulatedSteps` (+ `InputSchemas.Rig`)
 
 | Trials column | Mapping |
 | --- | --- |
-| `lickspout_positions` | `data` field. |
+| `lickspout_position_x` / `y1` / `y2` / `z` | Per-motor cumulative microstep count from the `AccumulatedSteps` stream, converted to millimetres via the rig manipulator calibration (`full_step_to_mm / microstep_resolution`) and re-referenced to the session-start position (displacement **relative to session start**, mm). The manipulator is a continuously-sampled hardware value, so — like the go cue — each trial takes the sample within its `[start_time, stop_time)` window nearest the start. `Motor{i}` drives `Axis(i + 1)` (X, Y1, Y2, Z). `None` when no sample falls in the trial window. The rig and `AccumulatedSteps` streams are required inputs (`build` raises if either is missing with trials present). |
 
 ### From `trainer_state.json` and `acquisition.json` (autoTrain — can be disregarded)
 
@@ -155,3 +142,5 @@ These were mapped during exploration but are no longer in scope:
 | 2026-06-17 | `auto_waterL` / `auto_waterR` now encode no auto-response (`is_auto_reward_right` is `None`) and missing trials as `0` instead of `NULL`. The columns are non-nullable (`int`, default `0`). |
 | 2026-06-20 | Added `reward_size_left` / `reward_size_right` (reward volume in uL) from `task_parameters.reward_size`, and `side_bias` from the per-trial `TrialMetrics` event (`bias` field). |
 | 2026-06-20 | `reward_probabilityL` / `reward_probabilityR` now read the block probability from `trial.metadata.p_reward_left` / `p_reward_right` instead of the top-level per-trial `trial.p_reward_left` / `p_reward_right`. |
+| 2026-07-22 | `lickspout_position_x` / `y1` / `y2` / `z` now derive from the `HarpManipulator` `AccumulatedSteps` stream (microsteps → mm via the `InputSchemas.Rig` manipulator calibration, `full_step_to_mm / microstep_resolution`), sampled per trial via the closest sample in the `[start_time, stop_time)` window and re-referenced to the session-start position (displacement relative to session start, mm), replacing the static `InitialManipulatorPosition` software event. `Motor{i}` maps to `Axis(i + 1)` (X, Y1, Y2, Z). The rig and `AccumulatedSteps` streams are required when there are trials (`build` raises if either is missing). Column descriptions corrected from `um` to `mm`. |
+| 2026-07-24 | `reward_size_left` / `reward_size_right` moved from session-level `task_parameters.reward_size` to per-trial `Trial.reward_size` (fields `.left` / `.right`). The columns are now nullable — `None` when the trial is missing. A missing `TaskLogic` stream no longer raises; session distribution columns are simply null. `min_reward_each_block` moved from `CoupledTrialGenerator` to `CoupledWarmupTrialGenerator`. |

pyproject.toml

@@ -17,7 +17,7 @@ readme = "README.md"
 version = "0.1.0"
 
 dependencies = [
-    "aind-behavior-dynamic-foraging[data] @ git+https://github.com/AllenNeuralDynamics/Aind.Behavior.DynamicForaging.git@baab12133b22f599c1ba0583260eca9eca216cc0",
+    "aind-behavior-dynamic-foraging[data] @ git+https://github.com/AllenNeuralDynamics/Aind.Behavior.DynamicForaging.git@ac5ddbf909c9375b9e8875d6a5f90796cfa98653",
     "ipykernel",
 ]
 
@@ -37,7 +37,7 @@ full = [
 
 [dependency-groups]
 dev = [
-    'ruff',
+    'ruff<0.16',
     'coverage',
     'interrogate',
     'pytest',

src/dynamic_foraging_processing/nwb/acquisition/acquisition_builder.py

@@ -106,8 +106,7 @@ def get_lick_times(self, device: str, stream_name: str, port: str) -> np.ndarray
         ``HarpBehavior``/``DigitalInputState``, with left licks on ``DIPort0``
         and right licks on ``DIPort1``. The lickometer board exposes each side
         as its own device (``HarpLickometerLeft`` / ``HarpLickometerRight``)
-        with a ``LickState`` stream and a ``Channel0`` column. A lick time is a
-        timestamp at which the selected column's digital input is high.
+        with a ``LickState`` stream and a ``Channel0`` column.
 
         Parameters
         ----------
@@ -131,6 +130,7 @@ def get_lick_times(self, device: str, stream_name: str, port: str) -> np.ndarray
             data = self.loader.dataset.at("Behavior").at(device).at(stream_name).load().data
         except (KeyError, FileNotFoundError):
             return np.array([])
+        data = data[data["MessageType"] == "EVENT"]
         licks = data[data[port].fillna(False).astype(bool)]
         return licks.index.to_numpy()
 

src/dynamic_foraging_processing/pipeline/_pipeline.py

@@ -26,6 +26,7 @@
 import numpy as np
 import pandas as pd
 import pynwb
+from aind_data_schema.components.identifiers import DataAsset
 from aind_data_schema.core.processing import (
     Code,
     DataProcess,
@@ -279,8 +280,17 @@ def _write_processing(
                     experimenters=["Alex Piet", "Micah Woodard", "Bruno Cruz", "Arjun Sridhar"],
                     start_date_time=start_date_time,
                     end_date_time=end_date_time,
+                    pipeline_name="dynamic-foraging-processing-pipeline",
                 )
-            ]
+            ],
+            pipelines=[
+                Code(
+                    url=_CODE_URL,
+                    version=_PACKAGE_VERSION,
+                    name="dynamic-foraging-processing-pipeline",
+                    input_data=[DataAsset(name=Path(self.loader.path).stem)],
+                )
+            ],
         )
         processing.write_standard_file(output_directory=Path(output_path))