sollertia-experiment

Provides data acquisition and preprocessing runtimes for Sollertia platform data acquisition systems.

Detailed Description

This library is part of the Sollertia AI-assisted scientific data acquisition and processing platform, built on the Ataraxis framework and developed in the Sun (NeuroAI) lab at Cornell University. It is the system-agnostic data acquisition and preprocessing runtime of the platform: it manages the hardware, sessions, and preprocessing pipeline for one or more data acquisition systems and is designed to be extended with new systems over time. The Mesoscope-VR two-photon imaging system is the current reference acquisition system that ships with the library.

Every Sollertia acquisition system runs a Virtual Reality task, presenting an animal with a Unity-rendered linear infinite corridor while the system records behavior and (where applicable) brain activity data. The library provides the shared machinery these systems reuse — the session data hierarchy, the hardware-agnostic Virtual Reality task driver, and the acquisition-system configuration registry — alongside the bindings and runtime logic that are specific to each system. The system-specific code is tightly integrated with the platform hardware and is generally not designed to be reused in any other context; the shared machinery, however, is the reusable substrate new systems plug into. See the Data Acquisition Systems section for more details on currently supported acquisition systems and how the platform is extended.

The library pairs with the shared dataclasses, data hierarchy, and configuration tooling provided by sollertia-shared-assets, which supplies the AcquisitionSystems-keyed registry backbone that new systems register against, and feeds the downstream data processing pipeline exposed by sollertia-forgery. The platform ships a Claude Code marketplace whose experiment plugin exposes the library's sle mcp server and a set of domain skills for AI-assisted use; human operators interact with the library through its command line interface, while AI agents work through the MCP server and skills (see AI-Assisted Use).

Features

System-agnostic acquisition and preprocessing runtime, designed to be extended with new acquisition systems.
Mesoscope-VR reference acquisition system combining two-photon imaging with a Unity Virtual Reality behavioral task.
A single consolidated sle command line interface for hardware discovery, configuration, acquisition, and data management.
Hardware abstraction through binding classes for Zaber motors, machine-vision cameras, and microcontrollers.
Session-based data management with distributed, redundant long-term storage.
An sle mcp MCP server and a Claude Code marketplace plugin for AI-assisted operation.
Apache 2.0 License.

Dependencies

For users, all Python library dependencies are installed automatically by all supported installation methods. Each acquisition system additionally relies on external software and hardware components that must be installed and configured separately; for the Mesoscope-VR system, these are documented under the Mesoscope-VR Data Acquisition System section.

For developers, see the Developers section for information on installing additional development dependencies.

Installation

Source

Note, installation from source is highly discouraged for anyone who is not an active project developer.

Download this repository to the local machine using the preferred method, such as git-cloning. Use one of the stable releases that include precompiled binary and source code distribution (sdist) wheels.
If the downloaded distribution is stored as a compressed archive, unpack it using the appropriate decompression tool.
cd to the root directory of the prepared project distribution.
Run pip install . to install the project and its dependencies. Alternatively, if using a distribution with precompiled binaries, use pip install WHEEL_PATH, replacing 'WHEEL_PATH' with the path to the wheel file.

pip

Use the following command to install the library and all of its dependencies via pip: pip install sollertia-experiment

Data Acquisition Systems

A data acquisition (and runtime control) system can be broadly defined as a collection of hardware and software tools used to conduct training or experiment sessions that acquire scientific data. Each data acquisition system can use one or more machines (PCs) to acquire the data, with this library (sollertia-experiment) typically running on the main data acquisition machine. Additionally, each system can use any number of long-term storage destinations, such as a remote storage Server, a Network-Attached-Storage (NAS) volume, or both, to safely store the data after the acquisition (with redundancy and parity).

In the Sollertia platform, every data acquisition system runs a Unity Virtual Reality task in a linear infinite corridor and is built around the main tool used to acquire its scientific data. For example, the current reference system is the Mesoscope-VR system, which is built around the 2-Photon Random Access Mesoscope (2P-RAM) used to acquire brain activity data. Other systems may center on a different acquisition modality. Due to this inherent specialization, each Sollertia platform data acquisition system is treated as an independent unit that contributes its own bindings, runtime logic, and preprocessing steps, while plugging into the shared platform machinery for everything else.

Note, Since each data acquisition system is unique, this section is iteratively expanded to include system-specific assembly instructions for each supported acquisition system (each gets its own section). Commonly, updates to this section coincide with major or minor library version updates.

Extending the Platform

The library is designed so that a new acquisition system can be added by composing the shared machinery rather than rewriting it. The reusable substrate spans this library and sollertia-shared-assets:

The acquisition-system configuration registry. This library defines a shared SystemConfiguration base class and an AcquisitionSystems-keyed registry in cross_system/system_configuration.py. Each system defines a concrete subclass (the Mesoscope-VR system contributes MesoscopeSystemConfiguration in mesoscope_vr/system.py) and registers it at import time via register_system_configuration(). Registration is the only system-specific wiring the configuration file lifecycle requires; resolution, creation, and loading of the on-disk YAML are all shared.
The shared session data hierarchy and registries. The 4-level root/project/animal/session hierarchy, the SessionData model, and the per-system record registries (session descriptors, hardware states, and experiment configurations) are owned by sollertia-shared-assets. A new system contributes its record schemas to those registries and reuses the rest of the hierarchy unchanged.
The hardware-agnostic Virtual Reality task driver. The vr_task driver encapsulates the MQTT contract with the Unity game engine, the editor bridge that drives scene activation and Play Mode, and the cue-sequence-to-trial decomposition. The driver is hardware-agnostic: it surfaces typed Virtual Reality task events that any acquisition system can dispatch against its own hardware.

Note, the platform does not expose a generic runtime base class. MesoscopeVRSystem (in mesoscope_vr/system_controller.py) is currently the only acquisition-system controller; each new system implements its own controller against the shared seams above rather than subclassing a common runtime.

For the end-to-end process of designing and building a new acquisition system, use the experiment plugin's system-design-pipeline and acquisition-system-design skills (plus acquisition-system-runtime for the runtime layer). The microcontroller-interface, zaber-interface, and vr-driver-interface skills cover the individual hardware and Unity coupling seams. See AI-Assisted Use for the full skill inventory.

Mesoscope-VR Data Acquisition System

This is the current reference Sollertia platform data acquisition system. The system broadly consists of four major parts:

The 2-Photon Random Access Mesoscope (2P-RAM), assembled by Thor Labs and controlled by ScanImage software. The Mesoscope control and data acquisition are performed by a dedicated computer referred to as the 'ScanImagePC'. This PC is assembled and configured by MBF Bioscience. The only modification carried out during assembly was the configuration of a Server Message Block (SMB) protocol access to the root directory used by the ScanImage software to save the Mesoscope data.
The Unity game engine running the Virtual Reality game world used in all experiments to control the task environment and resolve the task logic. The virtual environment runs on the main data acquisition computer referred to as the 'VRPC' and relies on the MQTT communication protocol and the sollertia-unity-tasks Unity package to bidirectionally interface with the virtual task environment.
The microcontroller-powered hardware that allows the animal to bidirectionally interface with various physical components (modules) of the Mesoscope-VR system.
A set of visual and IR-range cameras, used to acquire behavior video data.

Note, the hardware inventory, configuration parameters, and calibration workflows for this system are documented in depth by the mesoscope plugin's mesoscope-vr skill. Use the experiment plugin's system-health-check and acquisition-system-setup skills to verify the dependencies below before running sessions.

Main Dependency

Linux operating system. While the library may also work on Windows and (less likely) macOS, it has been explicitly written for and tested on the mainline 6.18 kernel and Ubuntu 24.04 LTS distribution of the GNU Linux operating system using Wayland window system architecture.

Software Dependencies

Note, This list only includes external dependencies, which are installed in addition to all dependencies automatically installed from pip / conda as part of library installation. The dependencies below have to be installed and configured on the VRPC before calling runtime commands via the command line interface (CLI) exposed by this library.

MQTT broker version 2.0.22. The broker runs on the VRPC, and the VRPC-side runtime reaches it on the default loopback IP (127.0.0.1) and Port (1883). Controlling the Mesoscope from a separate ScanImagePC additionally requires exposing the broker on the local network; see MQTT Broker Access.
FFMPEG. As a minimum, the version of FFMPEG should support H265 and H264 codecs with hardware acceleration (Nvidia GPU). This library was tested with the version 8.0.1.
MvImpactAcquire. This library is tested with version 2.9.2, which is freely distributed. Higher GenTL producer versions likely work too, but they require purchasing a license.
Zaber Launcher version 2025.12.30-1.
Unity Game Engine version 6000.3.3f1 LTS.

Hardware Dependencies

Note, These dependencies only apply to the VRPC. Hardware dependencies for the ScanImagePC are determined and controlled by MBF and ThorLabs. This library benefits from the ScanImagePC being outfitted with a 10-GB network card, but this is not a strict requirement.

Nvidia GPU. This library uses GPU hardware acceleration to encode acquired video data. Any Nvidia GPU with hardware encoding chip(s) should work as expected. The library was tested with RTX 4090.
A CPU with at least 12, preferably 16, physical cores. This library has been tested with AMD Ryzen 7950X CPU. It is recommended to use CPUs with 'full' cores, instead of those using a mixture of 'efficiency' and 'performance' cores for predictable performance of all library components.
A 10-Gigabit capable motherboard or Ethernet adapter, such as X550-T2. Primarily, this is required for the high-quality machine vision cameras used to record videos of the animal's face and body. The 10-Gigabit lines are also used for transferring the data between the PCs used in the data acquisition process and the destination machines used for long-term data storage (see acquired data management section for more details).

System Assembly

The Mesoscope-VR system consists of multiple interdependent components. It is continually refined with minor changes to optimize its performance and facilitate novel experiments and projects. Treat this section as a general system composition guide, but consult the original publication for each project over this section for instructions on building specific system implementations used to acquire the data featured in different publications.

Physical assembly and mounting of all hardware components mentioned in the specific subsections below is discussed in the main Mesoscope-VR assembly section.

Zaber Motors

All brain activity recordings with the Mesoscope require the animal to be head-fixed. To orient head-fixed animals on the Virtual Reality treadmill (running wheel) and promote task performance, the system uses three groups of motors controlled through Zaber motor controllers. The first group, the HeadBar, is used to position the animal's head in Z, Pitch, and Roll axes. Together with the movement axes of the Mesoscope, this allows for a wide range of motions necessary to align the Mesoscope objective with the brain imaging plane. The second group of motors, the LickPort, controls the position of the water delivery port (tube) (and sensor) in X, Y, and Z axes. This is used to ensure all animals have comfortable access to the water delivery tube, regardless of their head position. This motor group also controls the position of the aversive air puff valve, which is mounted directly above the LickPort. The third group of motors, the Wheel, controls the position of the running wheel in the X-axis relative to the head-fixed animal's body and is used to position the animal on the running wheel to promote good running behavior.

The current snapshot of Zaber motor configurations used by the Sollertia platform, alongside the motor parts list and electrical wiring instructions, is available in the Zaber configuration snapshot folder.

Warning! Zaber motors have to be configured correctly to work with this library. To (re)configure the motors to work with the library, apply the setting snapshots from the link above via the Zaber Launcher software. Read the instructions in the 'Applying Zaber Configuration' document for the correct application procedure.

Although highly discouraged, it is also possible to edit the motor settings manually. To configure the motors to work with this library, overwrite the non-volatile User Data of each motor device (controller) with the data expected by the shared cross_system Zaber binding layer:

User Data 0: Device CRC Code. This variable should store the CRC32-XFER checksum of the device's name (user-defined name). During runtime, the library generates the CRC32-XFER checksum of each device's name and compares it against the value stored inside the User Data 0 variable to ensure that each device is configured appropriately to work with the sollertia-experiment library. Note, Use the sle get checksum console command to generate the CRC32-XFER checksum for each device during manual configuration, as it uses the same code as used during runtime and, therefore, guarantees that the checksums match.
User Data 1: Device ShutDown Flag. This variable tracks whether the device was properly shut down during previous runtimes. The library sets this variable to 0 when each device is initialized at runtime startup and back to 1 during proper shutdown. This flag is only checked for devices marked as "unsafe" (see User Data 10 below). For unsafe devices that were not properly shut down, the system prompts for user confirmation before re-homing the motor at the beginning of the next runtime.
User Data 10: Device Unsafe Flag. This variable should be set to 1 for motors that can be positioned in a way that is not safe to home after power cycling (e.g., motors that could collide with other components during homing). When a motor marked as unsafe is not properly shut down (User Data 1 = 0), the system prompts the user for confirmation before proceeding with the homing sequence.
User Data 11: Device Park Position. This variable should be set to the position, in native motor units, where the device should be moved as part of the 'park' command and the shut-down sequence. This is used to position all motors in a way that guarantees they can be safely 'homed' at the beginning of the next runtime. Therefore, each park position has to be selected so that each motor can move to their 'home' sensor without colliding with any other motor simultaneously moving towards their 'home' position. Note, The LickPort uses the 'park' position as the default imaging position. During runtime, it moves to the 'park' position if it has no animal-specific position to use during imaging. Therefore, the park position for the LickPort should always be set so that it cannot harm the animal mounted in the Mesoscope enclosure while moving to the park position from any other position.
User Data 12: Device Maintenance Position. This variable should be set to the position, in native motor units, where the device should be moved as part of the 'maintain' command. Primarily, this position is used during water delivery system calibration and the running-wheel surface maintenance. Typically, this position is calibrated to provide easy access to all hardware components of the system by moving all motors as far away from each other as reasonable.
User Data 13: Device Mount Position. This variable should be set to the position, in native motor units, where the device should be moved as part of the 'mount' command. For the LickPort, this position is usually far away from the animal, which facilitates mounting and unmounting the animal from the rig. For the HeadBar and the Wheel motor groups, this position is used as the default imaging position. Therefore, the HeadBar and the Wheel 'mount' positions should be set so that any (new) animal can be comfortably and safely mounted in the Mesoscope enclosure.

Note, for the binding-class patterns behind the User Data scheme and for safe manual motor positioning, use the experiment plugin's zaber-interface skill.

Behavior Cameras

To record the animal's behavior, the system uses two high-end machine-vision cameras with 2-MegaPixel resolution. The face_camera records the animal's face, while the body_camera records the body of the animal. To interface with the cameras, the system leverages customized ataraxis-video-system bindings.

Specific information about the cameras and related imaging hardware, as well as the snapshot of the configuration parameters used by each camera, is available in the camera configuration snapshot folder.

Note, for low-level camera configuration, use the video plugin's camera skills; for how the cameras integrate with the Mesoscope-VR system, use the mesoscope plugin's mesoscope-vr skill.

MicroControllers

To interface with all other hardware components other than cameras and Zaber motors, the Mesoscope-VR system uses Teensy 4.1 microcontrollers with specialized ataraxis-micro-controller code. Currently, the system uses three isolated microcontroller subsystems: Actor, Sensor, and Encoder.

For instructions on assembling and wiring the electronic components used in each microcontroller system, as well as the code running on each microcontroller, see the microcontroller repository.

Note, for the registry of paired Module and ModuleInterface classes, use the experiment plugin's microcontroller-interface skill; the communication and microcontroller plugins cover the underlying base API and firmware side.

Virtual Reality Task Environment (Unity)

The task environment used in all Mesoscope-VR experiments is rendered and controlled by the Unity game engine. To make Unity work with this library, each project-specific Unity task must use the bindings and assets released as part of the sollertia-unity-tasks repository. Follow the instructions from that repository to set up the Unity game engine to interface with this library and to create new virtual task environments.

Note, The Unity Editor must be running with the project open before starting an experiment session. The editor exposes an HTTP MCP Bridge that starts automatically with it, and this library uses that bridge to open the correct scene and to 'arm' (enter Play Mode) and 'disarm' (exit Play Mode) the task automatically. The operator does not open scenes or press the Unity 'play' button manually; they only confirm that the Virtual Reality display renders correctly during setup. Use the sle get unity command to verify the bridge is reachable before running a session.

Coordination via MQTT

The sollertia-experiment library and Unity coordinate the running task bidirectionally using the MQTT protocol. This communication is used to:

Verify scene configuration: At startup, sollertia-experiment requests the active Unity scene name and validates it against the scene name expected from the experiment configuration.
Decompose the cue sequence: Unity sends the active VR environment's wall cue sequence to sollertia-experiment, which decomposes it into individual trials. The spatial trial layout (per-trial cue sequences and segment lengths) is sourced from the Virtual Reality task template (defined in sollertia-unity-tasks and persisted as vr_configuration.yaml), not from the experiment configuration. The acquisition-side per-trial parameters (reward size, gas puff duration, etc.) from the experiment configuration are then joined back to each decomposed trial by trial name.
Synchronize runtime state: sollertia-experiment sends the per-cycle animal motion delta to Unity and receives stimulus-trigger events (each carrying the name of the trial that fired).
Control task guidance: sollertia-experiment sends task guidance state updates based on animal performance.

Two configuration files are preserved with each Virtual Reality task session's raw data for reproducibility: the experiment configuration (experiment_configuration.yaml, the acquisition-side experiment parameters) and the Virtual Reality task template (vr_configuration.yaml, the corridor cues, VR environment, and trial structures Unity renders). Together they capture both halves of the experiment.

Note, for the MQTT topic contract, the editor bridge, and the cue-sequence decomposition, use the experiment plugin's vr-driver-interface skill; for authoring tasks, scenes, and task templates on the Unity side, use the unity plugin.

Google Sheets API Integration

This library interacts with the shared Google Sheet files used by the Sollertia platform to track and communicate certain information about the animals that participate in all projects. Currently, this includes two files: the surgery log and the water restriction log. Primarily, this integration is used to ensure that all information about each experiment subject (animal) is stored in the same location (on the long-term storage machine(s)). Additionally, it is used to automate certain data logging tasks.

The boundary between the libraries is as follows: the credential-file plumbing (the platform credentials directory and the slsa configure credentials command) is owned by sollertia-shared-assets, while the actual surgery-log and water-log sheet reading and writing is owned by this library. Each sheet is optional and is only processed when its identifier (surgery_sheet_id, water_log_sheet_id) is set in the Mesoscope-VR system configuration.

Setting up Google Sheets API Access

Skip to the next section if a service Google Sheets API account already exists. Most lab members can safely ignore this section, as service accounts are managed at the platform level rather than by individual lab members.

Log into the Google Cloud Console.
Create a new project.
Navigate to APIs & Services → Library and enable the Google Sheets API for the project.
Under IAM & Admin → Service Accounts, create a service account. This generates a service account ID in the format of service-account@gserviceaccount.com.
Use Actions → Manage Keys and, if a key does not already exist, create a new key and download it in JSON format. This key is then used to access the Google Sheets.

Registering the Service Account Key

The downloaded key must be registered with the Sollertia platform on the VRPC before any runtime can read or write the Google Sheets. Register it using the sollertia-shared-assets CLI:

slsa configure credentials -c google -f /path/to/downloaded_key.json

This copies the key into the platform credentials directory under its canonical name (google_credentials.json). Use slsa get credentials -c google to report the registered path.

Critical! If the key is not registered but a Google Sheet identifier is configured for the system, the preprocessing runtime aborts with a FileNotFoundError when it attempts to resolve the credentials. Register the key before running sessions that rely on the surgery or water logs.

Adding Google Sheets Access to the Service Account

To access the surgery log and the water restriction log Google Sheets as part of this library's runtime, create and share these log files with the email of the service account created above. The service account requires Editor access to both files.

Note, This feature requires that both log files are formatted according to the available Sollertia platform templates. Otherwise, the parsing algorithm does not behave as expected, leading to runtime failures. Additionally, both log files have to be pre-filled in advance, as the processing code is not allowed to automatically generate new table (log) rows. Note, Currently, it is advised to pre-fill the data a month in advance. Since most experiments last for at most a month, this usually covers the entire experiment period for any animal.

Note, for the sheet schema contract and for authoring a processor for a new sheet layout, use the experiment plugin's google-sheets-processing skill.

ScanImage PC Assets

As mentioned above, the ScanImagePC is largely assembled and configured by external contractors. However, the PC requires additional assets and configuration post-assembly to make it compatible with sollertia-experiment-managed runtimes.

File System Access

To support the sollertia-experiment runtime, the ScanImagePC's filesystem must be accessible to the VRPC via the Server Message Block version 3 (SMB3) or equivalent protocol. Since ScanImagePC uses Windows, it is advised to use the SMB3 protocol, as all Windows machines support it natively with minimal configuration. As a minimum, the ScanImagePC must be configured to share the root Mesoscope output directory with the VRPC over the local network. This is required to fetch the data acquired by the Mesoscope during preprocessing and to retrieve the desktop screenshot generated during session setup. The Mesoscope acquisition itself is controlled over MQTT, not through the shared directory (see MATLAB Assets).

MQTT Broker Access

The Mesoscope-VR system controls the Mesoscope by exchanging MQTT messages with the runAcquisition MATLAB function running on the ScanImagePC. The MQTT broker runs on the VRPC and is shared with the Unity Virtual Reality task. Because the ScanImagePC connects to this broker over the local network, the broker must be configured to accept non-loopback connections.

By default, Mosquitto 2.0 binds only to the loopback interface (127.0.0.1), so a separate ScanImagePC cannot reach it. To expose the broker on the local network, add the following to the VRPC's mosquitto.conf:

listener 1883
allow_anonymous true

The listener 1883 directive binds all network interfaces, including loopback, so the VRPC-side runtime and Unity continue to connect on 127.0.0.1 without changes. The allow_anonymous true directive is required because defining an explicit listener disables anonymous access by default, which both the sollertia-experiment runtime and the MATLAB client rely on.

Note, anonymous access is acceptable because the Sollertia platform operates on an isolated local network. Deployments on shared or untrusted networks should configure broker authentication instead.

After updating the configuration, restart Mosquitto and ensure the VRPC firewall allows inbound connections on the broker port (1883). Finally, pass the VRPC's network address to the MATLAB function when arming the Mesoscope, for example runAcquisition(hSI, hSICtl, broker="tcp://VRPC-IP:1883").

Default Screenshot Directory

During runtime, the sollertia-experiment library prompts the user to generate a screenshot of the ScanImagePC desktop and place it in the network-shared mesoscope data directory (see above). The screenshot is used to store the information about the red-dot alignment, the acquisition parameters, and the state of the imaging plane at the beginning of each session. The library is statically configured to fetch the screenshot from the shared directory and does not look in any other directories. Therefore, it is advised to reconfigure the default output directory used by the 'Win + PrtSc' command on the ScanImagePC to save the screenshots into the shared Mesoscope output directory.

MATLAB Assets

ScanImage software is written in MATLAB and controls all aspects of Mesoscope data acquisition. While each runtime requires the experimenter to manually interface with the ScanImage GUI during Mesoscope preparation, all data acquisition runtimes using the sollertia-experiment library require the user to call the runAcquisition MATLAB function on the ScanImagePC. This function carries out multiple runtime-critical tasks, including setting up the acquisition, generating and applying the online motion correction, and servicing the acquisition commands the VRPC issues over MQTT. The function connects to the same MQTT broker as the Unity Virtual Reality task, using a dedicated Mesoscope topic namespace that does not overlap with the Unity topics, and reports command reception and progress back to the VRPC.

The runAcquisition function ships with this library under assets/mesoscope_vr. See the Mesoscope-VR ScanImage PC assets guide for instructions on deploying the function to the ScanImagePC and registering it with MATLAB's search path. The function relies on the MariusMotionEstimator and MariusMotionCorrector2 online motion-correction classes, which are provided as part of the ScanImage installation on the ScanImagePC, and on the MATLAB Industrial Communication Toolbox, which provides the mqttclient interface.

Note, for the ScanImagePC and runAcquisition setup, use the mesoscope plugin's mesoscope-vr skill.

Mesoscope-VR Assembly

This section is currently a placeholder. Since the final Mesoscope-VR system design is still a work in progress, it will be populated once the final design implementation is constructed and tested in the lab.

The Mesoscope-VR assembly mostly consists of two types of components. First, it includes custom components manufactured via 3D-printing or machining (for metalwork). Second, it consists of generic components available from vendors such as ThorLabs, which are altered in workshops to fit the specific requirements of the Mesoscope-VR system. The blueprints and CAD files for all components of the Mesoscope-VR system, including CAD renders of the assembled system, are available in the CAD and blueprint folder.

Acquired Data Structure and Management

The library defines a fixed structure for storing all acquired data which uses a 4-level directory tree hierarchy: root (volume), project, animal, and session. This structure is reused by all acquisition systems, and it is maintained across all long-term storage destinations. After each data acquisition runtime (session), all raw data is stored under the root/project/animal/session/raw_data directory stored on one or more machines mentioned below.

Currently, each Sollertia platform data acquisition system uses a main data acquisition PC and any number of long-term storage destinations (zero, one, or more):

The main data acquisition PC is used to acquire and preprocess the data. For example, the VRPC of the Mesoscope-VR system is the main data acquisition PC for that system. This PC is used to both acquire the data and, critically, to preprocess the data before it is moved to the long-term storage destinations.
Each long-term storage destination is a machine or storage volume to which the preprocessed data is transferred for long-term storage. The Mesoscope-VR system anticipates two common destinations by default: a Server (a high-performance compute server used as the primary long-term storage and analysis destination) and a NAS (a Network-Attached-Storage volume used for redundant 'cold' backup storage, typically located in a different physical location to provide data storage redundancy). Neither is required: a system can be configured with any number of destinations under arbitrary names, or with none at all.

Critical! Each configured long-term storage destination is expected to be mounted to the main acquisition PC filesystem (for example, using the Server Message Block 3 (SMB3) protocol). Therefore, each data acquisition system operates on the assumption that all configured storage destination filesystems are used contiguously and can be freely accessed by the main acquisition PC's Operating System.

Note, the library transfers the preprocessed data to every configured long-term storage destination and purges the redundant local copy from the acquisition machine once the transfer succeeds. If no destinations are configured, the data is retained on the acquisition machine and the preprocessing is limited to the on-premises data conversion and aggregation steps.

Root Directory (Volume)

All data acquisition systems and all long-term storage destinations keep ALL Sollertia platform projects under the same root directory. The exact location and name of the root directory on each machine is arbitrary but should generally remain fixed (unchanging) over the entire lifetime of that specific machine. On the main acquisition machine, this local data root is configured with the slsa configure data-root command and resolved at runtime, so it is not part of the system configuration file. The root directories of any long-term storage destinations remain configured through the system configuration.

Project Directory

When a new project is created, a project directory named after the project is created under the root directory of the main data acquisition machine. When the data is moved to the configured long-term storage destinations as part of preprocessing, the project directory is also created on these destinations, if it does not already exist.

The slsa configure project command also creates a configuration subdirectory under the root project directory. This directory stores all supported experiment configurations for the project. For the Mesoscope-VR system, the sle mesoscope run command searches the configuration directory for the .yaml file with the name of the target experiment to load the experiment data.

Animal Directory

When the library is used to acquire data for a new animal, it generates a new animal directory under the root and project directory combination. The directory uses the ID of the animal as its name.

All data acquisition systems also use a persistent_data subdirectory under the root animal directory to store data that is reused between data acquisition sessions; for the Mesoscope-VR system, the directory is created by the system runtime as needed.

Critical! The current Sollertia platform convention stipulates that all animal IDs should be numeric. While some library components do accept strings as inputs, it is expected that all animal IDs only consist of positive integers. Failure to adhere to this naming convention can lead to runtime errors and unexpected behavior of all library components!

Session Directory

Each time the library is used to acquire data, a new session directory is created under the root, project and animal directory combination. The session name is derived from the current UTC timestamp at the time of the session directory creation, accurate to microseconds. Primarily, this naming format was chosen to make all sessions acquired by the same acquisition system have unique and chronologically sortable names. The session name format follows the order of YYYY-MM-DD-HH-MM-SS-US.

Raw Data

All data acquired by this library is stored under the raw_data subdirectory, generated for each session. Overall, an example path to the acquired (raw) data can therefore look like this: /media/Data/Experiments/Template/666/2025-11-11-05-03-23-234123/raw_data/.

Note, This library treats both newly acquired and preprocessed data as raw. This is because preprocessing does not change the content of the data. Instead, preprocessing uses lossless compression to more efficiently package the data for transmission and can at any time be converted back to the original format.

Shared Raw Data

The section below briefly lists the data acquired by all Sollertia platform data acquisition systems. Note, each acquisition system also generates system-specific data, which is listed under acquisition-system-specific sections available after this section.

Note, For information about the processed data, see the main data processing library.

After acquisition and preprocessing, the raw_data directory of each acquisition system contains, as a minimum, the following files and subdirectories:

ax_checksum.txt: Stores the xxHash3-128 checksum used to verify data integrity when it is transferred to the long-term storage destination. The checksum is generated before the data leaves the main data acquisition system PC and, therefore, accurately captures the final state of the raw data before it enters storage.
hardware_state.yaml: Stores the snapshot of the dynamically calculated parameters used by the data acquisition system modules during runtime. These parameters are recalculated at the beginning of each data acquisition session and are rounded and stored using the appropriate floating point type (usually fp64) to minimize floating point rounding errors. This file is also used to determine which modules were used during runtime and, consequently, which data can be parsed from the .npz log files generated at runtime (see below).
session_data.yaml: Stores information necessary to maintain the same session data structure across all machines used during data acquisition and long-term storage. This file is used by all Sollertia platform libraries as an entry point for working with a session's data. The file also includes all available information about the identity and purpose of the session and can be used by human experimenters to identify the session. The file also stores the versions of the sollertia-experiment library and Python that were used to acquire the data.
session_descriptor.yaml: Stores session-type-specific information, such as the task parameters and experimenter notes. The contents of the file are different for each session type, although some fields are reused by all sessions. The contents for this file are partially written by the library (automatically) and, partially, by the experimenter (manually, at the end of each session). At the end of each runtime, a copy of the descriptor file is cached inside the persistent_data directory of the animal under a session-type-specific name (e.g., lick_training_descriptor.yaml), replacing any already existing copy. This is used to optionally restore certain runtime configuration parameters between session types that support this functionality.
surgery_metadata.yaml: Stores the data on the surgical intervention(s) performed on the animal that participated in the session. This data is extracted from the surgery log Google Sheet, so the file is only generated when a surgery sheet identifier is configured for the system. For most animals, the contents should be the same across all sessions.
system_configuration.yaml: Stores the configuration parameters of the data acquisition system that generated the session data. This is a snapshot of all dynamically addressable configuration parameters used by the system. When combined with the assembly instructions and the appropriate sollertia-experiment library version, it allows completely replicating the data acquisition system used to acquire the session's data.
behavior_data: Stores compressed .npz log files that contain all non-video behavior data acquired by the system. This includes all messages sent or received by each microcontroller, the timestamps for the frames acquired by each camera, and (if applicable) the main brain activity recording device (e.g.: Mesoscope). These logs also include session metadata, such as trials, task conditions, and system and runtime state transitions. Although the exact content of the behavior data directory can differ between acquisition systems, all systems used in the lab generate some form of non-video behavior data.
camera_data: Stores the behavior videos recorded by video cameras used by the acquisition system. Videos are named using the session name and camera identifier (e.g., {session_name}_face_camera.mp4, {session_name}_body_camera.mp4).
experiment_configuration.yaml: This file is only created for experiment sessions. It stores the acquisition-side configuration of the experiment task performed by the animal during runtime. The contents of the file differ for each data acquisition system, but each system generates a version of this file. The file contains enough information to fully replicate the acquisition-side experiment runtime on the same acquisition system and to process and analyze the acquired data.
vr_configuration.yaml: This file is created for every Virtual Reality task session. It stores a snapshot of the Virtual Reality task template — the linear infinite corridor cues, VR environment, and trial structures — that the Unity game engine renders during the session, sourced from the sollertia-unity-tasks task template. Together with experiment_configuration.yaml, it contains the information necessary to fully replicate the Virtual Reality environment used during the experiment.

Shared Temporary Data

The session data hierarchy additionally uses the following temporary marker files and directories which are cleared before the raw data is transmitted to the long-term storage destinations:

nk.bin: This marker is automatically cached to disk by the shared session model as part of creating a new session data hierarchy. Each runtime removes this marker file when it successfully completes its runtime preparation. If this marker exists when the runtime enters the shutdown cycle, this indicates that the runtime encountered a fatal error during startup and had to be terminated early. In this case, the session's data is silently deleted, as uninitialized sessions necessarily do not contain any valid data.
behavior_data_log: All behavior log entries are initially saved as individual .npy files. Each .npy file stores a serialized log message in the form of a uint8 (byte) NumPy array. Since messages are cached to disk as soon as they are received by the DataLogger to minimize data loss in case of emergency shutdowns, the temporary behavior_data_log directory is used to store these messages during runtime. Frequently, the directory accumulates millions of .npy files at runtime, making it challenging for human operators to work with the data. During preprocessing, individual .npy files are grouped by their source (what made the log entry, e.g.: VideoSystem, MicroController, Data Acquisition System, etc.) and are compressed into .npz archives, one for each source. The .npz archives are then moved to the behavior_data directory, and the behavior_data_log with the individual .npy files is deleted to conserve disk space.

Mesoscope-VR System Data

The Mesoscope-VR system instantiates a directory hierarchy both on the VRPC and the ScanImagePC. Below is the list of files and directories found on each of these machines.

Raw Data

The Mesoscope-VR system generates the following files and directories, in addition to those discussed in the shared raw data section, on the VRPC:

mesoscope_data: Stores all Mesoscope-acquired data. Since Mesoscope data is only acquired for experiment and window checking sessions, this directory is not created for training session types. During preprocessing, the directory contents are organized to automatically work with the cindra processing library. The directory contains:
- mesoscope_XXXXXX_XXXXXX.tiff: Recompressed Mesoscope frame stacks using LERC lossless compression.
- frame_invariant_metadata.json: ScanImage metadata constant across all frames (frame rate, plane/channel count, and ROI information).
- frame_variant_metadata.npz: Per-frame metadata (frame numbers, timestamps, acquisition triggers).
- cindra_parameters.json: cindra-compatible configuration used for further data processing.
- MotionEstimator.me: Reference motion estimator file.
- fov.roi: Region-of-interest file defining the imaging field.
- zstack.tiff: A high-resolution z-stack reference image for future alignment with histology data.
zaber_positions.yaml: Stores the snapshot of the positions used by the HeadBar, LickPort, and Wheel Zaber motor groups, taken at the end of the session's data acquisition. All positions are stored in native motor units. This file is created for all session types supported by the Mesoscope-VR system. As a backup, a copy of this file is also generated at the beginning of each session's runtime. This allows recovering from critical runtime failures, where the runtime may not be able to generate this snapshot. During both snapshot generation timepoints, a copy of the generated snapshot file is also cached inside the persistent_data directory of the animal to support restoring the motors to the same position during the next session.
mesoscope_positions.yaml: Stores the snapshot of the Mesoscope objective position in the physical axes (X, Y, Z, and Roll), the virtual ScanImage axes (Fast Z, Tip, and Tilt), and the laser power at the sample, taken at the end of the session's data acquisition. Note, This file relies on the experimenter updating the stored positions. It is only created for window checking and experiment sessions. A copy of this snapshot file is also saved to the persistent_data directory of the animal to support restoring the Mesoscope to the same imaging field during the next session.
window_screenshot.png: Stores the screenshot of the ScanImagePC screen. The screenshot should contain the image of the red-dot alignment, the view of the target cell layer, the Mesoscope position information, and the data acquisition parameters. Primarily, the screenshot is used by experimenters to quickly reference the imaging quality from each experiment session. This file is only created for window checking and experiment sessions. A copy of this file is saved to the persistent_data directory of the animal to help the user realign the red-dot to a similar position during the next session.

ScanImagePC

All Mesoscope-VR system data on the ScanImagePC is stored under the user-defined ScanImagePC root directory, which is expected to be mounted to the VRPC via the SMB3 or similar protocol. Under that root directory, the system creates the following directories and files:

mesoscope_data: This directory stores all Mesoscope-acquired data for the currently running session. The runAcquisition MATLAB function configures the ScanImage software to output all data to the mesoscope_data directory, which is shared by all sessions, animals, and projects. This allows using the same static output path for all ScanImage acquisitions.
session-specific mesoscope_data: At the end of each runtime, the Mesoscope-VR system renames the mesoscope_data directory to instead use the session name (id). Then, it generates an empty mesoscope_data directory for the next runtime. This way, all data of each completed session is stored under a unique directory named after that session. This step is crucial for data preprocessing, which identifies the session data directory and pulls it over to the VRPC based on the session name (id).
persistent_data: This directory is created for each unique project and animal combination, similar to the data structure created by sollertia-experiment on the main acquisition system PC. This directory contains the first experiment day's MotionEstimator.me and fov.roi files. These files are typically reused by all following data acquisition sessions to restore the imaging field to the same location as used on the first day. The full path to the persistent_data directory would typically look like root/project/animal/persistent_data.

Mesoscope-VR Temporary Data

The Mesoscope-VR system also generates the following temporary directory during runtime:

raw_mesoscope_frames: Stores uncompressed .TIFF stacks fetched by the VRPC from the ScanImagePC. This is done as part of data preprocessing to collect all data on the VRPC before executing individual preprocessing subroutines. The .TIFF stacks are then re-compressed using the Limited Error Raster Compression (LERC) scheme and are saved to the mesoscope_data directory. Once this process completes successfully, the raw_mesoscope_frames directory with all raw files is deleted to conserve disk space.

Note, the VRPC controls the Mesoscope acquisition over MQTT rather than through marker files, so no acquisition marker files are created during runtime. See the ScanImage PC Assets section for details on the MQTT command interface.

Acquiring Data

All user-facing library functionality is realized through a set of Command Line Interface (CLI) commands automatically exposed when the library is pip-installed into a python environment. The library exposes a single top-level sle command with two subcommand groups — a general, hardware-agnostic discovery group (sle get) shared by all acquisition systems, and the Mesoscope-VR system group (sle mesoscope) that combines configuration, maintenance, data acquisition, and data management for the Mesoscope-VR system — plus a standalone sle mcp command that starts the MCP server (see AI-Assisted Use). Each group contains subcommands that allow further configuring their runtime. Use the --help argument when calling any of the commands described below to see the list of supported arguments together with their descriptions and default values.

To use any of the commands described below, activate the python environment where the library is installed, e.g., with conda activate MYENV and type one of the commands described below.

Warning! Most commands described below use the terminal to communicate important runtime information to the user or request user feedback. Carefully read every message printed to the terminal during runtime. Failure to do so may damage the equipment or harm the animal!

CLI Command Overview

Command	Description
`sle get`	Discover and evaluate data acquisition system components (general)
`sle mesoscope configure system`	Generate the Mesoscope-VR data acquisition system configuration file
`sle mesoscope configure experiment`	Create a Mesoscope-VR experiment configuration from a task template
`sle mesoscope maintain`	Run the Mesoscope-VR system maintenance session
`sle mesoscope check-bridge`	Check whether the ScanImagePC runAcquisition loop is reachable
`sle mesoscope run`	Execute Mesoscope-VR data acquisition and training sessions
`sle mesoscope preprocess`	Preprocess a session's data and push it to long-term storage
`sle mesoscope delete`	Remove a session's data from all storage destinations
`sle mesoscope migrate`	Transfer an animal's sessions between projects
`sle mcp`	Start the MCP server backing the `sle get` and `sle mesoscope` tools

Step 0: Configuring the Data Acquisition System

Before acquiring data, each acquisition system has to be configured. This step is done in addition to assembling the system and installing the required hardware components. Typically, this only needs to be done when the acquisition system configuration or hardware changes, so most lab members can safely skip this step.

Use the sle mesoscope configure system command to generate the system configuration file. As part of its runtime, the command configures the host machine to remember the path to the generated configuration file, so all future sollertia-experiment runtimes on that machine automatically load and use the appropriate acquisition-system configuration parameters.

Note, the local data root — the directory under which all projects, animals, and sessions are stored on the main acquisition machine — is configured separately from the system configuration file, as it is a Sollertia platform-level setting shared across libraries. Set it on the main acquisition machine with the slsa configure data-root command (from the sollertia-shared-assets library) and view it with the slsa get data-root command. The data root must be configured before running data acquisition sessions. The root directories of any long-term storage destinations are not set this way; they remain part of the system configuration.

Note, Each acquisition system uses unique configuration parameters. Additionally, the sollertia-experiment library always assumes that any machine (PC) can only be used by a single data-acquisition system (is permanently a part of that acquisition system). Only the main PC of the data acquisition system (e.g.: the VRPC of the Mesoscope-VR system) that runs the sollertia-experiment library should be configured via this command.

The system configuration parameters are defined and owned by this library (sollertia-experiment): the SystemConfiguration base class lives in cross_system/system_configuration.py, and the Mesoscope-VR system's concrete MesoscopeSystemConfiguration lives in mesoscope_vr/system.py. For information about the available system configuration parameters, read the API documentation of this library and the mesoscope plugin's mesoscope-vr skill. Use the experiment plugin's system-health-check skill to validate the configuration before running sessions.

Step 1: Creating a Project

All data acquisition sessions require a valid project to run. To create a new project, use the slsa configure project command (from the sollertia-shared-assets library). This command can only be called on the main PC of a properly configured data-acquisition system (see Step 0 above). As part of its runtime, this command generates the root project directory on all machines that make up the data acquisition system.

Step 2: Creating an Experiment

All projects that involve scientific experiments also need to define at least one experiment configuration. Experiment configurations are unique for each data acquisition system and are stored inside .yaml files named after the experiment. To generate a new experiment configuration file, use the sle mesoscope configure experiment command. This command generates a precursor experiment configuration file inside the configuration subdirectory, stored under the root project directory on the main PC of the data acquisition system.

For information about the available experiment configuration parameters in the precursor file, read the API documentation of the appropriate data-acquisition system available from the sollertia-shared-assets library.

Mesoscope-VR Note: Mesoscope-VR experiments also require a valid Virtual Reality environment generated through the sollertia-unity-tasks package. The experiment configuration supplies the acquisition-side, per-trial parameters, while the Virtual Reality task template defines the corridor cues, VR environment, and spatial trial structures that Unity renders (persisted with the session as vr_configuration.yaml); the two are joined by trial name at runtime. After creating the experiment configuration, use the Unity package to generate the corresponding VR scene before running experiment sessions.

Step 3: Discovering System Components

Before running sessions, use sle get subcommands to discover and verify the components accessible to the data acquisition system:

Command	Description
`sle get zaber`	Discovers Zaber motor devices connected to the system
`sle get cameras`	Discovers cameras accessible to the system (OpenCV and Harvesters interfaces)
`sle get controllers`	Discovers microcontrollers accessible to the system
`sle get ports`	Lists available serial communication ports
`sle get unity`	Checks whether the Unity Editor MCP Bridge is reachable
`sle get checksum -i STRING`	Calculates the CRC32-XFER checksum for the input string

Note, project and experiment discovery is provided by the sollertia-shared-assets library. Use slsa get projects and slsa get experiments -p PROJECT to list the projects and experiment configurations stored under the data root.

Note, the Mesoscope imaging control bridge has its own pre-flight check outside the sle get group. Use sle mesoscope check-bridge to confirm the ScanImagePC's runAcquisition control loop is reachable before starting a Mesoscope imaging session.

Note, for an orchestrated pre-flight verification of configuration, storage mounts, and hardware connectivity, use the experiment plugin's system-health-check and acquisition-system-setup skills.

Step 4: Maintaining the Acquisition System

All acquisition systems contain modules that require frequent maintenance. Most of these modules are unique to each acquisition system. Therefore, this section is further broken into acquisition-system-specific subsections.

Mesoscope-VR

The Mesoscope-VR system contains two modules that require frequent maintenance: the water delivery system and the running wheel. To facilitate the maintenance of these modules, the sollertia-experiment library exposes the sle mesoscope maintain command.

This command is typically called at least twice during each day the system is used to acquire data. First, it is used at the beginning of the day to prepare the Mesoscope-VR system for runtime by filling the water delivery system. Second, it is used at the end of each day to empty the water delivery system.

Less frequently, this command is used to re-calibrate the water delivery system, typically as a result of replacing system components, such as tubing or the valve itself. The command is also occasionally used to replace the surface material of the running wheel, which slowly deteriorates as the wheel is used.

This command can also facilitate cleaning the wheel, which is typically done before and after each runtime to remove any biological contaminants left by each animal participating in experiment or training runtimes.

Note, This runtime uses a GUI-based interface for issuing commands and monitoring the system state.

Step 5: Acquiring Data

Each acquisition system supports one or more distinct types of data-acquisition sessions (runtimes). The set of supported session types is unique to each system; commonly this includes an 'experiment' session type, which is the primary use case for most acquisition systems in the lab. Some systems may also support one or more training session types, which often do not acquire any brain activity data, but otherwise behave similarly to experiment sessions.

All session commands require common parameters: user ID, project name, animal ID, and animal weight. These are provided to the parent sle mesoscope run command before specifying the session type.

Mesoscope-VR Session Commands

The Mesoscope-VR system supports four types of runtime sessions:

1. Window Checking Session

sle mesoscope run -u USER -p PROJECT -a ANIMAL -w WEIGHT window-checking

This session guides the user through finding the imaging plane and generating the reference MotionEstimator.me and zstack.tiff files for the checked animal. This session is typically used ~2–3 weeks after the surgical intervention and before any training or experiment sessions to assess the quality of the intervention and the suitability of including the animal in experiment cohorts.

2. Lick Training Session

sle mesoscope run -u USER -p PROJECT -a ANIMAL -w WEIGHT lick-training [OPTIONS]

All animals that participate in Mesoscope-VR experiments undergo a two-stage training protocol, with lick training being the first stage. During this runtime, the animals are head fixed in the Mesoscope enclosure for ~20 minutes. The primary goal of the runtime is to teach the animals to lick at the water tube to consume water rewards and associate the sound tone emitted at reward delivery with water coming out of the tube. During runtime, the running wheel is locked, so the animals cannot run.

Optional arguments include maximum training time, reward delay parameters, maximum water volume, and unconsumed reward tracking threshold.

3. Run Training Session

sle mesoscope run -u USER -p PROJECT -a ANIMAL -w WEIGHT run-training [OPTIONS]

This is the second stage of the mandatory two-stage Mesoscope-VR training protocol. During this runtime, the animals are head fixed in the Mesoscope enclosure for ~40 minutes. The primary goal of the runtime is to teach the animals to run on the wheel while head-fixed and to associate running with receiving water rewards. During runtime, the running wheel is unlocked, but the Virtual Reality screens are kept off, ensuring that the animal is not exposed to any visual cues until the first experiment day.

Optional arguments include maximum training time, speed and duration thresholds, threshold increase parameters, maximum water volume, idle time allowance, and unconsumed reward tracking threshold.

4. Experiment Session

sle mesoscope run -u USER -p PROJECT -a ANIMAL -w WEIGHT experiment -e EXPERIMENT [OPTIONS]

This session type is designed to execute the experiment specified in the target experiment_configuration.yaml file (see above). The system supports varying experiment configurations and Virtual Reality environments, offering experimenters the flexibility to run different projects using the same set of APIs and system hardware.

The -e/--experiment argument specifies the name of the experiment configuration to use.

Note, for the runtime state machine, the session types, and the behavior visualizer / control UI behind these commands, use the mesoscope plugin's mesoscope-vr-runtime skill.

Step 6: Preprocessing and Managing Data

All acquisition systems support several ways of handling the session's data acquired at runtime. For most runtimes, the choice of how to handle the data is made as part of the acquisition system shutdown sequence. However, in the case of unexpected runtime terminations, all data preprocessing steps can also be executed manually by calling the appropriate CLI command.

Note, the preprocess, delete, and migrate operations below are also exposed as sle mcp tools; the experiment plugin's data-management skill drives them (including bulk preprocessing) for AI-assisted use.

Preprocessing Data

The most commonly used operation is to preprocess the acquired data. This can be done manually by calling:

sle mesoscope preprocess -sp SESSION_PATH

Preprocessing consists of two major steps. The first step pulls all available data to the main data acquisition system machine (PC) and re-packages (re-compresses) the data to reduce its size without loss. The second step distributes (pushes) the data to all configured long-term storage destinations, such as a Server and a NAS.

Critical! It is imperative that all valid data acquired in the lab undergoes preprocessing as soon as possible. Only preprocessed data is stored in a way that maximizes its safety by using both redundancy and parity. Data that is not preprocessed may be lost in the case of emergency, which is considerably less likely for the preprocessed data.

When preprocessed data is successfully and safely pushed to long-term storage destinations, the preprocessing runtime removes the local copy of the session's data from the data acquisition system machine to conserve disk space.

Deleting Session Data

The second way of managing the data is primarily used during testing and when handling interrupted sessions that did not generate any valid data. This involves removing all session data from both the data acquisition system and all long-term storage destinations. This runtime is extremely dangerous and, if not used carefully, can permanently delete valid data. This mode can be triggered using:

sle mesoscope delete -sp SESSION_PATH

Warning! This command is not recommended for most users.

Migrating Animal Data Between Projects

To transfer all sessions for an animal from one project to another, use:

sle mesoscope migrate -s SOURCE_PROJECT -d DESTINATION_PROJECT -a ANIMAL_ID

This moves the animal's data across all accessible storage destinations: the sessions are relocated to the destination project, not copied.

Recovering from Interruptions

While it is not typical for the data acquisition or preprocessing pipelines to fail during runtime, it is possible. The library can recover or gracefully terminate the runtime for most code-generated errors, so this is usually not a concern. However, if a major interruption (i.e., power outage) occurs or one of the hardware assets malfunctions during runtime, manual intervention is typically required to recover the session's data and reset the acquisition system.

Data acquisition interruption

Data acquisition can be interrupted in two main ways, the first being due to an external asset failure, for example, if the ScanImagePC unexpectedly shuts down during Mesoscope-VR system runtime. In this case, the runtime pauses and instructs the user to troubleshoot the issue and then resume the runtime. This type of soft interruption is handled gracefully during runtime to exclude the data collected during the interruption from the output dataset. Generally, soft interruptions are supported for most external assets, which includes anything not managed directly by the sollertia-experiment library and the main data acquisition system PC. While inconvenient, these interruptions do not typically require specialized handling other than recovering and restoring the failed asset.

Note, While most soft interruptions typically entail resuming the interrupted runtime, it is also possible to instead terminate the runtime. To do so, press the Terminate Runtime button in the runtime GUI instead of trying to resume the runtime. In this case, the system attempts to execute a graceful shutdown procedure, saving all valid data in the process.

The second way involves interruption due to sollertia-experiment runtime failures or unexpected shut-downs of the main acquisition system PC. In these cases, manual user intervention is typically required to recover the useful data and reset the system before the acquisition can be restarted. The handling of such cases often consists of specific steps for each supported acquisition system. Typically, these hard interruptions are related to major issues, such as global facility power loss or severe malfunction of sensitive acquisition system components, such as microcontrollers and communication cables.

Mesoscope-VR

If the VRPC runtime unexpectedly interrupts at any point without executing the graceful shutdown, follow these instructions:

If the session involved Mesoscope imaging, shut down the Mesoscope acquisition process and make sure all required files (frame stacks, motion estimator data, and cranial window screenshot) have been generated and saved to the mesoscope_data directory.
If necessary, manually edit the session_descriptor.yaml, the mesoscope_positions.yaml, and the zaber_positions.yaml files to include actual runtime information. Estimate the volume of water delivered at runtime by manually reading the water tank level gauge.
Remove the animal from the Mesoscope enclosure. If necessary, use the Zaber Launcher app to directly interface with Zaber motors and move them in a way that allows the animal to be recovered from the enclosure.
Use Zaber Launcher to manually move the HeadBar group's Roll axis to have a positive angle (> 0 degrees). This is critical! If this is not done, the motor cannot home during the next session and instead collides with the movement guard, at best damaging the motor and, at worst, the Mesoscope.
If the session involved Mesoscope imaging, rename the mesoscope_data directory to use the session name. For example, from mesoscope_data → 2025-11-11-05-03-23-234123. Critical! if this is not done, the library may delete any leftover Mesoscope files during the next runtime and cannot properly preprocess the frames for the interrupted session during the next step.
Call sle mesoscope preprocess -sp SESSION_PATH and provide the path to the session directory of the interrupted session. This preprocesses and transfers all collected data to the long-term storage destinations. This preserves any data acquired before the interruption and prepares the system for running the next session.

Note, for inspecting or patching the per-session Zaber and Mesoscope position snapshots edited in step 2, use the mesoscope plugin's mesoscope-vr-snapshots skill (it owns the snapshot write tools). For the runtime recovery behavior, use the mesoscope-vr-runtime skill, and for safe manual motor positioning, the experiment plugin's zaber-interface skill.

Data preprocessing interruption

To recover from an error encountered during preprocessing, call sle mesoscope preprocess -sp SESSION_PATH and provide the path to the session directory of the interrupted session. The preprocessing pipeline automatically resumes an interrupted runtime from the nearest checkpoint.

AI-Assisted Use (MCP Server and Skills)

The library is built for AI-assisted operation. It ships a single MCP server, exposed through the sle mcp command, and a set of Claude Code skills distributed through the sollertia marketplace. Human operators interact with the library through the sle CLI documented above; AI agents work through the MCP server tools and the skills described here.

MCP Server

This library provides a single MCP server that exposes the tools backing the sle get and sle mesoscope CLI layers for AI agent integration. The server intentionally does not re-expose assets owned by the sollertia-shared-assets, ataraxis-video-system, and ataraxis-communication-interface dependencies; those are available through the dependencies' own MCP servers.

Starting the Server

Start the MCP server using the CLI:

sle mcp

The -t/--transport option selects the transport (stdio by default; also sse and streamable-http).

Available Tools

The general, hardware-agnostic tools mirror the sle get CLI layer:

Tool	Description
`get_zaber_devices_tool`	Discovers Zaber motor devices connected to the system
`get_checksum_tool`	Calculates the CRC32-XFER checksum for an input string
`get_zaber_device_settings_tool`	Reads configuration from a Zaber device's non-volatile memory
`set_zaber_device_setting_tool`	Writes configuration to a Zaber device's non-volatile memory
`validate_zaber_configuration_tool`	Validates a Zaber device's configuration for binding library use
`check_mount_accessibility_tool`	Verifies a filesystem path exists and is writable
`check_unity_bridge_tool`	Checks whether the Unity Editor MCP Bridge is reachable

The Mesoscope-VR-specific tools mirror the sle mesoscope CLI layer:

Tool	Description
`read_system_configuration_tool`	Loads the Mesoscope-VR system configuration
`write_system_configuration_tool`	Creates or replaces the Mesoscope-VR system configuration
`validate_system_configuration_tool`	Validates the system configuration and reports mount status
`verify_camera_configuration_tool`	Compares each camera's live GenICam config to its .yaml
`describe_system_configuration_schema_tool`	Returns the recursive Mesoscope-VR system configuration schema
`check_system_mounts_tool`	Verifies all configured filesystem paths are accessible
`check_mesoscope_bridge_tool`	Checks whether the ScanImagePC runAcquisition loop is reachable
`read_session_zaber_positions_tool`	Loads a session's ZaberPositions snapshot
`write_session_zaber_positions_tool`	Creates or replaces a session's ZaberPositions snapshot
`read_session_mesoscope_positions_tool`	Loads a session's MesoscopePositions snapshot
`write_session_mesoscope_positions_tool`	Creates or replaces a session's MesoscopePositions snapshot
`read_session_system_configuration_tool`	Loads a session's system configuration snapshot
`preprocess_session_tool`	Preprocesses a session's data on the host machine
`delete_session_tool`	Removes a session from all storage locations
`migrate_animal_tool`	Transfers all sessions for an animal between projects

Claude Desktop Configuration

Add the following to the Claude Desktop configuration file:

{
  "mcpServers": {
    "sollertia-experiment": {
      "command": "sle",
      "args": ["mcp"]
    }
  }
}

Marketplace Plugins

The sollertia marketplace ships five plugins, each backed by the MCP server of the library it targets. Installing a plugin registers its MCP server with compatible clients and makes its skills available:

Plugin	Targets	MCP server	Role
`assets`	sollertia-shared-assets	`slsa mcp`	Configuration authoring and shared session/subject/template asset I/O
`unity`	sollertia-unity-tasks	(relay)	Unity task authoring, VR scenes, and the MQTT contract; uses the slsa relay
`experiment`	sollertia-experiment	`sle mcp`	System-agnostic core: design, runtime, hardware interfaces, data management
`mesoscope`	sollertia-experiment	`sle mcp`	Mesoscope-VR system-specific skills (layered on the core plugins)
`forging`	sollertia-forgery	`sl-mcp`	Downstream behavior processing and analysis

The unity plugin uses the Unity relay served by the assets plugin's slsa MCP server and the McpBridge editor plugin; it requires the assets plugin. The mesoscope plugin requires both the experiment plugin's sle mcp server and the assets plugin's slsa mcp server. Low-level hardware work additionally draws on the ataraxis marketplace's video and communication plugins and their MCP servers; the sle mcp server intentionally omits the assets, video, and communication tools, which are served by those dependencies' own MCP servers.

Skills

The experiment plugin ships the system-agnostic core skills, and the mesoscope plugin ships the Mesoscope-VR system-specific skills. With the exception of system-health-check, which is user-invocable as a slash command, the skills are orchestrated by AI agents rather than invoked directly by operators.

Plugin	Skill	Purpose
`experiment`	`pipeline`	Orchestrate the end-to-end experiment lifecycle of an existing system
`experiment`	`system-design-pipeline`	Orchestrate designing and building a new acquisition system
`experiment`	`acquisition-system-design`	Design pattern: system configuration, binding classes, runtime orchestrator
`experiment`	`acquisition-system-runtime`	Runtime pattern: per-mode logic, state machine, dispatch, control UI
`experiment`	`acquisition-system-setup`	Discover and verify connected acquisition hardware
`experiment`	`system-health-check`	Pre-flight checks of configuration, mounts, and hardware (user-invocable)
`experiment`	`zaber-interface`	Implement Zaber motor interfaces and binding classes
`experiment`	`microcontroller-interface`	Paired Module + ModuleInterface registry and conventions
`experiment`	`vr-driver-interface`	VR task driver, Unity MQTT contract, and cue-sequence trial decomposition
`experiment`	`google-sheets-processing`	Implement Google Sheets SurgeryLog / WaterLog processors
`experiment`	`data-management`	Preprocess, migrate, and delete session data via `sle mcp`
`experiment`	`experiment-mcp-environment-setup`	Diagnose `sle mcp` server connectivity issues
`mesoscope`	`mesoscope-vr`	Mesoscope-VR hardware inventory, configuration, and bindings
`mesoscope`	`mesoscope-vr-runtime`	Mesoscope-VR state machine, orchestrator, UIs, and `sle mesoscope` CLI
`mesoscope`	`mesoscope-vr-snapshots`	Read/write per-session Zaber and Mesoscope position snapshots
`mesoscope`	`mesoscope-vr-session-schema`	Mesoscope-VR session descriptor and hardware-state field schema
`mesoscope`	`mesoscope-vr-experiment-schema`	Mesoscope-VR experiment configuration and trial-class field schema

Client Registration

MCP server registration and Claude Code skill assets for this library are distributed through the sollertia marketplace as part of the experiment plugin. Install the plugin from the marketplace to automatically register the MCP server with compatible clients and make all associated skills available.

API Documentation

See the API documentation for the detailed description of the library's internal modules and the CLI commands with their arguments. The top-level sollertia_experiment package does not export any Python-level API symbols; all user-facing functionality is realized through the sle command line interface.

Developers

This section provides installation, dependency, and build-system instructions for the developers that want to modify the source code of this library.

Installing the Project

Note, this installation method requires mamba version 2.3.2 or above. Currently, all automation pipelines require that mamba is installed through the miniforge3 installer.

Download this repository to the local machine using the preferred method, such as git-cloning.
If the downloaded distribution is stored as a compressed archive, unpack it using the appropriate decompression tool.
cd to the root directory of the prepared project distribution.
Install the core development dependencies into the base mamba environment via the mamba install tox uv tox-uv command.
Use the tox -e create command to create the project-specific development environment followed by the tox -e install command to install the project into that environment as a library.

Additional Dependencies

In addition to installing the project and all user dependencies, install the following dependencies:

Python version 3.14. This library currently supports a single Python version (requires-python >=3.14,<3.15). It is recommended to use a tool like pyenv to install and manage the required version.

Development Automation

This project uses tox for development automation. The following tox environments are available:

Environment	Description
`lint`	Runs ruff formatting, ruff linting, and mypy type checking
`stubs`	Generates the py.typed marker and .pyi stub files using the project's wheel distribution
`docs`	Builds the API documentation via Sphinx
`build`	Builds sdist and wheel distributions
`upload`	Uploads distributions to PyPI via twine
`install`	Builds and installs the project into its `sle_dev` mamba environment
`uninstall`	Uninstalls the project from its `sle_dev` mamba environment
`create`	Creates the project's `sle_dev` mamba development environment (Python 3.14)
`remove`	Removes the project's `sle_dev` mamba development environment
`provision`	Removes and recreates the `sle_dev` mamba environment
`export`	Exports the mamba environment as a .yml file and a spec.txt with revision history
`import`	Creates or updates the mamba environment from the stored .yml file

Run any environment using tox -e ENVIRONMENT. For example, tox -e lint.

Note, all pull requests for this project have to successfully complete the tox task before being merged. To expedite the task's runtime, use the tox --parallel command to run some tasks in parallel.

AI-Assisted Development

Claude Code skills and other AI development assets for Sollertia platform repositories are distributed through the ataraxis marketplace as part of the automation plugin. Install the plugin from the marketplace to make all associated development skills and tools available to compatible AI coding agents.

Automation Troubleshooting

Many packages used in tox automation pipelines (uv, mypy, ruff) and tox itself may experience runtime failures. In most cases, this is related to their caching behavior. If an unintelligible error is encountered with any of the automation components, deleting the corresponding cache directories (.tox, .ruff_cache, .mypy_cache, etc.) manually or via a CLI command typically resolves the issue.

Versioning

This project uses semantic versioning. See the tags on this repository for the available project releases.

Authors

Ivan Kondratyev (Inkaros)
Kushaan Gupta (kushaangupta)
Natalie Yeung
Katlynn Ryu (katlynn-ryu)
Jasmine Si

License

This project is licensed under the Apache 2.0 License: see the LICENSE file for details.

Acknowledgments

All Sun (NeuroAI) lab members for providing the inspiration and comments during the development of this library.
The creators of all other dependencies and projects listed in the pyproject.toml file.

Name		Name	Last commit message	Last commit date
Latest commit History 688 Commits
assets/mesoscope_vr		assets/mesoscope_vr
docs		docs
envs		envs
src/sollertia_experiment		src/sollertia_experiment
.gitattributes		.gitattributes
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
tox.ini		tox.ini

Folders and files

Latest commit

History

Repository files navigation

sollertia-experiment

Detailed Description

Features

Table of Contents

Dependencies

Installation

Source

pip

Data Acquisition Systems

Extending the Platform

Mesoscope-VR Data Acquisition System

Main Dependency

Software Dependencies

Hardware Dependencies

System Assembly

Zaber Motors

Behavior Cameras

MicroControllers

Virtual Reality Task Environment (Unity)

Coordination via MQTT

Google Sheets API Integration

Setting up Google Sheets API Access

Registering the Service Account Key

Adding Google Sheets Access to the Service Account

ScanImage PC Assets

File System Access

MQTT Broker Access

Default Screenshot Directory

MATLAB Assets

Mesoscope-VR Assembly

Acquired Data Structure and Management

Root Directory (Volume)

Project Directory

Animal Directory

Session Directory

Raw Data

Shared Raw Data

Shared Temporary Data

Mesoscope-VR System Data

Raw Data

ScanImagePC

Mesoscope-VR Temporary Data

Acquiring Data

CLI Command Overview

Step 0: Configuring the Data Acquisition System

Step 1: Creating a Project

Step 2: Creating an Experiment

Step 3: Discovering System Components

Step 4: Maintaining the Acquisition System

Mesoscope-VR

Step 5: Acquiring Data

Mesoscope-VR Session Commands

Step 6: Preprocessing and Managing Data

Preprocessing Data

Deleting Session Data

Migrating Animal Data Between Projects

Recovering from Interruptions

Data acquisition interruption

Mesoscope-VR

Data preprocessing interruption

AI-Assisted Use (MCP Server and Skills)

MCP Server

Starting the Server

Available Tools

Claude Desktop Configuration

Marketplace Plugins

Skills

Client Registration

API Documentation

Developers

Installing the Project

Additional Dependencies

Development Automation

AI-Assisted Development

Automation Troubleshooting

Versioning

Packages