ROS1 -> ROS2 (mcap) Bag Conversion

Quick Start (Demo)

We provide a demo script that automatically downloads test bagfiles and runs the conversion logic for you. The script also opens the converted file in Foxglove if it is installed.

# Run the demo (downloads data -> converts -> verifies)
./run_demo.sh

Prefer to test manually? You can download the sample split-bag dataset from Google Drive here.

---

TL;DR

bash ./install.sh

# Basic conversion
convert_bag /path/to/single_ros1.bag

# Series conversion (auto-split detection)
convert_bag /path/to/ros1_bag_folder --series
convert_bag /path/to/split_0.bag /path/to/split_1.bag --series

This repository provides a containerized solution for two primary tasks:

1. ROSbags Conversion: A utility to convert ROS 1 (.bag) files to ROS 2 (.mcap) format. Features include:

• Plugin System: Extendable architecture to modify data on the fly, inject timestamps, or split topics using Python plugins.
• Automatic Type Repair: Migrates custom messages without source code. Automatically fixes ROS1 vs ROS2 incompatibilities.
• Smart Schema Registry: Handles both standard ROS definitions and custom message definitions generated by plugins.
• Split-Bag Handling: Automatically detects split bags and injects /tf_static into every chunk. Every output file is self-contained.
• Crash-Safe: Implements graceful signal handling. Hit Ctrl+C anytime, and your MCAP file will still be valid and playable.
• Zero Dependencies: Dockerized solution. No need to install ROS 1 or build custom message packages locally.

2. ROS Noetic Development: A persistent environment for developing, building, and executing ROS 1 (Noetic) packages.

Prerequisites

• Docker: Ensure Docker and Docker Compose are installed.
• Permissions: The current user must have permission to run Docker commands (e.g., be in the docker group).

---

Part 1: Bag Converter

The converter runs as an ephemeral container. It mounts the target data directory, processes the files, and terminates.

Installation

Run the provided installation script to create a symbolic link for the execution wrapper and pull the docker image.

./install.sh

Note: Ensure ~/.local/bin is in your system $PATH.

Usage

The convert_bag command can be executed from any location.

1. Single File Conversion

Converts a single .bag file. The output .mcap file is created in the same directory as the input.

convert_bag /path/to/recording.bag

2. Series Conversion (Split Bags)

Use the --series flag when processing split bag files (e.g., _0.bag, _1.bag). This mode enables Static TF Injection (ensuring all parts have TF data) and generates metadata.yaml for seamless playback.

# Option A: Process all bags in a folder
convert_bag /path/to/data_folder --series

# Option B: Process specific list of files
convert_bag /path/to/data/bag_0.bag /path/to/data/bag_1.bag --series

---

Plugin System

The converter features a modular plugin architecture allowing users to manipulate messages during the conversion process (e.g., parsing raw strings, debayering images, or anonymizing data).

Architecture

The system uses a hook-based architecture:

1. Loader: plugin_manager.py scans the src/plugins/ directory.
2. Configuration: It reads src/plugins.yaml to configure active plugins and pass parameters (e.g., topic names, quality settings).
3. Execution: Plugins are initialized with their config. They process messages and can return multiple new messages (1-to-N expansion) or modify messages in place.

Configuration

Plugins are managed via src/plugins.yaml. You can pass arbitrary parameters to plugins here.

plugins:
  # Example: Image processing
  DebayerPlugin:
    target_topic: "camera_front"
    jpeg_quality: 95

  # Example: Custom parsing logic
  RobustelFixPlugin:
    input_filter: "robustel"
    output_topic: "/robustel/parsed"

How to Develop a Plugin (Click to Expand)

You can add custom logic by adding a Python script to the src/plugins/ directory.

1. Reference Example See src/plugins/debayer.py or src/plugins/robustel_fix.py for working examples.

2. Plugin Signature All plugins must inherit from BasePlugin. The process method must return a list of emissions and a boolean modification flag.

from typing import Any, List, Tuple, Optional
from plugin_manager import BasePlugin

class MyPlugin(BasePlugin):
    def process(self, topic: str, msg: Any, msg_type: str, timestamp: int) -> Tuple[List[Tuple[str, Any, str, Optional[str]]], bool]:
        """
        Args:
            timestamp: Nanoseconds from the bag file.
            
        Returns:
            Tuple(Emissions_List, Modified_Flag)
            
            Emissions_List format: 
            [ (Topic, Message, Type_String, Definition_Override) ]
        """
        
        # Access config from YAML
        my_param = self.config.get('my_param', 'default')

        # Example: Pass through unchanged
        return [(topic, msg, msg_type, None)], False

3. Hot-Reloading Because the src folder is mounted into the container, you do not need to rebuild the Docker image to test new plugins. Simply edit the Python file and run convert_bag.

---

Advanced Configuration

The script accepts optional arguments passed directly to the internal Python converter:

• --out-dir <path>: Forces a specific output directory.
• --with-plugins: Enables the plugin system (reads src/plugins.yaml).
• --dry-run: Validates input files, write permissions, and configuration without processing data.

Feature: Auto-Generating ROS 2 Message Definitions (Click to Expand)

The conversion tool embeds message definitions directly into the .mcap file. Modern visualization tools like Foxglove Studio read these embedded schemas automatically.

However, if you want to replay the bag using CLI tools (ros2 bag play) or inspect topics (ros2 topic echo), your local ROS 2 environment needs the compiled message packages installed.

Usage:

1. Run the extractor (in your ROS 2 environment):

python3 additional/extract_mcap_msgs.py /path/to/my_data.mcap --out-dir src/

2. Build the packages:

colcon build
source install/setup.bash

3. Check the packages:

ros2 interface list | grep <YourCustomMessage>

---

Part 2: ROS 1 Development Environment (Click to Expand)

The ros_dev service provides a full ROS Noetic desktop environment with GUI support.

Configuration

Map your host directories to the container by editing docker-compose.yml:

Host Path	Container Path	Description
~/repos	/home/dev/repos	Source code repositories.
./catkin_ws	/home/dev/catkin_ws	The active Catkin workspace.

Workflow

1. Start the Service:

docker compose up -d ros_dev

2. Access the Shell:

docker exec -it ros_pet_container bash

3. GUI Visualization: If your host supports X11 forwarding, you can run GUI tools (rviz) directly.

---

Troubleshooting

• convert_bag: command not found: Add ~/.local/bin to your $PATH.
• Changes to code not appearing: The src folder is mounted. Changes apply immediately. If you add system dependencies (pip/apt), run docker compose build converter.
• Custom Messages not showing in Foxglove: Ensure the plugin emits a standard type (e.g., std_msgs/String) or provides a valid definition override in the return tuple.

---

Improvement Proposals (TODO) (Click to Expand)

Pending Improvements

• MCAP-to-MCAP Tooling: Generalize the architecture to support MCAP-to-MCAP manipulation. This would allow using the plugin system (filtering, anonymization) on native ROS 2 data, not just during conversion.

Completed

• Auto-splitting: Series conversion with static TF injection is implemented.
• Plugin System V2: Parametric configuration, 1-to-N message expansion, and timestamp injection.
• Automated Image Publishing: CI/CD scripts (publish-image.sh) are in place.
• Pre-Flight Checks: Implement a --dry-run mode to validate paths and disk space.
• UX: Added tqdm progress bars and detailed summary reports.

---

Maintainer Guide

To update the system dependencies (Dockerfile):

1. Login: echo $GITHUB_TOKEN | docker login ghcr.io -u USER --password-stdin
2. Publish: ./publish-image.sh