Embedded Machine Learning for Dynamical Systems

Matthew O’Brien | Undergraduate Research Project

Embedded machine learning study comparing Vanilla Neural Networks (VNNs), Symplectic Neural Networks (SNNs), and Physics-Informed Neural Networks (PINNs) for simple harmonic oscillator (SHO) dynamics, with deployment on the Raspberry Pi Pico (RP2040).

Project focus: training models in Python, exporting learned parameters, deploying them in embedded C, and comparing behavior in terms of accuracy, rollout stability, runtime, memory, and embedded feasibility.

Out-of-Distribution (OOD) Phase Trajectory Comparison: The SNN maintains a stable circular trajectory consistent with true system dynamics, while the VNN diverges over time due to poor generalization outside the training distribution.

Basic usage

To work with the training pipelines:

• Vanilla Neural Network: python3 VanillaNet/SHOVanillaNetV2.py
• Symplectic Neural Network: python3 Strupnet/SHOStrumpnet.py
• Physics-Informed Neural Network: python3 PINN/SHO_PINN.py

To build embedded firmware:

• mkdir build && cd build
• cmake ..
• make pico_vnn
• make pico_snn
• make pico_pinn

To analyze results, use the model-specific scripts, benchmark outputs, and plots stored across:

• Benchmarks/
• SimpleHarmonicOscillator/
• VanillaNet/
• Strupnet/
• PINN/

Summary

This project asks a simple question: can neural networks for dynamical systems remain useful once deployed on constrained embedded hardware?

To study that, this repository uses the simple harmonic oscillator as a controlled physical system and compares three model classes:

• VNN — standard feedforward baseline
• SNN — structure-aware symplectic model
• PINN — physics-informed model with constrained learning

The goal is not only to compare prediction quality in Python, but to examine how each model behaves after export and deployment onto a real microcontroller.

Workflow

1. Generate SHO trajectories and train neural models offline in Python.
2. Export learned parameters into embedded-friendly formats (headers/JSON).
3. Compile model-specific firmware targets for Raspberry Pi Pico.
4. Run on-device inference and compare phase-space behavior, rollout quality, runtime, and memory footprint.

Releases

This repository currently contains three major model lines, each representing a different release path:

• pico_vnn
• pico_snn
• pico_pinn

Architecture at a glance

All three models use a 2D SHO phase-space state (p, q) as input and output, but they differ in structure, parameter count, and rollout method.

Model	Core architecture	Hidden structure	Trainable parameters	Rollout / integrator view
VNN	Feedforward MLP with residual one-step state update	Linear(2,32) -> ReLU -> Linear(32,2)	162	Explicit Euler-style neural step
SNN (SympNet)	Structure-preserving P-SympNet (method='P', layers=2, max_degree=2)	2 symplectic kick layers	6 (2 layers × (a:1 + w:2))	Symplectic kick composition (kick-drift style Hamiltonian update)
PINN	Physics-informed FCN learning SHO state derivatives with physics residual loss	Linear(2,32) -> Tanh -> Linear(32,32) -> Tanh -> Linear(32,32) -> Tanh -> Linear(32,2)	2274	RK4 rollout over learned dynamics (in analysis script)

Integrator summary (quick view):

• VNN: one neural forward-Euler style residual step per timestep.
• SNN: two-layer symplectic kick composition (structure-preserving update).
• PINN: neural vector field integrated with RK4 for rollout.

Results

Model	Train MRSE (%)	IID MRSE (%)	OOD MRSE (%)	Avg. Runtime (s)	Avg. Energy (mJ)	Flash (KB)	RAM (KB)
VNN	19.90	20.53	85.61	0.268	29.48	26.1	11.4
SNN	4.69	4.65	4.68	0.240	26.40	25.5	11.4
PINN	6.21	6.70	55.55	2.671	293.81	36.4	11.4

Dependencies

• TensorFlow
• NumPy
• Pandas
• Matplotlib
• Scikit-learn
• SciPy
• strupnet
• pyserial

This project combines Python-based model development with C-based embedded deployment on the Raspberry Pi Pico.