A hands-on curriculum for robotics, computer vision, and AI using MATLAB and UR robots
Website: https://teaching.ioc.upc.edu/from-code-to-robot/
From Code to Robot is an open, modular, and hands-on curriculum designed to teach the foundations of:
- 🤖 Robotics
- 👁️ Computer Vision
- 🧠 Artificial Intelligence
using MATLAB and Universal Robots (UR) manipulators.
The project combines theory with practical experimentation through simulation, real robotic platforms, and executable MATLAB scripts. It is intended for undergraduate and graduate courses in:
- Industrial Engineering
- Robotics
- Computer Science
- Automation and Control
- Artificial Intelligence
The curriculum is organized as a flexible collection of tutorials, demos, exercises, and projects that instructors can easily adapt to different courses and academic levels.
Provide students with a practical and interdisciplinary framework to design, simulate, perceive, plan, and execute robotic tasks using modern tools in robotics, computer vision, and artificial intelligence.
The curriculum bridges the gap between theory and real robotic applications through reproducible hands-on exercises using MATLAB and UR robots.
- Develop open teaching materials for robotics education
- Promote interdisciplinary learning across robotics, vision, and AI
- Facilitate adoption of practical robotics courses
- Support experimentation with industrial robotic platforms
- Publish teaching innovation and educational research outcomes
- Provide reusable tutorials, demos, and documentation
- Engineering students
- Robotics researchers
- Educators and instructors
- AI and computer vision practitioners
- Laboratories introducing hands-on robotics education
✅ Hands-on and project-based learning
✅ Modular curriculum adaptable to many courses
✅ Integration of Robotics + Vision + AI
✅ MATLAB-centered workflow
✅ Simulation and real robot execution
✅ Open educational resources
✅ Materials available in:
- English
- Spanish
- Catalan
The curriculum is intentionally modular.
It can be used:
- As a complete interdisciplinary robotics course
- As independent modules in robotics, vision, or AI courses
- In undergraduate or master-level programs
- In laboratory sessions, projects, or tutorials
This flexibility allows instructors to adapt the materials to their own teaching objectives and course structures.
After completing the curriculum, students will be able to:
- Model and control robotic manipulators
- Solve forward and inverse kinematics problems
- Plan trajectories and analyze robot motion
- Understand robot dynamics and decentralized control
- Train and validate object detection models using YOLOv8
- Build datasets and apply data augmentation techniques
- Integrate perception with robotic manipulation
- Implement robotic task planning using Reinforcement Learning
- Design intelligent robotic behaviors using Q-learning and Deep Q-learning
- Connect AI decision-making with robot execution
- Develop complete robotics pipelines from perception to action
The curriculum is organized into three interconnected pillars.
This module introduces the foundations of robotic manipulators through MATLAB simulations and experiments with UR robots.
- DH parameter modeling
- Forward and inverse kinematics
- Differential kinematics and Jacobians
- Singularities and redundancy
- Trajectory planning
- Dynamic modeling
- Motion control
- Simulation with Simulink
- MATLAB live scripts
- Symbolic Math Toolbox
- Robotics System Toolbox
- Simulink models
- UR3 simulations using:
- docker-based reproducible ROS 2 Jazzy environment.
- Real robot experiments using:
- Universal Robots Support Package
Students learn how to mathematically model, simulate, and control industrial robotic manipulators in realistic scenarios.
This module focuses on object detection and robotic perception using deep learning techniques.
- YOLOv8 object detection
- Dataset generation and annotation
- Bounding box labeling
- Data augmentation
- Transfer learning
- Model training and validation
- Camera calibration
- Vision-to-robot coordinate transformation
- MATLAB Deep Learning Toolbox
- Transfer learning with pretrained YOLOv8 models
- Integration with robotic manipulation tasks
- Detection of custom objects for robot interaction
Students learn how to train vision systems capable of detecting objects and providing spatial information for robotic manipulation.
This module introduces robotic task planning using Reinforcement Learning techniques.
- STRIPS representations
- Task planning
- Q-learning
- Deep Q-learning
- Sequential decision-making
- Action pruning
- Intelligent robotic behaviors
- MATLAB Reinforcement Learning Toolbox
- Planning pipelines connected to robot execution
- AI-driven task sequencing
- Integration with robotics and computer vision modules
Students learn how robots can autonomously make decisions, optimize task execution, and plan complex behaviors.
The curriculum is based on MATLAB and several MathWorks toolboxes for robotics, computer vision, and artificial intelligence.
- MATLAB (recommended version: R2025a or R2025b)
- Simulink
- Robotics System Toolbox
- Symbolic Math Toolbox
- Deep Learning Toolbox
- Reinforcement Learning Toolbox
- Computer Vision Toolbox
- Image Processing Toolbox
- Robotics System Toolbox Support Package for Universal Robots UR Series Manipulators
- Universal Robots UR3 / UR5 manipulators
- RGB camera
From Code to Robot is a project developed at the Universitat Politècnica de Catalunya and funded by MathWorks and Universal Robots.
-
Robotics
- Constantin Sul (Universitat Politècnica de Catalunya)
- Prof. Jan Rosell (Universitat Politècnica de Catalunya)
-
Computer Vision and Artificial Intelligence
- Noel Nathan Planell (Universitat Politècnica de Catalunya)
- Prof. Isiah Zaplana (Universitat Politècnica de Catalunya)
- Jennifer Gago (MathWorks)
- Carlos Pérez (Universal Robots)
Current Version: 0.0.1
Manipulability ellipsoid
Torque control
