Skip to content

Commit a59f55e

Browse files
Merge pull request #12 from LearningToOptimize/ar/improve_class01
Ar/improve class01
2 parents a5f1aba + eaaa7d9 commit a59f55e

File tree

8 files changed

+180
-31
lines changed

8 files changed

+180
-31
lines changed

README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,16 +30,16 @@ This student-led course explores modern techniques for controlling — and learn
3030
| 1 | 08/22/2025 | Lecture — Andrew Rosemberg | Course map; why PDE-constrained **optimization**; tooling overview; stability & state-space dynamics; Lyapunov; discretization issues | |
3131
| 2 | 08/29/2025 | Lecture | Numerical **optimization** for control (grad/SQP/QP); ALM vs. interior-point vs. penalty methods | |
3232
| 3 | 09/05/2025 | Lecture | Pontryagin’s Maximum Principle; shooting & multiple shooting; LQR, Riccati, QP viewpoint (finite / infinite horizon) | |
33-
| 4 | 09/12/2025 | Lecture | Dynamic Programming & Model-Predictive Control | |
33+
| 4 | 09/12/2025 | **External seminar 1** - Joaquim Dias Garcia| Dynamic Programming & Model-Predictive Control | |
3434
| 5 | 09/19/2025 | Lecture | **Nonlinear** trajectory **optimization**; collocation; implicit integration | |
35-
| 6 | 09/26/2025 | **External seminar 1** - Henrique Ferrolho | Trajectory **optimization** on robots in Julia Robotics | |
35+
| 6 | 09/26/2025 | **External seminar 2** - Henrique Ferrolho | Trajectory **optimization** on robots in Julia Robotics | |
3636
| 7 | 10/03/2025 | Lecture | Essentials of PDEs for control engineers; weak forms; FEM/FDM review | |
37-
| 8 | 10/10/2025 | **External seminar 2** TBD (speaker to be confirmed) | Topology **optimization** | |
38-
| 9 | 10/17/2025 | **External seminar 3 — François Pacaud** | GPU-accelerated optimal control | |
37+
| 8 | 10/10/2025 | **External seminar 3** TBD (speaker to be confirmed) | Topology **optimization** | |
38+
| 9 | 10/17/2025 | **External seminar 4** — François Pacaud | GPU-accelerated optimal control | |
3939
|10 | 10/24/2025 | Lecture - Michael Klamkin | Physics-Informed Neural Networks (PINNs): formulation & pitfalls | |
40-
|11 | 10/31/2025 | **External seminar 4** - Chris Rackauckas | Neural Differential Equations: PINNs + classical solvers | |
40+
|11 | 10/31/2025 | **External seminar 5** - Chris Rackauckas | Neural Differential Equations: PINNs + classical solvers | |
4141
|12 | 11/07/2025 | Lecture - Pedro Paulo | Neural operators (FNO, Galerkin Transformer); large-scale surrogates | |
42-
|13 | 11/14/2025 | **External seminar 5** - Charlelie Laurent | Scalable PINNs / neural operators; CFD & weather applications | |
42+
|13 | 11/14/2025 | **External seminar 6** - Charlelie Laurent | Scalable PINNs / neural operators; CFD & weather applications | |
4343
|14 | 11/21/2025 | Lecture | Robust control & min-max DDP (incl. PDE cases); chance constraints; Data-driven control & RL-in-the-loop | |
4444

4545
## Reference Material

class01/background_materials/math_basics.jl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ md"
3737
| | | |
3838
|-----------:|:--|:------------------|
3939
| Lecturer | : | Rosemberg, Andrew |
40+
| Date | : | 28 of July, 2025 |
4041
4142
# Background Math (_Welcome to Pluto!_)
4243

class01/background_materials/optimization_basics.jl

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,12 @@ end
3434

3535
# ╔═╡ 0df8b65a-0527-4545-bf11-00e9912bced0
3636
md"""
37+
38+
| | | |
39+
|-----------:|:--|:------------------|
40+
| Lecturer | : | Rosemberg, Andrew |
41+
| Date | : | 28 of July, 2025 |
42+
3743
# Background – Modeling Optimization Problems in JuMP 🏗️
3844
3945
This short Pluto notebook walks you through three small optimisation models of increasing

class01/background_materials/optimization_motivation.jl

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -251,9 +251,10 @@ using Unitful
251251

252252
# ╔═╡ 533cbe59-a206-418e-a397-b21c68e314da
253253
md"
254-
Author: Andrew Rosemberg
255-
256-
Date: 28 of July, 2025
254+
| | | |
255+
|-----------:|:--|:------------------|
256+
| Lecturer | : | Rosemberg, Andrew |
257+
| Date | : | 28 of July, 2025 |
257258
"
258259

259260
# ╔═╡ 01c44cc2-68d5-11f0-2860-05c9ffbde13a

class01/class01_intro.jl

Lines changed: 161 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ end
1818

1919
# ╔═╡ 13b12c00-6d6e-11f0-3780-a16e73360478
2020
begin
21+
class_dir = @__DIR__
2122
import Pkg
2223
Pkg.activate(".")
2324
Pkg.instantiate()
@@ -28,6 +29,7 @@ begin
2829
using HypertextLiteral
2930
using PlutoTeachingTools
3031
using ShortCodes, MarkdownLiteral
32+
import Images: load
3133
end
3234

3335
# ╔═╡ b6ba1231-2942-4f06-8252-22f02553bb57
@@ -39,42 +41,50 @@ using ForwardDiff
3941
# ╔═╡ ec473e69-d5ec-4d6a-b868-b89dadb85705
4042
ChooseDisplayMode()
4143

44+
# ╔═╡ 8d7a34ef-5a2d-41a8-ac55-39ab00d7e432
45+
md"
46+
| | | |
47+
|-----------:|:--|:------------------|
48+
| Lecturer | : | Rosemberg, Andrew |
49+
| Date | : | 28 of July, 2025 |
50+
"
51+
4252
# ╔═╡ 1f774f46-d57d-4668-8204-dc83d50d8c94
4353
md"# Intro - Optimal Control and Learning
4454
4555
In this course, we are interested in problems with the following structure:
4656
4757
```math
4858
\begin{equation}
49-
\!\!\!\!\!\!\!\!\min_{\substack{(\mathbf y_1,\mathbf x_1)\\\mathrm{s.t.}}}
59+
\!\!\!\!\!\!\!\!\min_{\substack{(\mathbf u_1,\mathbf x_1)\\\mathrm{s.t.}}}
5060
\!\underset{%
51-
\phantom{\substack{(\mathbf y_1,\mathbf x_1)\\\mathrm{s.t.}}}%
52-
\!\!\!\!\!\!\!\!\!\!(\mathbf y_1,\mathbf x_1)\in\mathcal X_1(\mathbf x_0)%
61+
\phantom{\substack{(\mathbf u_1,\mathbf x_1)\\\mathrm{s.t.}}}%
62+
\!\!\!\!\!\!\!\!\!\!(\mathbf u_1,\mathbf x_1)\in\mathcal X_1(\mathbf x_0)%
5363
}{%
54-
\!\!\!\!f(\mathbf x_1,\mathbf y_1)%
64+
\!\!\!\!c(\mathbf x_1,\mathbf y_1)%
5565
}
5666
+\mathbb{E}_1\Bigl[
5767
\quad \cdots
5868
5969
\;+\;\mathbb{E}_t\Bigl[
60-
\min_{\substack{(\mathbf y_t,\mathbf x_t)\\\mathrm{s.t.}}}
70+
\min_{\substack{(\mathbf u_t,\mathbf x_t)\\\mathrm{s.t.}}}
6171
\!\underset{%
62-
\phantom{\substack{(\mathbf y_t,\mathbf x_t)\\\mathrm{s.t.}}}%
63-
\!\!\!\!(\mathbf y_t,\mathbf x_t)\in\mathcal X_t(\mathbf x_{t-1},w_t)%
72+
\phantom{\substack{(\mathbf u_t,\mathbf x_t)\\\mathrm{s.t.}}}%
73+
\!\!\!\!(\mathbf u_t,\mathbf x_t)\in\mathcal X_t(\mathbf x_{t-1},w_t)%
6474
}{%
65-
\!\!\!\!\!\!\!\!\!\!f(\mathbf x_t,\mathbf y_t)%
75+
\!\!\!\!\!\!\!\!\!\!c(\mathbf x_t,\mathbf u_t)%
6676
}
6777
+\mathbb{E}_{t+1}[\cdots]
6878
\Bigr].
6979
\end{equation}
7080
```
71-
which minimizes a first stage cost function $f(\mathbf{x}_1,
72-
\mathbf{y}_1)$ and the expected value of future costs over possible
81+
which minimizes a first stage cost function $c(\mathbf{x}_1,
82+
\mathbf{u}_1)$ and the expected value of future costs over possible
7383
values of the exogenous stochastic variable $\{w_{t}\}_{t=2}^{T} \in
7484
\Omega$.
7585
7686
Here, $\mathbf{x}_0$ is the initial system state and the
77-
control decisions $\mathbf{y}_t$ are obtained at every period $t$
87+
control decisions $\mathbf{u}_t$ are obtained at every period $t$
7888
under a feasible region defined by the incoming state
7989
$\mathbf{x}_{t-1}$ and the realized uncertainty $w_t$. $\mathbf{E}_t$ represents the expected value over future uncertainties $\{w_{\tau}\}_{\tau=t}^{T}$. This
8090
optimization program assumes that the system is entirely defined by
@@ -87,7 +97,7 @@ constraints can be generally posed as:
8797
\begin{align}
8898
&\mathcal{X}_t(\mathbf{x}_{t-1}, w_t)=
8999
\begin{cases}
90-
\mathcal{T}(\mathbf{x}_{t-1}, w_t, \mathbf{y}_t) = \mathbf{x}_t \\
100+
f(\mathbf{x}_{t-1}, w_t, \mathbf{u}_t) = \mathbf{x}_t \\
91101
h(\mathbf{x}_t, \mathbf{y}_t) \geq 0
92102
\end{cases}
93103
\end{align}
@@ -98,24 +108,116 @@ constraints can be generally posed as:
98108
md"""
99109
where the outgoing state of the system $\mathbf{x}_t$ is a
100110
transformation based on the incoming state, the realized uncertainty,
101-
and the control variables. $h(\mathbf{x}_t, \mathbf{y}_t) \geq 0$
102-
captures the state constraints. Markov Decision Process (MDPs) refer
103-
to $\mathcal{T}$ as the "transition kernel" of the system. State and
111+
and the control variables. In the Markov Decision Process (MDP) framework, we refer to $f$ as the "transition kernel" of the system. State and
104112
control variables are restricted further by additional constraints
105113
captured by $h(\mathbf{x}_t, \mathbf{y}_t) \geq 0$. We
106-
consider policies that map the past information into decisions. In
114+
consider policies that map the past information into decisions: $\pi_t : (\mathbf{x}_{t-1}, w_t) \rightarrow \mathbf{x}_t$. In
107115
period $t$, an optimal policy is given by the solution of the dynamic
108116
equations:
109117
110118
```math
111119
\begin{align}
112-
V_{t}(\mathbf{x}_{t-1}, w_t) = &\min_{\mathbf{x}_t, \mathbf{y}_t} \quad \! \! f(\mathbf{x}_t, \mathbf{y}_t) + \mathbf{E}[V_{t+1}(\mathbf{x}_t, w_{t+1})] \\
113-
& \text{ s.t. } \quad\mathbf{x}_t = \mathcal{T}(\mathbf{x}_{t-1}, w_t, \mathbf{y}_t) \nonumber \\
114-
& \quad \quad \quad \! \! h(\mathbf{x}_t, \mathbf{y}_t) \geq 0. \nonumber
120+
V_{t}(\mathbf{x}_{t-1}, w_t) = &\min_{\mathbf{x}_t, \mathbf{u}_t} \quad \! \! c(\mathbf{x}_t, \mathbf{u}_t) + \mathbf{E}_{t+1}[V_{t+1}(\mathbf{x}_t, w_{t+1})] \\
121+
& \text{ s.t. } \quad\mathbf{x}_t = f(\mathbf{x}_{t-1}, w_t, \mathbf{u}_t) \nonumber \\
122+
& \quad \quad \quad \! \! h(\mathbf{x}_t, \mathbf{u}_t) \geq 0. \nonumber
115123
\end{align}
116124
```
125+
```math
126+
\implies \pi_t^{*}(\mathbf{x}_{t-1}, w_t) \in \{\mathbf{x}_t \;|\; \exists u_t \;:\; c(\mathbf{x}_t, \mathbf{u}_t) + \mathbf{E}_{t+1}[V_{t+1}(\mathbf{x}_t, w_{t+1})] = V_{t}(\mathbf{x}_{t-1}, w_t) \}
127+
```
128+
129+
"""
130+
131+
# ╔═╡ 1d7092cd-0044-4d38-962a-ce3214c48c24
132+
md"""
133+
Function $V_{t}(\mathbf{x}_{t-1}, w_t)$ is refered to as the value function. To find the optimal policy for the $1^{\text{st}}$ stage, we need to find the optimal policy for the entire horizon $\{t=2,\cdots,T\}$ or at least estimate the "optimal" value function.
134+
"""
135+
136+
# ╔═╡ 60ba261a-f2eb-4b45-ad6d-b6042926ccab
137+
load(joinpath(class_dir, "indecision_tree.png"))
138+
139+
# ╔═╡ 15709f7b-943e-4190-8f40-0cfdb8772183
140+
md"""
141+
Notice that the number of "nodes" to be evaluated (either decisions or their cost) grows exponetially with the number of stages. This the the *Curse of dimensionality*
142+
in stochastic programming.
143+
144+
"""
145+
146+
# ╔═╡ 5d7a4408-21ff-41ec-b004-4b0a9f04bb4f
147+
question_box(md"Can you name a few ways to try and/or solve this problem?")
148+
149+
# ╔═╡ c08f511e-b91d-4d17-a286-96469c31568a
150+
md"## Example: Robotic Arm Manipulation"
151+
152+
# ╔═╡ b3129bcb-c24a-4faa-a5cf-f69ce518ea87
153+
load(joinpath(class_dir, "nlp_robot_arm.png"))
154+
155+
# ╔═╡ c1f43c8d-0616-4572-bb48-dbb71e40adda
156+
md"""
157+
The tip of the second link is computed using the direct geometric model:
158+
159+
```math
160+
p(\theta_{1},\theta_{2}) \;=\;
161+
\begin{cases}
162+
x = L_{1}\,\sin\theta_{1} \;+\; L_{2}\,\sin\!\bigl(\theta_{1}+\theta_{2}\bigr),\\[6pt]
163+
y = L_{1}\,\cos\theta_{1} \;+\; L_{2}\,\cos\!\bigl(\theta_{1}+\theta_{2}\bigr).
164+
\end{cases}
165+
\tag{1}
166+
```
117167
"""
118168

169+
# ╔═╡ 57d896ca-221a-4cfc-b37a-be9898fac923
170+
begin
171+
md"""
172+
**State**
173+
```math
174+
\mathbf{x}_t=\begin{bmatrix}\theta_{1,t}&\theta_{2,t}&\dot\theta_{1,t}&\dot\theta_{2,t}\end{bmatrix}^{\!\top}
175+
```
176+
177+
**Control**
178+
```math
179+
\mathbf{u}_t=\boldsymbol\tau_t=\begin{bmatrix}\tau_{1,t}&\tau_{2,t}\end{bmatrix}^{\!\top}
180+
```
181+
182+
**Dynamics** (Euler sample time Δt)
183+
```math
184+
\mathbf{x}_{t+1}=f_d(\mathbf{x}_t,\mathbf{u}_t)
185+
\;\;\equiv\;
186+
\begin{bmatrix}
187+
\boldsymbol\theta_t+\Delta t\,\dot{\boldsymbol\theta}_t\\[2pt]
188+
\dot{\boldsymbol\theta}_t+\Delta t\,\mathcal{M}^{-1}(\boldsymbol\theta_t)(B(\boldsymbol\theta_t)\boldsymbol\tau_t + F(w_t) - C(\boldsymbol\theta_t,\boldsymbol{\dot\theta})\bigr)
189+
\end{bmatrix}
190+
```
191+
192+
**Stage cost**
193+
194+
```math
195+
c(\mathbf{x}_t,\mathbf{u}_t)=
196+
\underbrace{\|p(\boldsymbol\theta_t)-p_{\text{target}}\|_2^{2}}_{\text{tracking}}
197+
+\;\lambda_\tau\|\boldsymbol\tau_t\|_2^{2}\;,
198+
\qquad \lambda_\tau>0 .
199+
```
200+
201+
Terminal cost
202+
$V_T(\mathbf{x}_T)=\|p(\boldsymbol\theta_T)-p_{\text{target}}\|_2^{2}$.
203+
204+
**Constraints**
205+
206+
```math
207+
h(\mathbf{x}_t,\mathbf{u}_t)\ge 0\;:\;
208+
\begin{cases}
209+
\theta_{\min}\le\boldsymbol\theta_t\le\theta_{\max} &\text{(joint limits)}\\
210+
|\dot{\boldsymbol\theta}_t|\le\dot\theta_{\max} &\text{(velocity limits)}\\
211+
|\boldsymbol\tau_t|\le\tau_{\max} &\text{(actuator limits)}
212+
\end{cases}
213+
```
214+
215+
"""
216+
end
217+
218+
# ╔═╡ e2d3d160-d3b6-41f2-a8bc-2878ba71e78c
219+
220+
119221
# ╔═╡ 52005382-177b-4a11-a914-49a5ffc412a3
120222
section_outline(md"A Crash Course:",md" (Continuous-Time) Dynamics
121223
")
@@ -361,16 +463,41 @@ Foldable(md"All mechanical systems can be written this way. Why?", md"""
361463
362464
Manipulator Dynamics Equations are a way of rewriting the Euler--Lagrange equations.
363465
364-
> In the calculus of variations and classical mechanics, the Euler–Lagrange equations are a system of second-order ordinary differential equations whose solutions are stationary points of the given action functional. The equations were discovered in the 1750s by Swiss mathematician Leonhard Euler and Italian mathematician Joseph-Louis Lagrange.
466+
#### 🚀 Detour: The Principle of Least Action 🚀
365467
468+
In the calculus of variations and classical mechanics, the Euler–Lagrange equations are a system of second-order ordinary differential equations whose solutions are stationary points of the given action functional.
469+
470+
> The equations were discovered in the 1750s by Swiss mathematician Leonhard Euler and Italian mathematician Joseph-Louis Lagrange.
471+
472+
In classical mechanics:
473+
366474
```math
367475
L = \underbrace{\frac{1}{2} v^{\top}M(q)v}_{\text{Kinematic Energy}} - \underbrace{U(q)}_{\text{Potential Energy}}
368476
```
369477
370-
What can you say about $M(q)$? When do we have a problem inverting it?
478+
A curve ($q^\star(t)$) is physically realised iff it is a stationary
479+
point of ($\mathcal{S}$) :
480+
481+
```math
482+
\delta\mathcal{S}=0
483+
\;\;\Longrightarrow\;\;
484+
\frac{d}{dt}\!\bigl(\tfrac{\partial L}{\partial\dot q}\bigr)
485+
- \frac{\partial L}{\partial q}=0
486+
\quad\Longrightarrow\quad
487+
M(q)\,\ddot q + C(q,\dot q)\,\dot q + \nabla U(q)=0 .
488+
```
371489
372490
""")
373491

492+
# ╔═╡ f3d155c6-5384-481a-8373-582e753ea8d6
493+
question_box(md"What can you say about $M(q)$? When do we have a problem inverting it?")
494+
495+
# ╔═╡ b9aeab8a-f8ea-4310-8568-5d6bda0bb4d3
496+
question_box(md"Can you derive the stationary condition?")
497+
498+
# ╔═╡ e1dc6ecf-4e62-415a-a620-0731953c5ab4
499+
500+
374501
# ╔═╡ 5f35a169-887f-477f-b010-167627f7ce4c
375502
md"## Linear Systems
376503
@@ -1029,8 +1156,18 @@ end
10291156
# ╔═╡ Cell order:
10301157
# ╟─13b12c00-6d6e-11f0-3780-a16e73360478
10311158
# ╟─ec473e69-d5ec-4d6a-b868-b89dadb85705
1159+
# ╟─8d7a34ef-5a2d-41a8-ac55-39ab00d7e432
10321160
# ╟─1f774f46-d57d-4668-8204-dc83d50d8c94
10331161
# ╟─a0f71960-c97c-40d1-8f78-4b1860d2e0a2
1162+
# ╟─1d7092cd-0044-4d38-962a-ce3214c48c24
1163+
# ╟─60ba261a-f2eb-4b45-ad6d-b6042926ccab
1164+
# ╟─15709f7b-943e-4190-8f40-0cfdb8772183
1165+
# ╟─5d7a4408-21ff-41ec-b004-4b0a9f04bb4f
1166+
# ╟─c08f511e-b91d-4d17-a286-96469c31568a
1167+
# ╟─b3129bcb-c24a-4faa-a5cf-f69ce518ea87
1168+
# ╟─c1f43c8d-0616-4572-bb48-dbb71e40adda
1169+
# ╟─57d896ca-221a-4cfc-b37a-be9898fac923
1170+
# ╠═e2d3d160-d3b6-41f2-a8bc-2878ba71e78c
10341171
# ╟─52005382-177b-4a11-a914-49a5ffc412a3
10351172
# ╟─8ea866a6-de0f-4812-8f59-2aebec709243
10361173
# ╟─2be161cd-2d4c-4778-adca-d45f8ab05f98
@@ -1048,6 +1185,9 @@ end
10481185
# ╟─f10927fe-d392-4374-bad1-ab5ac85b8116
10491186
# ╟─b8b206ef-cdc5-4cc9-9b55-70d711ba2a9e
10501187
# ╟─a09de9e4-7ecc-4d23-9135-384077f0c03f
1188+
# ╟─f3d155c6-5384-481a-8373-582e753ea8d6
1189+
# ╟─b9aeab8a-f8ea-4310-8568-5d6bda0bb4d3
1190+
# ╠═e1dc6ecf-4e62-415a-a620-0731953c5ab4
10511191
# ╟─5f35a169-887f-477f-b010-167627f7ce4c
10521192
# ╟─e860d92b-cc8f-479b-a0fc-e5f7a11ae1fd
10531193
# ╟─bb4bfa72-bf69-41f5-b017-7cbf31653bae

class01/indecision_tree.png

214 KB
Loading

class01/nlp_robot_arm.png

1.99 MB
Loading

class04/class04.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Class 4 — 09/12/2025
22

3-
**Presenter:** TBD
3+
**Presenter:** Joaquim Dias Garcia
4+
**Liason:** TBD
45

56
**Topic:** Dynamic Programming & Model-Predictive Control
67

0 commit comments

Comments
 (0)