diff --git a/dp.md b/dp.md index 72bd1cce..d8e395dc 100644 --- a/dp.md +++ b/dp.md @@ -811,8 +811,9 @@ $$ More often, we refer to $v_\gamma$ by simply calling it the optimal value function. As for the finite-horizon setting, the infinite horizon discounted model does not require history-dependent policies, since for any $\pi \in \Pi^{HR}$ there exists a $\pi^{\prime} \in \Pi^{MR}$ with identical total discounted reward: + $$ -v_\gamma^*(s) \equiv \max_{\pi \in \Pi^{HR}} v_\gamma^\pi(s)=\max_{\pi \in \Pi^{MR}} v_\gamma^\pi(s) . +v_\gamma^*(s) \equiv \max_{\pi \in \Pi^{HR}} v_\gamma^\pi(s)=\max_{\pi \in \Pi^{MR}} v_\gamma^\pi(s) $$ ## Random Horizon Interpretation of Discounting