Stochastic control with rough paths

Abstract

We study a class of controlled differential equations driven by rough paths (or rough path realizations of Brownian motion) in the sense of Lyons. It is shown that the value function satisfies a HJB type equation; we also establish a form of the Pontryagin maximum principle. Deterministic problems of this type arise in the duality theory for controlled diffusion processes and typically involve anticipating stochastic analysis. We make the link to old work of Davis and Burstein (Stoch Stoch Rep 40:203–256, 1992) and then prove a continuous-time generalization of Roger’s duality formula [SIAM J Control Optim 46:1116–1132, 2007]. The generic case of controlled volatility is seen to give trivial duality bounds, and explains the focus in Burstein–Davis’ (and this) work on controlled drift. Our study of controlled rough differential equations also relates to work of Mazliak and Nourdin (Stoch Dyn 08:23, 2008).

Appendix: RDEs with Controlled Drift

Theorem 29

(RDE with controlled drift) Let $p \in [2,3)$. Let ${\mathbf {\eta }} \in \mathcal {C}^{0,{{\text {p-var}}}}$ a geometric p-variation rough path. Let $\gamma > p$. Let U be the subset of a separable Banach space. Let $b: {\mathbb {R}} ^e \times U \rightarrow {\mathbb {R}} ^e$ such that $b(\cdot ,u) \in \mathrm{Lip}^1( { \mathbb {R}} ^e)$ uniformly in $u \in U$ (i.e. $\sup _{u\in U} ||b(\cdot ,u)||_{ \mathrm{Lip}^1( {\mathbb {R}} ^e)} < \infty$) and such that $u \mapsto b(\cdot ,u)$ is measurable. Let $\sigma _1,\ldots ,\sigma _d \in \mathrm{Lip} ^\gamma ( {\mathbb {R}} ^e)$. Let $\mu : [0,T] \rightarrow U$ be measurable, i.e. $\mu \in \mathcal {M}$.

(i) There exists a unique $Y \in \mathcal {C}^{0,{{\text {p-var}}}}$ that solves

$$\begin{aligned} Y_t = y_0 + \int _0^t b( Y_r, \mu _r ) dr + \int _0^t \sigma ( Y ) d{\mathbf {\eta }}_r. \end{aligned}$$

Moreover the mapping

$$\begin{aligned} (x_0,\mathbf {\eta }) \mapsto Y \in \mathcal {C}^{0,{{\text {p-var}}}} \end{aligned}$$

is locally Lipschitz continuous, uniformly in $\mu \in \mathcal {M}$.

(ii) Assume that $u \mapsto b(\cdot ,u)$ is Lipschitz. If we use the topology of convergence in measure on $\mathcal {M}$, then

$$\begin{aligned} \mathcal {M} \times {\mathbb {R}} ^e \times \mathcal {C}^{0,{{\text {p-var}}}}&\rightarrow \mathcal {C}^{0,{{\text {p-var}}}} \nonumber \\ (\mu ,x_0,{\mathbf {\eta }})&\mapsto Y, \end{aligned}$$

(4.16)

is continuous.

(iii) Assume that $u \mapsto b(\cdot ,u)$ is Lipschitz. If $\nu : \Omega \times [0,T] \rightarrow U$ is progressively measurable and $\mathbf {B}$ is the Stratonovich rough path lift of a Brownian motion B, then ¶

$$\begin{aligned} \left. Y \right| _{\mu =\nu ,{\mathbf {\eta }}=\mathbf {B}} = \tilde{Y}, \qquad -a.s., \end{aligned}$$

(4.17)

where $\tilde{Y}$ is the (classical) solution to the controlled SDE

$$\begin{aligned} \tilde{Y}_t = y_0 + \int _0^t b( \tilde{Y}_r, \nu _r ) dr + \int _0^t \sigma ( \tilde{Y} ) \circ dB_r. \end{aligned}$$

(iv) If $\sigma _1,\ldots ,\sigma _d \in \mathrm{Lip}^{\gamma +2}( { \mathbb {R}} ^e)$, we can write $Y = \phi (t, \hat{Y}_t)$ where $\phi$ is the solution flow to the RDE

$$\begin{aligned} \phi (t,x) = x + \int _0^t \sigma (\phi (r,x)) d{\mathbf {\eta }}_r, \end{aligned}$$

and $\hat{Y}$ solves the classical ODE

$$\begin{aligned} \hat{Y}_t = x_0 + \int _0^t \hat{b}(r, \hat{Y}_r, \mu _r) dr, \end{aligned}$$

where we define componentwise

$$\begin{aligned} \hat{b}(t,x,u)_i = \sum _k \partial _{x_k} (\phi ^{-1})_i( t, \phi (t,x) ) b_k(\phi (t,x), u). \end{aligned}$$

Remark 30

In the last case, i.e. point (iv), we can immediately use results in [15] (Theorem 10.53) to also handle linear vector fields.

Remark 31

Given a rough path $\mathbf {\eta }$ on [0, T], the time-inverted object $\overleftarrow{ \mathbf {\eta } }_t := \mathbf {\eta }_{T-t}$ is again a rough path. We can hence solve controlled, backward RDEs using the previous Theorem by inverting time.

Proof

Denote for $\mu \in \mathcal {M}$

$$\begin{aligned} Z^\mu _t(\cdot ) := \int _0^t b(\cdot ,\mu _r) dr, \end{aligned}$$

which is a well defined Bochner integral in the space $\mathrm{Lip}^1( { \mathbb {R}} ^e)$ (indeed, by assumption on b, $\int _0^t ||b(\cdot ,\mu _r)||_{\mathrm{Lip}^1( {\mathbb {R}} ^e)} dr < \infty$). Then $Z^\mu \in C^{{{\text {1-var}}}}( [0,T], \mathrm{Lip}^1( {\mathbb {R}} ^e) )$. Indeed

$$\begin{aligned} ||Z^\mu ||_{{{\text {1-var}}}}&\le \int _0^T ||b(\cdot , \mu _r) ||_{\mathrm{Lip}^1( {\mathbb {R}} ^e)} dr \nonumber \\&\le \int _0^T \sup _{u \in \mathcal {U}} ||b(\cdot , u) ||_{\mathrm{Lip}^1( {\mathbb {R}} ^e)} dr, \end{aligned}$$

(4.18)

independent of $\mu \in \mathcal {M}$.

By Theorem 33 we get a unique solution to the RDE

$$\begin{aligned} dY = f(Y) dZ^\mu + \sigma (Y) d{\mathbf {\eta }}, \end{aligned}$$

where $f: {\mathbb {R}} ^e \rightarrow L(\mathrm{Lip}^1( {\mathbb {R}} ^e), {\mathbb {R}} ^e)$ is the evaluation operator, i.e. $f(y) V := V(y)$. This gives existence of the controlled RDE as well as continuity in the starting point and in ${\mathbf {\eta }}$. By (4.18), this is independent of $\mu \in \mathcal {M}$ and we hence have shown (i).

Concerning (ii), assume now that $U \ni u \mapsto b(\cdot ,u) \in \mathrm{Lip} ^1$ is Lipschitz. Using the representation given in the proof of (i) it is sufficient to realize that if $\mu ^n \rightarrow \mu \in \mathcal {M}$ in measure, then $Z^{\mu ^n} \rightarrow Z^\mu$ in $C^{{{\text {1-var}}}}([0,T],\mathrm{Lip}^1( {\mathbb {R}} ^e))$.

Concerning (iii): First of all, we can regard $\nu$ as a measurable mapping from $(\Omega , {\mathcal {F}} )$ into the space of all measurable mappings from $[0,T] \rightarrow U$ with the topology of convergence in measure. Indeed, if U is a compact subset of a separable Banach space, then this follows from the equivalence of weak and strong measurability for Banach space valued mappings (Pettis Theorem, see Section V.4 in [35]). If U is a general subset of a separable Banach space, then define $\nu ^n: \Omega \rightarrow \mathcal {M}$ with $\nu ^n(\omega )_t := \Phi ^n( \nu (\omega )_t )$. Here $\Phi ^n$ is a (measurable) nearest-neighbor projection on $\{x_1,\ldots , x_n\}$, the sequence $(x_k)_{k \ge 0}$ being dense in the Banach space. Then $\nu ^n$ is taking values in a compact set and hence by the previous case, is measurable as a mapping to $\mathcal {M}$. Finally $\nu$ is the pointwise limit of the $\nu ^n$ and hence also measurable.

Hence $\left. Y \right| _{\mu =\nu ,{\mathbf {\eta }}=\mathbf {B}}$ is measurable, as the concatenation of measurable maps (here we use the joint continuity of RDE solutions in the control and the rough path, i.e. continuity of the mapping (4.16)).

Now, to get the equality (4.17): we can argue as in [14] using the Riemann sum representation of stochastic integral.

(iv) This follows from Theorem 1 in [9] or Theorem 2 in [5]. $\square$

Remark 32

One can also prove “by hand” existence of a solution, using a fixpoint argument, like the one used in [16]. This way one arrives at the same regularity demands on the coefficients. Using the infinite dimensional setting makes it possible to immediately quote existing results on existence, which shortens the proof immensely. We thank Terry Lyons for drawing our attention to this fact.

In the proof of the previous theorem we needed the following version of Theorem 6.2.1 in [25].

Theorem 33

Let V, W, Z be some Banach spaces. Let tensor products be endowed with the projective tensor norm.⁸ Let $p \in [2,3)$ ${\mathbf {\eta }} \in \mathcal {C}^{0,{{\text {p-var}}}}(W)$ and $Z \in C^{{{\text {q-var}}}}([0,T], V)$ for some $1/q > 1 - 1/p$. Let $f: Z \rightarrow L(V,Z)$ be $\mathrm{Lip}^1$, let $g: Z \rightarrow L(W,Z)$ be $\mathrm{Lip}^\gamma$, $\gamma > p$. Then there exists a unique solution $Y \in \mathcal {C} ^{0,{{\text {p-var}}}}(Z)$ to the RDE

$$\begin{aligned} dY = f(Y) dZ + g(Y) d{\mathbf {\eta }}, \end{aligned}$$

in the sense of Lyons.⁹

Moreover for every $R>0$ there exists $C = C(R)$ such that

$$\begin{aligned} \rho _{{{\text {p-var}}}}( Y, \bar{Y} ) \le C ||Z - \bar{Z}||_{{{\text {q-var}}}}. \end{aligned}$$

whenever (Z, X) and $(\bar{Z}, X)$ are two driving paths with $||Z||_{{{\text {q-var}}}}, ||\bar{Z}||_{{{\text {q-var}}}}, ||X||_{{{\text {p-var}}}} \le R$.

Proof

Since Z and X have complementary Young regularity (i.e. $1/p + 1/q > 1$) there is a canonical joint rough path $\lambda$ over (Z, X), where the missing integrals of Z and the cross-integrals of Z and X are defined via Young integration. So we have

$$\begin{aligned} \lambda _{s,t} = 1 + \left( \begin{array}{c} Z_{s,t} \\ X_{s,t} \end{array} \right) + \left( \begin{array}{cc} \int _s^t Z_{s,r} \otimes dZ_r &{} \int _s^t Z_{s,r} \otimes d\eta _r \\ \int _s^t \eta _{s,r} \otimes dZ_r &{} \int _s^t \eta _{s,r} \otimes d\eta _r \end{array} \right) \end{aligned}$$

Then, by Theorem 6.2.1 in [25], there exists a unique solution to the RDE

$$\begin{aligned} dY = h(Y) d\lambda , \end{aligned}$$

where $h = (f, g)$.

We calculate how $\lambda$ depends on Z. For the first level we have of course $||\lambda ^{(1)} - \bar{\lambda }^{(1)}||_{{{\text {p-var}}}} \le ||Z - \bar{Z}||_{{{\text {q-var}}}}$. For the second level we have, by Young’s inequality,

$$\begin{aligned} \left| \int _{s}^{t} Z_{s,r} dZ_r - \int _{s}^{t} \bar{Z}_{s,r} d\bar{Z}_r\right|&\le \left| \int _{s}^{t} Z_{s,r} d\left[ Z_r - \bar{Z}_r \right] \right| +|\int _{s}^{t} Z_{s,r} - \bar{Z}_{s,r} d\bar{Z}_r| \\&\le c ||Z||_{{{\text {q-var}}};[s,t]} ||Z - \bar{Z}||_{{{\text {q-var}}};[s,t]}\\&\quad + c ||Z - \bar{Z}||_{{{\text {q-var}}};[s,t]} ||Z||_{{{\text {q-var}}};[s,t]}. \end{aligned}$$

and similarily

$$\begin{aligned} |\int _{s}^{t} X_{s,r} dZ_r - \int _{s}^{t} X_{s,r} d\bar{Z}_r| \le c ||X||_{{{\text {p-var}}};[s,t]} ||Z||_{{{\text {q-var}}};[s,t]}. \end{aligned}$$

Together this gives

$$\begin{aligned} \rho _{{{\text {p-var}}}}( \lambda , \bar{\lambda }) \le c ||Z - \bar{Z}||_{{{\text {q-var}}}}. \end{aligned}$$

Plugging this into the continuity estimate of Theorem 6.2.1 in [25] we get

$$\begin{aligned} \rho _{{\text {p-var}}}( Y, \bar{Y} ) \le C ||Z - \bar{Z}||_{{{\text {q-var}}}} \end{aligned}$$

as desired. $\square$

Remark 34

The following Lemma as well as the proof the Pontryagin principle use the concept of controlled rough paths in the sense of Gubinelli [16]. Even though it is usually set up in Hölder spaces, the modification to variation spaces poses no problem (see [29, Section4.1]). In particular we define for a path Y controlled by $\eta$ with derivative $Y'$

$$\begin{aligned} ||Y,Y'||_{\eta ,{{\text {p-var}}}} := ||Y'||_{{{\text {p-var}}}} + ||R||_{ {\text {p/2-var}} }, \end{aligned}$$

where $R_{s,t} := Y_{s,t} - Y'_s \eta _{s,t}$.

Lemma 35

Let ${\mathbf {\eta }}$ be a geometric p-variation rough path, $p \in (2,3)$. Let M be controlled by $\eta$ and for $\varepsilon \in (0,1]$ let $(A^\varepsilon , (A')^\varepsilon )$ be controlled by $\eta$ with

$$\begin{aligned} ||A^\varepsilon , (A')^\varepsilon ||_{\eta ,{{\text {p-var}}}} = O(\varepsilon ). \end{aligned}$$

Let $X^\varepsilon$ solve

$$\begin{aligned} dX^\varepsilon _t&= dA^\varepsilon _t + A_t d{\mathbf {\eta }}_t \nonumber \\ X^\varepsilon _0&= 0. \end{aligned}$$

(4.19)

Then

$$\begin{aligned} \rho _{{\text {p-var}}}( X^\varepsilon , 0 ) = O(\varepsilon ). \end{aligned}$$

Proof

For every $\varepsilon$ there exists a unique solution solution to (4.19), and it is uniformly bounded in $\varepsilon \in (0,1]$.

This follows from [15, Theorem10.53], by considering $(A^\varepsilon , {\mathbf {\eta }})$ as a joint rough path.

We can hence, uniformly in $\varepsilon$, replace the linear vector field by a bounded one.

We then get from [14, Theorem8.5]

$$\begin{aligned} \rho _{{\text {p-var}}}(X^\varepsilon , X^0 ) \le c\, \rho _{{\text {p-var}}}( (A^\varepsilon , {\mathbf {\eta }}), (0, {\mathbf {\eta }}) ). \end{aligned}$$

where $X^0$ denotes the unique solution to (4.19) with $A^\varepsilon$ replaced by the constant 0-path.

Obviously $X^0 \equiv 0$ and it easy to see that

$$\begin{aligned} \rho _{{\text {p-var}}}( (A^\varepsilon , {\mathbf {\eta }}), (0, {\mathbf {\eta }}) ) = O(\varepsilon ), \end{aligned}$$

which yields the desired result. $\square$