Skip to main content
SpringerLink
Log in

Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations

  • Original Paper
  • Published:
Journal of Nonlinear Science Aims and scope Submit manuscript

Abstract

High-dimensional partial differential equations (PDEs) appear in a number of models from the financial industry, such as in derivative pricing models, credit valuation adjustment models, or portfolio optimization models. The PDEs in such applications are high-dimensional as the dimension corresponds to the number of financial assets in a portfolio. Moreover, such PDEs are often fully nonlinear due to the need to incorporate certain nonlinear phenomena in the model such as default risks, transaction costs, volatility uncertainty (Knightian uncertainty), or trading constraints in the model. Such high-dimensional fully nonlinear PDEs are exceedingly difficult to solve as the computational effort for standard approximation methods grows exponentially with the dimension. In this work, we propose a new method for solving high-dimensional fully nonlinear second-order PDEs. Our method can in particular be used to sample from high-dimensional nonlinear expectations. The method is based on (1) a connection between fully nonlinear second-order PDEs and second-order backward stochastic differential equations (2BSDEs), (2) a merged formulation of the PDE and the 2BSDE problem, (3) a temporal forward discretization of the 2BSDE and a spatial approximation via deep neural nets, and (4) a stochastic gradient descent-type optimization procedure. Numerical results obtained using TensorFlow in Python illustrate the efficiency and the accuracy of the method in the cases of a 100-dimensional Black–Scholes–Barenblatt equation, a 100-dimensional Hamilton–Jacobi–Bellman equation, and a nonlinear expectation of a 100-dimensional G-Brownian motion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Amadori, A.L.: Nonlinear integro-differential evolution problems arising in option pricing: a viscosity solutions approach. Differ. Integral Equ. 16(7), 787–811 (2003)

    MathSciNet  MATH  Google Scholar 

  • Avellaneda, M., Arnon, L., Parás, A.: Pricing and hedging derivative securities in markets with uncertain volatilities. Appl. Math. Finance 2, 73–88 (1995)

    Article  Google Scholar 

  • Bally, V., Pagès, G.: A quantization algorithm for solving multi-dimensional discrete-time optimal stopping problems. Bernoulli 9(6), 1003–1049 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39(3), 930–945 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  • Bayraktar, E., Young, V.: Pricing options in incomplete equity markets via the instantaneous Sharpe ratio. Ann. Finance 4(4), 399–429 (2008)

    Article  MATH  Google Scholar 

  • Bayraktar, E., Milevsky, M.A., Promislow, S.D., Young, V.R.: Valuation of mortality risk via the instantaneous Sharpe ratio: applications to life annuities. J. Econ. Dyn. Control 33(3), 676–691 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Bender, C., Denk, R.: A forward scheme for backward SDEs. Stoch. Process. Appl. 117(12), 1793–1812 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Bender, C., Schweizer, N., Zhuo, J.: A primal-dual algorithm for BSDEs. Math. Finance 27(3), 866–901 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  • Bergman, Y.Z.: Option pricing with differential interest rates. Rev. Financ. Stud. 8(2), 475–500 (1995)

    Article  Google Scholar 

  • Bismut, J.-M.: Conjugate convex functions in optimal stochastic control. J. Math. Anal. Appl. 44, 384–404 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  • Bouchard, B., Elie, R., Touzi, N.: Discrete-time approximation of BSDEs and probabilistic schemes for fully nonlinear PDEs. In: Advanced financial modelling, vol. 8 of Radon Series on Computational and Applied Mathematics, Walter de Gruyter, Berlin, pp. 91–124 (2009)

  • Bouchard, B.: Lecture notes on BSDEs: main existence and stability results. Ph.D. thesis, CEREMADE-Centre de Recherches en MAthématiques de la DEcision (2015)

  • Bouchard, B., Touzi, N.: Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations. Stoch. Process. Appl. 111(2), 175–206 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Briand, P., Labart, C.: Simulation of BSDEs by Wiener chaos expansion. Ann. Appl. Probab. 24(3), 1129–1171 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Cai, Z.: Approximating quantum many-body wave-functions using artificial neural networks (2017). arXiv:1704.05148

  • Carleo, G., Troyer, M.: Solving the quantum many-body problem with artificial neural networks. Science 355(6325), 602–606 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Chang, D., Liu, H., Xiong, J.: A branching particle system approximation for a class of FBSDEs. Probab. Uncertain. Quant. Risk 1, 9 (2016). 34

    Article  MathSciNet  Google Scholar 

  • Chassagneux, J.-F.: Linear multistep schemes for BSDEs. SIAM J. Numer. Anal. 52(6), 2815–2836 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Chassagneux, J.-F., Crisan, D.: Runge-Kutta schemes for backward stochastic differential equations. Ann. Appl. Probab. 24(2), 679–720 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Chassagneux, J.-F., Richou, A.: Numerical stability analysis of the Euler scheme for BSDEs. SIAM J. Numer. Anal. 53(2), 1172–1193 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Chassagneux, J.-F., Richou, A.: Numerical simulation of quadratic BSDEs. Ann. Appl. Probab. 26(1), 262–304 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Cheridito, P., Soner, H.M., Touzi, N., Victoir, N.: Second-order backward stochastic differential equations and fully nonlinear parabolic PDEs. Comm. Pure Appl. Math. 60(7), 1081–1110 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Chiaramonte, M., Kiener, M.: Solving differential equations using neural networks. Machine Learning Project (2013)

  • Crépey, S., Gerboud, R., Grbac, Z., Ngor, N.: Counterparty risk and funding: the four wings of the TVA. Int. J. Theor. Appl. Finance 16(2), 1350006 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Crisan, D., Manolarakis, K.: Probabilistic methods for semilinear partial differential equations. Applications to finance. M2AN Math. Model. Numer. Anal. 44 44(5), 1107–1133 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Crisan, D., Manolarakis, K.: Solving backward stochastic differential equations using the cubature method: application to nonlinear pricing. SIAM J. Financ. Math. 3(1), 534–571 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Crisan, D., Manolarakis, K.: Second order discretization of backward SDEs and simulation with the cubature method. Ann. Appl. Probab. 24(2), 652–678 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Crisan, D., Manolarakis, K., Touzi, N.: On the Monte Carlo simulation of BSDEs: an improvement on the Malliavin weights. Stoch. Process. Appl. 120(7), 1133–1158 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2(4), 303–314 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Darbon, J., Osher, S.: Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. Res. Math. Sci. 3, 19 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Dehghan, M., Nourian, M., Menhaj, M.B.: Numerical solution of Helmholtz equation by the modified Hopfield finite difference techniques. Numer. Methods Partial Differ. Equ. 25(3), 637–656 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Delarue, F., Menozzi, S.: A forward–backward stochastic algorithm for quasi-linear PDEs. Ann. Appl. Probab. 16(1), 140–184 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Douglas Jr., J., Ma, J., Protter, P.: Numerical methods for forward–backward stochastic differential equations. Ann. Appl. Probab. 6(3), 940–968 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  • E, W., Hutzenthaler, M., Jentzen, A., Kruse, T.: Linear scaling algorithms for solving high-dimensional nonlinear parabolic differential equations. (2017a). arXiv:1607.03295

  • E, W., Hutzenthaler, M., Jentzen, A., Kruse, T.: On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations. (2017b). arXiv:1708.03223

  • E, W., Han, J., Jentzen, A.: Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat. 5(4), 349–380 (2017c)

  • Ekren, I., Muhle-Karbe, J.: Portfolio choice with small temporary and transient price impact (2017). arXiv:1705.00672

  • El Karoui, N., Peng, S., Quenez, M.C.: Backward stochastic differential equations in finance. Math. Finance 7(1), 1–71 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  • Fahim, A., Touzi, N., Warin, X.: A probabilistic numerical method for fully nonlinear parabolic PDEs. Ann. Appl. Probab. 21(4), 1322–1364 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Forsyth, P.A., Vetzal, K.R.: Implicit solution of uncertain volatility/transaction cost option pricing models with discretely observed barriers. Appl. Numer. Math. 36(4), 427–445 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Fu, Y., Zhao, W., Zhou, T.: Efficient spectral sparse grid approximations for solving multi-dimensional forward backward SDEs. Discrete Contin. Dyn. Syst. Ser. B 22(9), 3439–3458 (2017)

    MathSciNet  MATH  Google Scholar 

  • Geiss, S., Ylinen, J.: Decoupling on the Wiener space, related Besov spaces, and applications to BSDEs. (2014). arXiv:1409.5322

  • Geiss, C., Labart, C.: Simulation of BSDEs with jumps by Wiener chaos expansion. Stoch. Process. Appl. 126(7), 2123–2162 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (Fort Lauderdale, FL, USA, 11–13 Apr 2011), G. Gordon, D. Dunson, and M. Dudk, Eds., vol. 15 of Proceedings of Machine Learning Research, PMLR, pp. 315–323

  • Gobet, E., Lemor, J.-P.: Numerical simulation of BSDEs using empirical regression methods: theory and practice. (2008). arXiv:0806.4447

  • Gobet, E., Labart, C.: Solving BSDE with adaptive control variate. SIAM J. Numer. Anal. 48(1), 257–277 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Gobet, E., Turkedjiev, P.: Approximation of backward stochastic differential equations using Malliavin weights and least-squares regression. Bernoulli 22(1), 530–562 (2016a)

    Article  MathSciNet  MATH  Google Scholar 

  • Gobet, E., Turkedjiev, P.: Linear regression MDP scheme for discrete backward stochastic differential equations under general conditions. Math. Comp. 85(299), 1359–1391 (2016b)

    Article  MathSciNet  MATH  Google Scholar 

  • Gobet, E., Lemor, J.-P., Warin, X.: A regression-based Monte Carlo method to solve backward stochastic differential equations. Ann. Appl. Probab. 15(3), 2172–2202 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Gobet, E., López-Salas, J.G., Turkedjiev, P., Vázquez, C.: Stratified regression Monte-Carlo scheme for semilinear PDEs and BSDEs with large scale parallelization on GPUs. SIAM J. Sci. Comput. 38(6), C652–C677 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Grohs, P., Hornung, F., Jentzen, A., von Wurstemberger, P.: A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black–Scholes partial differential equations (2018)

  • Guo, W., Zhang, J., Zhuo, J.: A monotone scheme for high-dimensional fully nonlinear PDEs. Ann. Appl. Probab. 25(3), 1540–1580 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Guyon, J., Henry-Labordère, P.: The uncertain volatility model: a Monte Carlo approach. J. Comput. Finance 14(3), 37–61 (2011)

    Article  Google Scholar 

  • Han, J., Jentzen, A., E, W.: Overcoming the curse of dimensionality: solving high-dimensional partial differential equations using deep learning (2017). arXiv:1707.02568

  • Han, J., E, W.: Deep learning approximation for stochastic control problems (2016). arXiv:1611.07422

  • Henry-Labordère, P., Oudjane, N., Tan, X., Touzi, N., Warin, X.: Branching diffusion representation of semilinear PDEs and Monte Carlo approximation (2016). arXiv:1603.01727

  • Henry-Labordère, P.: Counterparty risk valuation: a marked branching diffusion approach (2012). arXiv:1203.2369

  • Henry-Labordère, P., Tan, X., Touzi, N.: A numerical algorithm for a class of BSDEs via the branching process. Stoch. Process. Appl. 124(2), 1112–1140 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Hinton, G., Deng, L., Yu, D., Dahl, G.E., Mohamed, A-r, Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T.N., Kingsbury, B.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29(6), 82–97 (2012)

    Article  Google Scholar 

  • Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)

    Article  MATH  Google Scholar 

  • Huijskens, T.P., Ruijter, M.J., Oosterlee, C.W.: Efficient numerical Fourier methods for coupled forward–backward SDEs. J. Comput. Appl. Math. 296, 593–612 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of The 32nd International Conference on Machine Learning (ICML) (2015)

  • Jentzen, A., Kuckuck, B., Neufeld, A., von Wurstemberger, P.: Strong error analysis for stochastic gradient descent optimization algorithms (2018). arXiv:1801.09324

  • Karatzas, I., Shreve, S.E.: Brownian motion and stochastic calculus, 2nd edn, vol. 113 of Graduate Texts in Mathematics. Springer, New York, (1991)

  • Khoo, Y., Lu, J., Ying, L.: Solving parametric PDE problems with artificial neural networks (2017). arXiv:1707.03351

  • Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)

  • Kloeden, P.E., Platen, E.: Numerical Solution of Stochastic Differential Equations, vol. 23 of Applications of Mathematics (New York). Springer, Berlin (1992)

  • Kong, T., Zhao, W., Zhou, T.: Probabilistic high order numerical schemes for fully nonlinear parabolic PDEs. Commun. Comput. Phys. 18(5), 1482–1503 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  • Labart, C., Lelong, J.: A parallel algorithm for solving BSDEs. Monte Carlo Methods Appl. 19(1), 11–39 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw. 9(5), 987–1000 (1998)

    Article  Google Scholar 

  • Laurent, J.-P., Amzelek, P., Bonnaud, J.: An overview of the valuation of collateralized derivative contracts. Rev. Deriv. Res. 17(3), 261–286 (2014)

    Article  MATH  Google Scholar 

  • LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  • LeCun, Y., Bottou, L., Orr, G.B., Müller, K.R.: Efficient BackProp, pp. 9–50. Springer, Berlin, Heidelberg (1998)

    Google Scholar 

  • LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  • Lee, H., Kang, I.S.: Neural algorithm for solving differential equations. J. Comput. Phys. 91(1), 110–131 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  • Leland, H.E.: Option pricing and replication with transactions costs. J. Finance 40(5), 1283–1301 (1985)

    Article  Google Scholar 

  • Lemor, J.-P., Gobet, E., Warin, X.: Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations. Bernoulli 12(5), 889–916 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Lionnet, A., dos Reis, G., Szpruch, L.: Time discretization of FBSDE with polynomial growth drivers and reaction-diffusion PDEs. Ann. Appl. Probab. 25(5), 2563–2625 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Ma, J., Yong, J.: Forward–backward stochastic differential equations and their applications, vol. 1702 of Lecture Notes in Mathematics. Springer, Berlin (1999)

  • Ma, J., Protter, P., Yong, J.M.: Solving forward-backward stochastic differential equations explicitly—a four step scheme. Probab. Theory Relat. Fields 98(3), 339–359 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  • Ma, J., Protter, P., San Martín, J., Torres, S.: Numerical method for backward stochastic differential equations. Ann. Appl. Probab. 12(1), 302–316 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Maruyama, G.: Continuous Markov processes and stochastic equations. Rend. Circ. Mat. Palermo (2) 4, 48–90 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  • McKean, H.P.: Application of Brownian motion to the equation of Kolmogorov–Petrovskii–Piskunov. Commun. Pure Appl. Math. 28(3), 323–331 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  • Meade Jr., A.J., Fernández, A.A.: The numerical solution of linear ordinary differential equations by feedforward neural networks. Math. Comput. Model. 19(12), 1–25 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  • Mehrkanoon, S., Suykens, J.A.: Learning solutions to partial differential equations using LS-SVM. Neurocomputing 159, 105–116 (2015)

    Article  Google Scholar 

  • Milstein, G.N.: Approximate integration of stochastic differential equations. Teor. Verojatnost. i Primenen. 19, 583–588 (1974)

    MathSciNet  Google Scholar 

  • Milstein, G.N., Tretyakov, M.V.: Numerical algorithms for forward–backward stochastic differential equations. SIAM J. Sci. Comput. 28(2), 561–582 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Milstein, G.N., Tretyakov, M.V.: Discretization of forward–backward stochastic differential equations and related quasi-linear parabolic equations. IMA J. Numer. Anal. 27(1), 24–44 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Moreau, L., Muhle-Karbe, J., Soner, H.M.: Trading with small price impact. Math. Finance 27(2), 350–400 (2017)

    Article  MathSciNet  Google Scholar 

  • Øksendal, B.: Stochastic differential equations. An introduction with applications. Universitext. Springer, Berlin (1985)

    MATH  Google Scholar 

  • Pardoux, E., Peng, S.: Backward, stochastic differential equations and quasilinear parabolic partial differential equations. In: Stochastic Partial Differential Equations and their Applications (Charlotte, NC 1991), vol. 176 of Lecture Notes in Control and Information Sciences, pp. 200–217. Springer, Berlin (1992)

  • Pardoux, É., Peng, S.: Adapted solution of a backward stochastic differential equation. Syst. Control Lett. 14(1), 55–61 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  • Pardoux, E., Tang, S.: Forward-backward stochastic differential equations and quasilinear parabolic PDEs. Probab. Theory Relat. Fields 114(2), 123–150 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Peng, S.: \(G\)-expectation, \(G\)-Brownian motion and related stochastic calculus of Itô type. In: Stochastic Analysis and Applications, vol. 2 of Abel Symposium, pp. 541–567. Springer, Berlin (2007)

  • Peng, S.: Nonlinear expectations and stochastic calculus under uncertainty (2010). arXiv:1002.4546

  • Peng, S.: Nonlinear expectations, nonlinear evaluations and risk measures. In: Stochastic Methodsin Finance, vol. 1856 of Lecture Notes in Mathematics, pp. 165–253. Springer, Berlin (2004)

  • Peng, S.: Probabilistic interpretation for systems of quasilinear parabolic partial differential equations. Stoch. Stoch. Rep. 37(1–2), 61–74 (1991)

    MathSciNet  MATH  Google Scholar 

  • Peng, S.: Nonlinear expectations and nonlinear Markov chains. Chin. Ann. Math. Ser. B 26(2), 159–184 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Petersen, P., Voigtlaender, F.: Optimal approximation of piecewise smooth functions using deep relu neural networks (2017). arXiv:1709.05289

  • Pham, H.: Feynman–Kac representation of fully nonlinear PDEs and applications. Acta Math. Vietnam. 40(2), 255–269 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Possamaï, D., Mete Soner, H., Touzi, N.: Homogenization and asymptotics for small transaction costs: the multidimensional case. Commun. Partial Differ. Equ. 40(11), 2005–2046 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Ramuhalli, P., Udpa, L., Udpa, S.S.: Finite-element neural networks for solving differential equations. IEEE Trans. Neural Netw. 16(6), 1381–1392 (2005)

    Article  Google Scholar 

  • Rasulov, A., Raimova, G., Mascagni, M.: Monte Carlo solution of Cauchy problem for a nonlinear parabolic equation. Math. Comput. Simulation 80(6), 1118–1123 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Ruder, S.: An overview of gradient descent optimization algorithms (2016). arXiv:1609.04747

  • Ruijter, M.J., Oosterlee, C.W.: A Fourier cosine method for an efficient computation of solutions to BSDEs. SIAM J. Sci. Comput. 37(2), A859–A889 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Ruijter, M.J., Oosterlee, C.W.: Numerical Fourier method and second-order Taylor scheme for backward SDEs in finance. Appl. Numer. Math. 103, 1–26 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Technical report, California University of San Diego La Jolla, Institute for Cognitive Science (1985)

  • Ruszczynski, A., Yao, J.: A dual method for backward stochastic differential equations with application to risk valuation (2017). arXiv:1701.06234

  • Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    Article  Google Scholar 

  • Sirignano, J., Spiliopoulos, K.: DGM: a deep learning algorithm for solving partial differential equations (2017). arXiv:1708.07469

  • Skorohod, A.V.: Branching diffusion processes. Teor. Verojatnost. i Primenen. 9, 492–497 (1964)

    MathSciNet  Google Scholar 

  • Tadmor, E.: A review of numerical methods for nonlinear partial differential equations. Bull. Am. Math. Soc. (N.S.) 49(4), 507–554 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Thomée, V.: Galerkin finite element methods for parabolic problems, vol. 25 of Springer Series in Computational Mathematics. Springer, Berlin (1997)

  • Turkedjiev, P.: Two algorithms for the discrete time approximation of Markovian backward stochastic differential equations under local conditions. Electron. J. Probab. 20(50), 49 (2015)

    MathSciNet  MATH  Google Scholar 

  • von Petersdorff, T., Schwab, C.: Numerical solution of parabolic equations in high dimensions. M2AN Math. Model. Numer. Anal. 38(1), 93–127 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Warin, X.: Variations on branching methods for non linear PDEs (2017). arXiv:1701.07660

  • Watanabe, S.: On the branching process for Brownian particles with an absorbing boundary. J. Math. Kyoto Univ. 4, 385–398 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  • Windcliff, H., Wang, J., Forsyth, P.A., Vetzal, K.R.: Hedging with a correlated asset: solution of a nonlinear pricing PDE. J. Comput. Appl. Math. 200(1), 86–115 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, J.: A numerical scheme for BSDEs. Ann. Appl. Probab. 14(1), 459–488 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, G., Gunzburger, M., Zhao, W.: A sparse-grid method for multi-dimensional backward stochastic differential equations. J. Comput. Math. 31(3), 221–248 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Zhao, W., Zhou, T., Kong, T.: High order numerical schemes for second-order FBSDEs with applications to stochastic optimal control. Commun. Comput. Phys. 21(3), 808–834 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

Sebastian Becker and Jiequn Han are gratefully acknowledged for their helpful and inspiring comments regarding the implementation of the deep 2BSDE method.

Author information

Authors and Affiliations

  1. ETH Zurich, Zurich, Switzerland

    Christian Beck & Arnulf Jentzen

  2. Princeton University, Princeton, USA

    Weinan E

  3. Beijing Institute of Big Data Research, Beijing, China

    Weinan E

Authors
  1. Christian Beck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weinan E
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arnulf Jentzen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Beck.

Additional information

Communicated by Paul Newton.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Source Codes

Appendix: Source Codes

1.1 A.1: A Python Code for the Deep 2BSDE Method used in Subsection 4.1

The following Python code, Python code 1, is a simplified version of Python code 3 in Appendix A.3.

figure a
figure b
figure c
figure d
figure e

1.2 A.2: A Matlab Code for the Branching Diffusion Method used in Subsection 4.1

The following Matlab code is a slightly modified version of the Matlab code in (E et al. (2017c), Subsection 6.2).

figure f
figure g

1.3 A.3: A Python Code for the Deep 2BSDE Method used in Subsection 4.3

The following Python code is based on the Python code in (E et al. (2017c), Subsection 6.1).

figure h
figure i
figure j
figure k
figure l
figure m

1.4 A.4: A Matlab Code for the Classical Monte Carlo Method used in Subsection 4.4

The following Matlab code is a slightly modified version of the Matlab code in (E et al. (2017c), Subsection 6.3).

figure n

1.5 A.5: A Matlab Code for the Finite Differences Method used in Subsection 4.6

The following Matlab code is inspired by the Matlab code in (E et al. (2017b), MATLAB code 7 in Section 3).

figure o
figure p

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beck, C., E, W. & Jentzen, A. Machine Learning Approximation Algorithms for High-Dimensional Fully Nonlinear Partial Differential Equations and Second-order Backward Stochastic Differential Equations. J Nonlinear Sci 29, 1563–1619 (2019). https://doi.org/10.1007/s00332-018-9525-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00332-018-9525-3

Keywords

Search

Navigation