Improving the Efficiency of Deep Learning Algorithms for Solving High-Dimensional Differential Equations Using Recurrent and Residual Neural Network Architectures

High-dimensional partial differential equations (PDEs) form the foundation of complex process modeling in various scientific and engineering applications, including finance, physics, and optimal control. However, classical numerical methods are adversely affected by the curse of dimensionality making them inapplicable for large-scale problems. Recently, however, deep learning-based approaches have provided a new toolbox for these high-dimensional PDEs that includes methods such as Deep Backward Stochastic Differential Equation (Deep BSDE) method.
Our approach, drawing on a much more sophisticated deep learning backbone with the radical use of neural networks (in our case, Residual Neural Network and Long Short-Term Memory network (LSTM) integrated into the Deep BSDE setup. Despite its success in various applications, feedforward neural networks have some drawbacks as they are not designed to deal with issues like vanishing gradients or dependencies between timeframes.
We apply our method on several benchmark problems including the nonlinear Black–Scholes equation, the Hamilto--Jacobi--Bellman (HJB) equation and the Allen–Cahn equation in high-dimensional settings (up to 100 dimension). The ResNet/LSTM-based approach developed here achieves consistently better performance with lower relative errors than the baseline feedforward based approach, faster convergence rates and greater computational efficiency. Overall, the results highlight the promise of using advanced neural architectures in the Deep BSDE approach and provide a scalable and efficient method for high-dimensional PDEs. This study sets a foundation for data-driven computational approaches and represents novel opportunities in scientific machine learning.