EI、Scopus 收录
中文核心期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于神经网络的差分方程快速求解方法

蒋子超 江俊扬 姚清河 杨耿超

蒋子超, 江俊扬, 姚清河, 杨耿超. 基于神经网络的差分方程快速求解方法. 力学学报, 2021, 53(7): 1912-1921 doi: 10.6052/0459-1879-21-040
引用本文: 蒋子超, 江俊扬, 姚清河, 杨耿超. 基于神经网络的差分方程快速求解方法. 力学学报, 2021, 53(7): 1912-1921 doi: 10.6052/0459-1879-21-040
Jiang Zichao, Jiang Junyang, Yao Qinghe, Yang Gengchao. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921 doi: 10.6052/0459-1879-21-040
Citation: Jiang Zichao, Jiang Junyang, Yao Qinghe, Yang Gengchao. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921 doi: 10.6052/0459-1879-21-040

基于神经网络的差分方程快速求解方法

doi: 10.6052/0459-1879-21-040
基金项目: 国家重点研发计划 (2018YFE9103900); 国家自然科学基金 (119723874)和广东省促进经济高质量发展专项资金 (GDOE[2019]A01)资助项目
详细信息
    作者简介:

    姚清河, 副教授, 主要研究方向: 计算流体力学、并行算法、偏微分方程数值解. E-mail: yaoqhe@mail.sysu.edu.cn

  • 中图分类号: TP183

A FAST SOLVER BASED ON DEEP NEURAL NETWORK FOR DIFFERENCE EQUATION

  • 摘要: 近年来, 人工神经网络(artificial neural networks, ANN), 尤其是深度神经网络(deep neural networks, DNN)由于其在异构平台上的高计算效率与对高维复杂系统的拟合能力而成为一种在数值计算领域具有广阔前景的新方法. 在偏微分方程数值求解中, 大规模线性方程组的求解通常是耗时最长的步骤之一, 因此, 采用神经网络方法求解线性方程组成为了一种值得期待的新思路. 但是, 深度神经网络的直接预测仍在数值精度方面仍有明显的不足, 成为其在数值计算领域广泛应用的瓶颈之一. 为打破这一限制, 本文提出了一种结合残差网络结构与校正迭代方法的求解算法. 其中, 残差网络结构解决了深度网络模型的网络退化与梯度消失等问题, 将网络的损失降低至经典网络模型的1/5000; 修正迭代的方法采用同一网络模型对预测解的反复校正, 将预测解的残差下降至迭代前的10−5倍. 为验证该方法的有效性与通用性, 本文将该方法与有限差分法结合, 对热传导方程与伯格方程进行了求解. 数值结果表明, 本文所提出的算法对于规模大于1000的方程组具有10倍以上的加速效果, 且数值误差低于二阶差分格式的离散误差.

     

  • 图  1  残差网络结构示意图

    Figure  1.  Schematic of the Res-Net

    图  2  求解不同大小线性方程时Res-Net与非Res-Net的损失函数对比

    Figure  2.  Comparison of the loss function for different sizes of the linear equations with Res-Net and no Res-Net

    图  3  迭代算法示意图

    Figure  3.  Schematic of the iteration algorithm

    图  4  (a)不同放大因子时残差随迭代的变化情况与(b)放大因子对收敛时残差的影响

    Figure  4.  (a) Variation of residuals with iterations for different amplification factors and (b) effect of amplification factors on convergence residuals

    图  5  算法求解器架构示意图

    Figure  5.  The schematic of the linear solver

    图  6  求解不同大小线性方程各种算法的求解时间

    Figure  6.  Solving times of DNN and conventional algorithms for different size of linear equations

    7  (a)二维热传导方程的数值求解结果与(b)相对经典方法的求解误差

    7.  (a) Numerical solution of the 2D heat conduction equation and (b) the error distribution

    图  7  (a)二维热传导方程的数值求解结果与(b)相对经典方法的求解误差 (续)

    Figure  7.  (a) Numerical solution of the 2D heat conduction equation and (b) the error distribution (continued)

    8  不同时刻下(a)一维Burges方程的数值求解结果与(b)求解误差

    8.  Numerical solution of (a) the 1D Burgers equation and (b) the error distribution

    图  8  不同时刻下(a)一维Burges方程的数值求解结果与(b)求解误差 (续)

    Figure  8.  Numerical solution of (a) the 1D Burgers equation and (b) the error distribution (continued)

  • [1] Alshemali B, Kalita J. Improving the reliability of deep neural networks in NLP: A review. Knowledge-Based Systems, 2020, 191: 1-19
    [2] Zagoruyko S, Komodakis N. Deep compare: A study on using convolutional neural networks to compare image patches. Computer Vision and Image Understanding, 2017, 164: 38-55 doi: 10.1016/j.cviu.2017.10.007
    [3] 刘村, 李元祥, 周拥军等. 基于卷积神经网络的视频图像超分辨率重建方法. 计算机应用研究, 2019, 036(4): 1256-1260+1274 (Liu Cun, Li Yuanxiang, Zhou Yongjun, et al. Video image super—resolution reconstruction method based on convolutional neural network. Application Research of Computers, 2019, 036(4): 1256-1260+1274 (in Chinese)
    [4] Lagaris IE, Likas A, Fotiadis DI. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 1998, 9(5): 987-1000 doi: 10.1109/72.712178
    [5] Ray D, Hesthaven JS. An artificial neural network as a troubled-cell indicator. Journal of Computational Physics, 2018, 367: 166-191 doi: 10.1016/j.jcp.2018.04.029
    [6] Chan S, Elsheikh AH. A machine learning approach for efficient uncertainty quantification using multiscale methods. Journal of Computational Physics, 2018, 354: 493-511 doi: 10.1016/j.jcp.2017.10.034
    [7] Wang Y, Cheung SW, Chung ET, et al. Deep multiscale model learning. Journal of Computational Physics, 2020, 406: 109071 doi: 10.1016/j.jcp.2019.109071
    [8] Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 1989, 2(4): 303-314 doi: 10.1007/BF02551274
    [9] Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Networks, 1989, 2(5): 359-366 doi: 10.1016/0893-6080(89)90020-8
    [10] Owhadi H. Bayesian numerical homogenization. Multiscale Modeling & Simulation, 2015, 13(3): 812-828
    [11] Raissi M, Perdikaris P, Karniadakis GE. Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 2017, 348: 683-693 doi: 10.1016/j.jcp.2017.07.050
    [12] Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378: 686-707 doi: 10.1016/j.jcp.2018.10.045
    [13] 谢晨月, 袁泽龙, 王建春等. 基于人工神经网络的湍流大涡模拟方法. 力学学报, 2021, 53(1): 1-16 (Xie Chenyue, Yuan Zelong, Wang Jianchun, et al. Artificial neural network-based subgrid-scale models for large-eddy simulation of turbulence. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(1): 1-16 (in Chinese)
    [14] Golub GH, Loan CFV. Matrix Computations, fourth ed. Baltimore: JHU Press, 2013
    [15] Barrett R, Berry M, Chan TF, et al. Templates for the solution of linear systems: Building blocks for iterative methods. Mathematics of Computation, 1995, 64(211): 1349-1352 doi: 10.2307/2153507
    [16] Fedkiw R, Stam J, Jensen HW. Visual simulation of smoke//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: Association for Computing Machinery, 2001
    [17] 蒋子超, 江俊扬, 孙哲等. 面向大规模并行的全隐式托卡马克MHD数值模拟. 中山大学学报(自然科学版), doi: 10.13471/j.cnki.acta.snus.2020.06.13.2020B067

    Jiang Zichao, Jiang Junyang, Sun Zhe, et al. Large-scale parallel simulation of MHD in tokamak based on a fully implicit method. Acta Scientiarum Naturalium Universitatis Sunyatseni, doi: 10.13471/j.cnki.acta.snus.2020.06.13.2020B067 (in Chinese)
    [18] Wang L, Mendel JM. Matrix computations and equation solving using structured networks and training//29th IEEE Conference on Decision and Control, Honolulu. New York: IEEE, 2002. 1747-1750
    [19] Wang L, Mendel JM. Three-dimensional structured networks for matrix equation solving. IEEE Transactions on Computers, 1991, 40(12): 1337-1346 doi: 10.1109/12.106219
    [20] Liao W, Wang J, Wang J. A discrete-time recurrent neural network for solving systems of complex-valued linear equations//International Conference in Swarm Intelligence, Beijing. Heidelberg: Springer, Advances in Swarm Intelligence, 2010. 315-320
    [21] Polycarpou MM, Ioannou PA. A neural-type parallel algorithm for fast matrix inversion//5th International Parallel Processing Symposium, Anaheim. New York: IEEE, 1991. 108-113
    [22] Cichocki A, Unbehauen R. Neural networks for solving systems of linear equations and related problems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1992, 39(2): 124-138 doi: 10.1109/81.167018
    [23] Zhou Z, Chen L, Wan L. Neural network algorithm for solving system of linear equations//2009 International Conference on Computational Intelligence and Natural Computing, Wuhan. New York: IEEE, 2009. 7-10.
    [24] Wu G, Wang J, Hootman J. A recurrent neural network for computing pseudoinverse matrices. Mathematical and Computer Modelling, 1994, 20(1): 13-21 doi: 10.1016/0895-7177(94)90215-1
    [25] Youshen X, Jun W, Hung DL. Recurrent neural networks for solving linear inequalities and equations. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1999, 46(4): 452-462 doi: 10.1109/81.754846
    [26] Takala J, Burian A, Ylinen M. A comparison of recurrent neural networks for inverting matrices//Signals, Circuits and Systems, 2003, IASI. New York: IEEE, International Symposium on 2003. 545-548
    [27] Li Z, Cheng H, Guo H. General recurrent neural network for solving generalized linear matrix equation. Complexity, 2017, 2017: 1-7
    [28] Meng X, Karniadakis GE. A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems. Journal of Computational Physics, 2020, 401: 109020 doi: 10.1016/j.jcp.2019.109020
    [29] Brunton SL, Noack BR, Koumoutsakos P. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 2020, 52: 477-508 doi: 10.1146/annurev-fluid-010719-060214
    [30] Xiao X, Zhou Y, Wang H, et al. A novel CNN-based poisson solver for fluid simulation. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(3): 1454-1465 doi: 10.1109/TVCG.2018.2873375
    [31] He KM, Zhang XY, Ren SQ, et al. Delving deep into rectifiers: surpassing human-level performance on image net classification//IEEE International Conference on Computer Vision (ICCV), Santiago. New York: IEEE, 2015. 1026-1034
    [32] Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 1994, 5(2): 157-166 doi: 10.1109/72.279181
    [33] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks//13th International Conference on Artificial Intelligence and Statistics, Sardinia. Italy: Microtome Publishing, 2010. 249-256
    [34] He K, Sun J. Convolutional neural networks at constrained time cost//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston. New York: IEEE, 2015. 5353-5360
    [35] Srivastava RK, Greff K, Schmidhuber J. Highway networks. 2015, arXiv: 1505.00387
    [36] Qin T, Wu K, Xiu D. Data driven governing equations approximation using deep neural networks. Journal of Computational Physics, 2019, 395: 620-635 doi: 10.1016/j.jcp.2019.06.042
    [37] He KM, Zhang XY, Ren SQ, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas. New York: IEEE, 2016. 770-778
    [38] Chang B, Meng L, Haber E, et al. Multi-level residual networks from dynamical systems view. 2017, arXiv: 1710.10348
    [39] Abadi M, Agarwal A, Barham P, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv e-prints, 2016, arXiv: 1603.04467
    [40] Whitham GB. Linear and Nonlinear Waves. New Jersey: John Wiley & Sons, 2011
    [41] Bateman H. Some recent researches on the motion of fluids. Monthly Weather Review, 1915, 43: 163 doi: 10.1175/1520-0493(1915)43<163:SRROTM>2.0.CO;2
    [42] Piscopo ML, Spannowsky M, Waite P. Solving differential equations with neural networks: Applications to the calculation of cosmological phase transitions. Physical Review D, 2019, 100(1): 12
  • 加载中
图(10)
计量
  • 文章访问数:  360
  • HTML全文浏览量:  96
  • PDF下载量:  118
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-01-23
  • 录用日期:  2021-06-21
  • 网络出版日期:  2021-06-21
  • 刊出日期:  2021-07-18

目录

    /

    返回文章
    返回