EI、Scopus 收录
中文核心期刊
Quick Search Issue Search Advanced Search
Volume 53 Issue 7
Jul.  2021
Turn off MathJax
Article Contents
Abstract
References
Related Articles
Jiang Zichao, Jiang Junyang, Yao Qinghe, Yang Gengchao. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921. DOI: 10.6052/0459-1879-21-040
Citation: Jiang Zichao, Jiang Junyang, Yao Qinghe, Yang Gengchao. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921. DOI: 10.6052/0459-1879-21-040
shu

A FAST SOLVER BASED ON DEEP NEURAL NETWORK FOR DIFFERENCE EQUATION

  • School of Aeronautics and Astronautics, Guangzhou 510006, China

  • Received Date: January 22, 2021
  • Accepted Date: June 20, 2021
  • Available Online: June 20, 2021
    Graphical Abstract
    • Abstract
  • In recent years, artificial neural networks (ANNs), especially deep neural networks (DNNs), have become a promising new approach in the field of numerical computation due to their high computational efficiency on heterogeneous platforms and their ability to fit high-dimensional complex systems. In the process of numerically solving the partial differential equations, the large-scale linear equations are usually the most time-consuming problems; therefore, utilizing the neural network methods to solve linear equations has become a promising new idea. However, the direct prediction of deep neural networks still has obvious shortcomings in numerical accuracy, which becomes one of the bottlenecks for its application in the field of numerical computation. To break this limitation, a solving algorithm combining Residual network architecture and correction iteration method is proposed in this paper. In this paper, a deep neural network-based method for solving linear equations is proposed to accelerate the solving process of partial differential equations on heterogeneous platforms. Specifically, Residual network resolves the problems of network degradation and gradient vanishing of deep network models, reducing the loss of the network to 1/5000 of the classical network model; the correction iteration method iteratively reduce the error of the prediction solution based on the same network model, and the residual of the prediction solution has been decreased to 10−5 times of that before the iteration. To verify the effectiveness and universality of the proposed method, we combined the method with the finite difference method to solve the heat conduction equation and the Burger’s equation. Numerical results demonstrate that the algorithm has more than 10 times the acceleration effect for equations of size larger than 1000, and the numerical error is lower than the discrete error of the second-order difference scheme.
    • FullText(HTML)
    • References (42)
  • [1]
    Alshemali B, Kalita J. Improving the reliability of deep neural networks in NLP: A review. Knowledge-Based Systems, 2020, 191: 1-19
    [2]
    Zagoruyko S, Komodakis N. Deep compare: A study on using convolutional neural networks to compare image patches. Computer Vision and Image Understanding, 2017, 164: 38-55 doi: 10.1016/j.cviu.2017.10.007
    [3]
    刘村, 李元祥, 周拥军等. 基于卷积神经网络的视频图像超分辨率重建方法. 计算机应用研究, 2019, 036(4): 1256-1260+1274 (Liu Cun, Li Yuanxiang, Zhou Yongjun, et al. Video image super—resolution reconstruction method based on convolutional neural network. Application Research of Computers, 2019, 036(4): 1256-1260+1274 (in Chinese)
    [4]
    Lagaris IE, Likas A, Fotiadis DI. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 1998, 9(5): 987-1000 doi: 10.1109/72.712178
    [5]
    Ray D, Hesthaven JS. An artificial neural network as a troubled-cell indicator. Journal of Computational Physics, 2018, 367: 166-191 doi: 10.1016/j.jcp.2018.04.029
    [6]
    Chan S, Elsheikh AH. A machine learning approach for efficient uncertainty quantification using multiscale methods. Journal of Computational Physics, 2018, 354: 493-511 doi: 10.1016/j.jcp.2017.10.034
    [7]
    Wang Y, Cheung SW, Chung ET, et al. Deep multiscale model learning. Journal of Computational Physics, 2020, 406: 109071 doi: 10.1016/j.jcp.2019.109071
    [8]
    Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 1989, 2(4): 303-314 doi: 10.1007/BF02551274
    [9]
    Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Networks, 1989, 2(5): 359-366 doi: 10.1016/0893-6080(89)90020-8
    [10]
    Owhadi H. Bayesian numerical homogenization. Multiscale Modeling & Simulation, 2015, 13(3): 812-828
    [11]
    Raissi M, Perdikaris P, Karniadakis GE. Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 2017, 348: 683-693 doi: 10.1016/j.jcp.2017.07.050
    [12]
    Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378: 686-707 doi: 10.1016/j.jcp.2018.10.045
    [13]
    谢晨月, 袁泽龙, 王建春等. 基于人工神经网络的湍流大涡模拟方法. 力学学报, 2021, 53(1): 1-16 (Xie Chenyue, Yuan Zelong, Wang Jianchun, et al. Artificial neural network-based subgrid-scale models for large-eddy simulation of turbulence. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(1): 1-16 (in Chinese)
    [14]
    Golub GH, Loan CFV. Matrix Computations, fourth ed. Baltimore: JHU Press, 2013
    [15]
    Barrett R, Berry M, Chan TF, et al. Templates for the solution of linear systems: Building blocks for iterative methods. Mathematics of Computation, 1995, 64(211): 1349-1352 doi: 10.2307/2153507
    [16]
    Fedkiw R, Stam J, Jensen HW. Visual simulation of smoke//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: Association for Computing Machinery, 2001
    [17]
    蒋子超, 江俊扬, 孙哲等. 面向大规模并行的全隐式托卡马克MHD数值模拟. 中山大学学报(自然科学版), doi: 10.13471/j.cnki.acta.snus.2020.06.13.2020B067

    Jiang Zichao, Jiang Junyang, Sun Zhe, et al. Large-scale parallel simulation of MHD in tokamak based on a fully implicit method. Acta Scientiarum Naturalium Universitatis Sunyatseni, doi: 10.13471/j.cnki.acta.snus.2020.06.13.2020B067 (in Chinese)
    [18]
    Wang L, Mendel JM. Matrix computations and equation solving using structured networks and training//29th IEEE Conference on Decision and Control, Honolulu. New York: IEEE, 2002. 1747-1750
    [19]
    Wang L, Mendel JM. Three-dimensional structured networks for matrix equation solving. IEEE Transactions on Computers, 1991, 40(12): 1337-1346 doi: 10.1109/12.106219
    [20]
    Liao W, Wang J, Wang J. A discrete-time recurrent neural network for solving systems of complex-valued linear equations//International Conference in Swarm Intelligence, Beijing. Heidelberg: Springer, Advances in Swarm Intelligence, 2010. 315-320
    [21]
    Polycarpou MM, Ioannou PA. A neural-type parallel algorithm for fast matrix inversion//5th International Parallel Processing Symposium, Anaheim. New York: IEEE, 1991. 108-113
    [22]
    Cichocki A, Unbehauen R. Neural networks for solving systems of linear equations and related problems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1992, 39(2): 124-138 doi: 10.1109/81.167018
    [23]
    Zhou Z, Chen L, Wan L. Neural network algorithm for solving system of linear equations//2009 International Conference on Computational Intelligence and Natural Computing, Wuhan. New York: IEEE, 2009. 7-10.
    [24]
    Wu G, Wang J, Hootman J. A recurrent neural network for computing pseudoinverse matrices. Mathematical and Computer Modelling, 1994, 20(1): 13-21 doi: 10.1016/0895-7177(94)90215-1
    [25]
    Youshen X, Jun W, Hung DL. Recurrent neural networks for solving linear inequalities and equations. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1999, 46(4): 452-462 doi: 10.1109/81.754846
    [26]
    Takala J, Burian A, Ylinen M. A comparison of recurrent neural networks for inverting matrices//Signals, Circuits and Systems, 2003, IASI. New York: IEEE, International Symposium on 2003. 545-548
    [27]
    Li Z, Cheng H, Guo H. General recurrent neural network for solving generalized linear matrix equation. Complexity, 2017, 2017: 1-7
    [28]
    Meng X, Karniadakis GE. A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems. Journal of Computational Physics, 2020, 401: 109020 doi: 10.1016/j.jcp.2019.109020
    [29]
    Brunton SL, Noack BR, Koumoutsakos P. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 2020, 52: 477-508 doi: 10.1146/annurev-fluid-010719-060214
    [30]
    Xiao X, Zhou Y, Wang H, et al. A novel CNN-based poisson solver for fluid simulation. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(3): 1454-1465 doi: 10.1109/TVCG.2018.2873375
    [31]
    He KM, Zhang XY, Ren SQ, et al. Delving deep into rectifiers: surpassing human-level performance on image net classification//IEEE International Conference on Computer Vision (ICCV), Santiago. New York: IEEE, 2015. 1026-1034
    [32]
    Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 1994, 5(2): 157-166 doi: 10.1109/72.279181
    [33]
    Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks//13th International Conference on Artificial Intelligence and Statistics, Sardinia. Italy: Microtome Publishing, 2010. 249-256
    [34]
    He K, Sun J. Convolutional neural networks at constrained time cost//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston. New York: IEEE, 2015. 5353-5360
    [35]
    Srivastava RK, Greff K, Schmidhuber J. Highway networks. 2015, arXiv: 1505.00387
    [36]
    Qin T, Wu K, Xiu D. Data driven governing equations approximation using deep neural networks. Journal of Computational Physics, 2019, 395: 620-635 doi: 10.1016/j.jcp.2019.06.042
    [37]
    He KM, Zhang XY, Ren SQ, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas. New York: IEEE, 2016. 770-778
    [38]
    Chang B, Meng L, Haber E, et al. Multi-level residual networks from dynamical systems view. 2017, arXiv: 1710.10348
    [39]
    Abadi M, Agarwal A, Barham P, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv e-prints, 2016, arXiv: 1603.04467
    [40]
    Whitham GB. Linear and Nonlinear Waves. New Jersey: John Wiley & Sons, 2011
    [41]
    Bateman H. Some recent researches on the motion of fluids. Monthly Weather Review, 1915, 43: 163 doi: 10.1175/1520-0493(1915)43<163:SRROTM>2.0.CO;2
    [42]
    Piscopo ML, Spannowsky M, Waite P. Solving differential equations with neural networks: Applications to the calculation of cosmological phase transitions. Physical Review D, 2019, 100(1): 12
    • Related Articles

  • Related Articles

    [1]Wei Chang, Fan Yuchen, Zhou Yongqing, Liu Xin, Li Chi, Wang Heyang. SOLVING UNSTEADY PARTIAL DIFFERENTIAL EQUATIONS USING TIME-WEIGHTED PHYSICS-INFORMED NEURAL NETWORK[J]. Chinese Journal of Theoretical and Applied Mechanics, 2025, 57(3): 755-766. DOI: 10.6052/0459-1879-24-289
    [2]Wei Chang, Fan Yuchen, Zhou Yongqing, Zhang Chaoqun, Liu Xin, Wang Heyang. SELF-REGRESSIVE PHYSICS-INFORMED NEURAL NETWORK BASED ON RUNGE-KUTTA METHOD FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS[J]. Chinese Journal of Theoretical and Applied Mechanics, 2024, 56(8): 2482-2493. DOI: 10.6052/0459-1879-24-106
    [3]Wang Changtao, Dai Honghua, Zhang Zhe, Wang Xuechuan, Yue Xiaokui. PARALLEL ACCELERATED LOCAL VARIATIONAL ITERATION METHOD AND ITS APPLICATION IN ORBIT COMPUTATION[J]. Chinese Journal of Theoretical and Applied Mechanics, 2023, 55(4): 991-1003. DOI: 10.6052/0459-1879-22-592
    [4]Hu Zhenyu, Wang Zilu, Chen Jianqiang, Yuan Xianxu, Xiang Xinghao. PREDICTION OF CROSSFLOW TRANSITION BASED ON DEEP NEURAL NETWORKS[J]. Chinese Journal of Theoretical and Applied Mechanics, 2023, 55(1): 38-51. DOI: 10.6052/0459-1879-22-448
    [5]Zha Wenshu, Li Daolun, Shen Luhang, Zhang Wen, Liu Xuliang. REVIEW OF NEURAL NETWORK-BASED METHODS FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS[J]. Chinese Journal of Theoretical and Applied Mechanics, 2022, 54(3): 543-556. DOI: 10.6052/0459-1879-21-617
    [6]Hou Shujuan, Liang Huiyan, Wang Quanzhong, Han Xu. STUDY ON NONLINEAR ELASTIC HOMOGENIZATION WITH ITERATIVE METHOD[J]. Chinese Journal of Theoretical and Applied Mechanics, 2018, 50(4): 837-846. DOI: 10.6052/0459-1879-18-039
    [7]Ma Xiuteng, Zhai Yanbo, Xie Shouyong. HHT METHOD WITH CONSTRAINTS VIOLATION CORRECTION IN THE INDEX 2 EQUATIONS OF MOTION FOR MULTIBODY SYSTEMS[J]. Chinese Journal of Theoretical and Applied Mechanics, 2017, 49(1): 175-181. DOI: 10.6052/0459-1879-16-275
    [8]Liu Wei, Zhang Laiping, He Xin, He Lixin, Zhang Hanxin. AN IMPLICIT ALGORITHM FOR DISCONTINUOUS GALERKIN METHOD BASED ON NEWTON/GAUSS-SEIDEL ITERATIONS[J]. Chinese Journal of Theoretical and Applied Mechanics, 2012, (4): 792-796. DOI: 10.6052/0459-1879-11-352
    [9]An iterative combined approximation approach for structural static reanalysis of topological modifications[J]. Chinese Journal of Theoretical and Applied Mechanics, 2004, 36(5): 611-616. DOI: 10.6052/0459-1879-2004-5-2003-194
    [10]A NEW VIOLATION CORRECTION METHOD FOR CONSTRAINT MULTIBODY SYSTEMS 1)[J]. Chinese Journal of Theoretical and Applied Mechanics, 1998, 30(3): 300-306. DOI: 10.6052/0459-1879-1998-3-1995-130

  • Cited by

    Periodical cited type(0)

    Other cited types(2)

Catalog

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1767) PDF downloads (380) Cited by(2)
    Related

    Download Center

    Author Center

    Copyright © Editorial Office of the Chinese Journal of Mechanics   京ICP备05039218号-1

    Address：15 Beishihuan Xi Lu, Haidian District, Beijing, China   China Pos：100190

    Tel：010-62536271

    Email：lxxb@cstam.org.cn

    微信公众号
    RSS

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return