EI、Scopus 收录
中文核心期刊
Quick Search Issue Search Advanced Search
Volume 53 Issue 7
Jul.  2021
Turn off MathJax
Article Contents
Abstract
References
Jiang Zichao, Jiang Junyang, Yao Qinghe, Yang Gengchao. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921. DOI: 10.6052/0459-1879-21-040
Citation: Jiang Zichao, Jiang Junyang, Yao Qinghe, Yang Gengchao. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921. DOI: 10.6052/0459-1879-21-040
shu

A FAST SOLVER BASED ON DEEP NEURAL NETWORK FOR DIFFERENCE EQUATION

  • School of Aeronautics and Astronautics, Guangzhou 510006, China

  • Received Date: January 22, 2021
  • Accepted Date: June 20, 2021
  • Available Online: June 20, 2021
    Graphical Abstract
    • Abstract
  • In recent years, artificial neural networks (ANNs), especially deep neural networks (DNNs), have become a promising new approach in the field of numerical computation due to their high computational efficiency on heterogeneous platforms and their ability to fit high-dimensional complex systems. In the process of numerically solving the partial differential equations, the large-scale linear equations are usually the most time-consuming problems; therefore, utilizing the neural network methods to solve linear equations has become a promising new idea. However, the direct prediction of deep neural networks still has obvious shortcomings in numerical accuracy, which becomes one of the bottlenecks for its application in the field of numerical computation. To break this limitation, a solving algorithm combining Residual network architecture and correction iteration method is proposed in this paper. In this paper, a deep neural network-based method for solving linear equations is proposed to accelerate the solving process of partial differential equations on heterogeneous platforms. Specifically, Residual network resolves the problems of network degradation and gradient vanishing of deep network models, reducing the loss of the network to 1/5000 of the classical network model; the correction iteration method iteratively reduce the error of the prediction solution based on the same network model, and the residual of the prediction solution has been decreased to 10−5 times of that before the iteration. To verify the effectiveness and universality of the proposed method, we combined the method with the finite difference method to solve the heat conduction equation and the Burger’s equation. Numerical results demonstrate that the algorithm has more than 10 times the acceleration effect for equations of size larger than 1000, and the numerical error is lower than the discrete error of the second-order difference scheme.
    • FullText(HTML)
    • References (42)
  • [1]
    Alshemali B, Kalita J. Improving the reliability of deep neural networks in NLP: A review. Knowledge-Based Systems, 2020, 191: 1-19
    [2]
    Zagoruyko S, Komodakis N. Deep compare: A study on using convolutional neural networks to compare image patches. Computer Vision and Image Understanding, 2017, 164: 38-55 doi: 10.1016/j.cviu.2017.10.007
    [3]
    刘村, 李元祥, 周拥军等. 基于卷积神经网络的视频图像超分辨率重建方法. 计算机应用研究, 2019, 036(4): 1256-1260+1274 (Liu Cun, Li Yuanxiang, Zhou Yongjun, et al. Video image super—resolution reconstruction method based on convolutional neural network. Application Research of Computers, 2019, 036(4): 1256-1260+1274 (in Chinese)
    [4]
    Lagaris IE, Likas A, Fotiadis DI. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 1998, 9(5): 987-1000 doi: 10.1109/72.712178
    [5]
    Ray D, Hesthaven JS. An artificial neural network as a troubled-cell indicator. Journal of Computational Physics, 2018, 367: 166-191 doi: 10.1016/j.jcp.2018.04.029
    [6]
    Chan S, Elsheikh AH. A machine learning approach for efficient uncertainty quantification using multiscale methods. Journal of Computational Physics, 2018, 354: 493-511 doi: 10.1016/j.jcp.2017.10.034
    [7]
    Wang Y, Cheung SW, Chung ET, et al. Deep multiscale model learning. Journal of Computational Physics, 2020, 406: 109071 doi: 10.1016/j.jcp.2019.109071
    [8]
    Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 1989, 2(4): 303-314 doi: 10.1007/BF02551274
    [9]
    Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Networks, 1989, 2(5): 359-366 doi: 10.1016/0893-6080(89)90020-8
    [10]
    Owhadi H. Bayesian numerical homogenization. Multiscale Modeling & Simulation, 2015, 13(3): 812-828
    [11]
    Raissi M, Perdikaris P, Karniadakis GE. Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 2017, 348: 683-693 doi: 10.1016/j.jcp.2017.07.050
    [12]
    Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378: 686-707 doi: 10.1016/j.jcp.2018.10.045
    [13]
    谢晨月, 袁泽龙, 王建春等. 基于人工神经网络的湍流大涡模拟方法. 力学学报, 2021, 53(1): 1-16 (Xie Chenyue, Yuan Zelong, Wang Jianchun, et al. Artificial neural network-based subgrid-scale models for large-eddy simulation of turbulence. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(1): 1-16 (in Chinese)
    [14]
    Golub GH, Loan CFV. Matrix Computations, fourth ed. Baltimore: JHU Press, 2013
    [15]
    Barrett R, Berry M, Chan TF, et al. Templates for the solution of linear systems: Building blocks for iterative methods. Mathematics of Computation, 1995, 64(211): 1349-1352 doi: 10.2307/2153507
    [16]
    Fedkiw R, Stam J, Jensen HW. Visual simulation of smoke//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: Association for Computing Machinery, 2001
    [17]
    蒋子超, 江俊扬, 孙哲等. 面向大规模并行的全隐式托卡马克MHD数值模拟. 中山大学学报(自然科学版), doi: 10.13471/j.cnki.acta.snus.2020.06.13.2020B067

    Jiang Zichao, Jiang Junyang, Sun Zhe, et al. Large-scale parallel simulation of MHD in tokamak based on a fully implicit method. Acta Scientiarum Naturalium Universitatis Sunyatseni, doi: 10.13471/j.cnki.acta.snus.2020.06.13.2020B067 (in Chinese)
    [18]
    Wang L, Mendel JM. Matrix computations and equation solving using structured networks and training//29th IEEE Conference on Decision and Control, Honolulu. New York: IEEE, 2002. 1747-1750
    [19]
    Wang L, Mendel JM. Three-dimensional structured networks for matrix equation solving. IEEE Transactions on Computers, 1991, 40(12): 1337-1346 doi: 10.1109/12.106219
    [20]
    Liao W, Wang J, Wang J. A discrete-time recurrent neural network for solving systems of complex-valued linear equations//International Conference in Swarm Intelligence, Beijing. Heidelberg: Springer, Advances in Swarm Intelligence, 2010. 315-320
    [21]
    Polycarpou MM, Ioannou PA. A neural-type parallel algorithm for fast matrix inversion//5th International Parallel Processing Symposium, Anaheim. New York: IEEE, 1991. 108-113
    [22]
    Cichocki A, Unbehauen R. Neural networks for solving systems of linear equations and related problems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1992, 39(2): 124-138 doi: 10.1109/81.167018
    [23]
    Zhou Z, Chen L, Wan L. Neural network algorithm for solving system of linear equations//2009 International Conference on Computational Intelligence and Natural Computing, Wuhan. New York: IEEE, 2009. 7-10.
    [24]
    Wu G, Wang J, Hootman J. A recurrent neural network for computing pseudoinverse matrices. Mathematical and Computer Modelling, 1994, 20(1): 13-21 doi: 10.1016/0895-7177(94)90215-1
    [25]
    Youshen X, Jun W, Hung DL. Recurrent neural networks for solving linear inequalities and equations. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1999, 46(4): 452-462 doi: 10.1109/81.754846
    [26]
    Takala J, Burian A, Ylinen M. A comparison of recurrent neural networks for inverting matrices//Signals, Circuits and Systems, 2003, IASI. New York: IEEE, International Symposium on 2003. 545-548
    [27]
    Li Z, Cheng H, Guo H. General recurrent neural network for solving generalized linear matrix equation. Complexity, 2017, 2017: 1-7
    [28]
    Meng X, Karniadakis GE. A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems. Journal of Computational Physics, 2020, 401: 109020 doi: 10.1016/j.jcp.2019.109020
    [29]
    Brunton SL, Noack BR, Koumoutsakos P. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 2020, 52: 477-508 doi: 10.1146/annurev-fluid-010719-060214
    [30]
    Xiao X, Zhou Y, Wang H, et al. A novel CNN-based poisson solver for fluid simulation. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(3): 1454-1465 doi: 10.1109/TVCG.2018.2873375
    [31]
    He KM, Zhang XY, Ren SQ, et al. Delving deep into rectifiers: surpassing human-level performance on image net classification//IEEE International Conference on Computer Vision (ICCV), Santiago. New York: IEEE, 2015. 1026-1034
    [32]
    Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 1994, 5(2): 157-166 doi: 10.1109/72.279181
    [33]
    Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks//13th International Conference on Artificial Intelligence and Statistics, Sardinia. Italy: Microtome Publishing, 2010. 249-256
    [34]
    He K, Sun J. Convolutional neural networks at constrained time cost//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston. New York: IEEE, 2015. 5353-5360
    [35]
    Srivastava RK, Greff K, Schmidhuber J. Highway networks. 2015, arXiv: 1505.00387
    [36]
    Qin T, Wu K, Xiu D. Data driven governing equations approximation using deep neural networks. Journal of Computational Physics, 2019, 395: 620-635 doi: 10.1016/j.jcp.2019.06.042
    [37]
    He KM, Zhang XY, Ren SQ, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas. New York: IEEE, 2016. 770-778
    [38]
    Chang B, Meng L, Haber E, et al. Multi-level residual networks from dynamical systems view. 2017, arXiv: 1710.10348
    [39]
    Abadi M, Agarwal A, Barham P, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv e-prints, 2016, arXiv: 1603.04467
    [40]
    Whitham GB. Linear and Nonlinear Waves. New Jersey: John Wiley & Sons, 2011
    [41]
    Bateman H. Some recent researches on the motion of fluids. Monthly Weather Review, 1915, 43: 163 doi: 10.1175/1520-0493(1915)43<163:SRROTM>2.0.CO;2
    [42]
    Piscopo ML, Spannowsky M, Waite P. Solving differential equations with neural networks: Applications to the calculation of cosmological phase transitions. Physical Review D, 2019, 100(1): 12
    • Related Articles

  • Cited by

    Periodical cited type(8)

    1. 李晓蔚，赵小亮，王剑. 异氰酸酯化氧化石墨烯/水性聚氨酯的合成及性能. 印染. 2023(02): 42-45 .
    2. 刘文娟. 氧化石墨烯改性混凝土的制备及力学性能和抗冻性能的研究. 功能材料. 2022(08): 8159-8164 .
    3. 王利军，朱艺，林本海. 氧化石墨烯-粉煤灰改性高水胶比注浆材料性能研究. 广东建材. 2022(09): 54-59 .
    4. 张金涛，文聘，晏石林，何大平. 多层氧化石墨烯的三维弹性常数及氧化度对力学性能的影响. 固体力学学报. 2022(05): 625-635 .
    5. 高扬. 原子力显微镜在二维材料力学性能测试中的应用综述. 力学学报. 2021(04): 929-943 . 本站查看
    6. 李东波，董仓，芦苇，苏梦龙. 基于矿渣粉与氧化石墨烯协同效应的水泥基材料冻融损伤力学性能研究. 应用力学学报. 2021(04): 1431-1440 .
    7. 刘俊，董淑宏，赵军华. 氢键对氧化石墨烯复合结构层间剪切行为影响研究. 固体力学学报. 2021(04): 443-454 .
    8. 李东波，张鸿驰，刘春燕，赵冬，芦苇. 氧化石墨烯与粉煤灰增强水泥基材料的协同机理及其抗压性能尺寸效应. 应用力学学报. 2021(05): 1869-1876 .

    Other cited types(13)

Catalog

    Get Citation
    PDF
    XML

    Article Metrics

    Article views PDF downloads Cited by(21)
    Related

    Download Center

    Author Center

    Copyright © Editorial Office of the Chinese Journal of Mechanics   京ICP备05039218号-1

    Address：15 Beishihuan Xi Lu, Haidian District, Beijing, China   China Pos：100190

    Tel：010-62536271

    Email：lxxb@cstam.org.cn

    微信公众号
    RSS

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return