SELF-REGRESSIVE PHYSICS-INFORMED NEURAL NETWORK BASED ON RUNGE-KUTTA METHOD FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS

Wei Chang; Fan Yuchen; Zhou Yongqing; Zhang Chaoqun; Liu Xin; Wang Heyang

doi:10.6052/0459-1879-24-106

Volume 56 Issue 8

Aug. 2024

Turn off MathJax

Article Contents

Abstract

References

Chinese Journal of Theoretical and Applied Mechanics > 2024 > 56(8): 2482-2493. > DOI: 10.6052/0459-1879-24-106 CSTR: 32045.14.0459-1879-24-106

Wei Chang, Fan Yuchen, Zhou Yongqing, Zhang Chaoqun, Liu Xin, Wang Heyang. Self-regressive physics-informed neural network based on Runge-Kutta method for solving partial differential equations. Chinese Journal of Theoretical and Applied Mechanics, 2024, 56(8): 2482-2493. DOI: 10.6052/0459-1879-24-106

Citation:

PDF (3357 KB)

SELF-REGRESSIVE PHYSICS-INFORMED NEURAL NETWORK BASED ON RUNGE-KUTTA METHOD FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS

*.
School of Mechanical Engineering, Tianjin University, Tianjin 300072, China
†.
Yantai Longyuan Power Technology Co., Ltd., Yantai 264006, Shandong, China

Received Date: March 03, 2024
Accepted Date: April 17, 2024
Available Online: April 17, 2024
Published Date: April 18, 2024

Graphical Abstract

Abstract

Abstract

Physics-informed neural networks discrete-time model (PINN-RK) is a product of combining deep learning techniques with Runge-Kutta method, which has excellent stability and high accuracy in solving partial differential equations. As an emerging computational tool, PINN-RK has been widely applied in solving various complex problems in scientific and engineering fields. However, due to the limitations of the Runge-Kutta algorithm itself, the PINN-RK model can only achieve single-step time prediction and has low computational efficiency. To achieve multi-step time prediction and improve the computational efficiency of the model, this paper proposes a novel self-regressive physics-informed neural networks model based on the Runge-Kutta method (SR-PINN-RK). The SR-PINN-RK model builds upon the PINN-RK model by incorporating a self-regressive time-advancing mechanism which allows the SR-PINN-RK model to learn the temporal dynamics of the partial differential equation more effectively, resulting in improved training performance and accuracy. In SR-PINN-RK model, except for the label data at the initial time given by the user, all other training labels are provided by the neural network model itself. The new PINN-RK model is a significant improvement over the PINN-RK model, with a much smaller number of training parameters and a significant boost in computational efficiency. This makes SR-PINN-RK model much faster and easier to train, while still maintaining the same level of accuracy. The SR-PINN-RK model uses a self-regressive mechanism to dynamically update the label data, which allows it to successfully achieve multi-step time prediction of partial differential equation solutions. This represents a remarkable improvement compared to the PINN-RK model, which is limited to single-step predictions. In order to verify the accuracy and computational efficiency of the SR-PINN-RK model, the Allen-Cahn equation and Burgers equation are solved using the SR-PINN-RK model and the predicted results are compared with the benchmark solutions in the literature. It is shown that the predicted solution of the SR-PINN-RK model is highly consistent with the benchmark solution qualitatively without significant differences. When it comes to the quantitative analysis, it can be observed that the maximum relative error in solving the Allen-Cahn and Burgers equations are both below 0.009, thereby exhibiting a remarkable level of precision in solving partial differential equations.

FullText(HTML)

References (35)

References

[1]	Yurtsever E, Lambert J, Carballo A, et al. A survey of autonomous driving: Common practices and emerging technologies. IEEE Access, 2020, 8: 58443-58469 doi: 10.1109/ACCESS.2020.2983149
[2]	He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916 doi: 10.1109/TPAMI.2015.2389824
[3]	Afkanpour A, Adeel S, Bassani H, et al. BERT for long documents: A case study of automated ICD coding. arXiv Preprint, arXiv: 2022, 221102519
[4]	Baker N, Alexander F, Bremer T, et al. Workshop report on basic research needs for scientific machine learning: Core technologies for artificial intelligence. USDOE Office of Science (SC), Washington DC, United States, 2019
[5]	Karniadakis GE, Kevrekidis IG, Lu L, et al. Physics-informed machine learning. Nature Reviews Physics, 2021, 3(6): 422-440 doi: 10.1038/s42254-021-00314-5
[6]	Brunton SL, Noack BR, Koumoutsakos P. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 2020, 52(1): 477-508 doi: 10.1146/annurev-fluid-010719-060214
[7]	Lu L, Jin P, Pang G, et al. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 2021, 3(3): 218-229 doi: 10.1038/s42256-021-00302-5
[8]	Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Networks, 1989, 2(5): 359-366 doi: 10.1016/0893-6080(89)90020-8
[9]	Lu L, Meng X, Mao Z, et al. DeepXDE: A deep learning library for solving differential equations. SIAM Review, 2021, 63(1): 208-228 doi: 10.1137/19M1274067
[10]	Margossian CC. A review of automatic differentiation and its efficient implementation. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2019, 9(4): e1305 doi: 10.1002/widm.1305
[11]	Baydin AG, Pearlmutter BA, Radul AA, et al. Automatic differentiation in machine learning: A survey. Journal of Marchine Learning Research, 2018, 18: 1-43
[12]	Rasmussen CE, Williams CK. Gaussian Processes for Machine Learning. MIT Press Cambridge MA, 2006
[13]	Raissi M, Perdikaris P, Karniadakis GE. Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 2017, 348: 683-693 doi: 10.1016/j.jcp.2017.07.050
[14]	Raissi M, Perdikaris P, Karniadakis GE. Numerical Gaussian processes for time-dependent and nonlinear partial differential equations. SIAM Journal on Scientific Computing, 2018, 40(1): A98-A172 doi: 10.1137/17M1120762
[15]	Khoo Y, Lu J, Ying L. Solving parametric PDE problems with artificial neural networks. European Journal of Applied Mathematics, 2021, 32(3): 421-435 doi: 10.1017/S0956792520000182
[16]	Zhu Y, Zabaras N, Koutsourelakis PS, et al. Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. Journal of Computational Physics, 2019, 394: 56-81 doi: 10.1016/j.jcp.2019.05.024
[17]	Dissanayake M, Phan-Thien N. Neural-network-based approximations for solving partial differential equations. Communications in Numerical Methods in Engineering, 1994, 10(3): 195-201 doi: 10.1002/cnm.1640100303
[18]	Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378: 686-707 doi: 10.1016/j.jcp.2018.10.045
[19]	宋家豪, 曹文博, 张伟伟. FD-PINN: 频域物理信息神经网络. 力学学报, 2023, 55(5): 1195-1205 (Song Jiahao, Cao Wenbo, Zhang Weiwei. FD-PINN: Frequency domain physics-informed neural network. Chinese Journal of Theoretical and Applied Mechanics, 2023, 55(5): 1195-1205 (in Chinese) doi: 10.6052/0459-1879-23-169 Song Jiahao, Cao Wenbo, Zhang Weiwei. FD-PINN: Frequency domain physics-informed neural network. Chinese Journal of Theoretical and Applied Mechanics, 2023, 55(5): 1195-1205 (in Chinese) doi: 10.6052/0459-1879-23-169
[20]	Jagtap AD, Kawaguchi K, Karniadakis GE. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. Journal of Computational Physics, 2020, 404: 109136 doi: 10.1016/j.jcp.2019.109136
[21]	Jin X, Cai S, Li H, et al. NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. Journal of Computational Physics, 2021, 426: 109951 doi: 10.1016/j.jcp.2020.109951
[22]	查文舒, 李道伦, 沈路航等. 基于神经网络的偏微分方程求解方法研究综述. 力学学报, 2022, 54(3): 543-556 (Zha Wenshu, Li Daolun, Shen Luhang, et al. Review of neural network-based methods for solving partial differential equations. Chinese Journal of Theoretical and Applied Mechanics, 2022, 54(3): 543-556 (in Chinese) doi: 10.6052/0459-1879-21-617 Zha Wenshu, Li Daolun, Shen Luhang, et al. Review of neural network-based methods for solving partial differential equations. Chinese Journal of Theoretical and Applied Mechanics, 2022, 54(3): 543-556 (in Chinese) doi: 10.6052/0459-1879-21-617
[23]	Raissi M, Karniadakis GE. Hidden physics models: Machine learning of nonlinear partial differential equations. Journal of Computational Physics, 2018, 357: 125-141 doi: 10.1016/j.jcp.2017.11.039
[24]	蒋子超, 江俊扬, 姚清河等. 基于神经网络的差分方程快速求解方法. 力学学报, 2021, 53(7): 1912-1921 (Jiang Zichao, Jiang Junyang, Yao Qinghe, et al. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921 (in Chinese) doi: 10.6052/0459-1879-21-040 Jiang Zichao, Jiang Junyang, Yao Qinghe, et al. A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921 (in Chinese) doi: 10.6052/0459-1879-21-040
[25]	Yuan L, Ni YQ, Deng XY, et al. A-PINN: Auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations. Journal of Computational Physics, 2022, 462: 111260 doi: 10.1016/j.jcp.2022.111260
[26]	Iserles A. A First Course in the Numerical Analysis of Differential Equations. Cambridge University Press, 2009
[27]	Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv Preprint, arXiv, 2014, 14126980
[28]	Byrd RH, Lu P, Nocedal J, et al. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 1995, 16(5): 1190-1208 doi: 10.1137/0916069
[29]	Hussain S, Shah A, Ayub S, et al. An approximate analytical solution of the Allen-Cahn equation using homotopy perturbation method and homotopy analysis method. Heliyon, 2019, 5(12): e03060 doi: 10.1016/j.heliyon.2019.e03060
[30]	Agarap AF. Deep learning using rectified linear units (relu). arXiv Preprint, arXiv, 2018, 180308375
[31]	Fan E. Extended tanh-function method and its applications to nonlinear equations. Physics Letters A, 2000, 277(4-5): 212-218 doi: 10.1016/S0375-9601(00)00725-8
[32]	Yin X, Goudriaan J, Lantinga EA, et al. A flexible sigmoid function of determinate growth. Annals of Botany, 2003, 91(3): 361-371 doi: 10.1093/aob/mcg029
[33]	Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010
[34]	Bakodah H, Al-zaid N, Mirzazadeh M, et al. Decomposition method for solving Burgers’ equation with Dirichlet and Neumann boundary conditions. Optik, 2017, 130: 1339-1346 doi: 10.1016/j.ijleo.2016.11.140
[35]	Torrey L, Shavlik J. Transfer learning//Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques. IGI Global, 2010: 242-264

[1]	Wei Chang, Fan Yuchen, Zhou Yongqing, Liu Xin, Li Chi, Wang Heyang. SOLVING UNSTEADY PARTIAL DIFFERENTIAL EQUATIONS USING TIME-WEIGHTED PHYSICS-INFORMED NEURAL NETWORK[J]. Chinese Journal of Theoretical and Applied Mechanics, 2025, 57(3): 755-766. DOI: 10.6052/0459-1879-24-289
[2]	Zha Wenshu, Li Daolun, Shen Luhang, Zhang Wen, Liu Xuliang. REVIEW OF NEURAL NETWORK-BASED METHODS FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS[J]. Chinese Journal of Theoretical and Applied Mechanics, 2022, 54(3): 543-556. DOI: 10.6052/0459-1879-21-617
[3]	Cunxu Fan. Conjugate second-order deflection differential equation of global shell axial symmetrical bending problem[J]. Chinese Journal of Theoretical and Applied Mechanics, 2007, 23(5): 704-707. DOI: 10.6052/0459-1879-2007-5-2006-442
[5]	THE CANONICAL HAMILTONIAN REPRESENTATIONS IN A CLASS OF PARTIAL DIFFERENTIAL EQUATION 1)[J]. Chinese Journal of Theoretical and Applied Mechanics, 1999, 31(3): 347-357. DOI: 10.6052/0459-1879-1999-3-1995-040
[6]	DIES OF BURGERS EQUATION USING A LATTICE BOLTZMANN METHOD[J]. Chinese Journal of Theoretical and Applied Mechanics, 1999, 31(2). DOI: 10.6052/0459-1879-1999-2-1995-016
[7]	THE ANALYTICAL SOLUTION OF DIFFERENTIAL EQUATION OF ELASTIC CURVED SURFACE OF STEPPED THIN RECTANGULAR PLATE ON WINKLER'S FOUNDATION[J]. Chinese Journal of Theoretical and Applied Mechanics, 1992, 24(6): 754-762. DOI: 10.6052/0459-1879-1992-6-1995-800
[8]	GENERAL SOLUTION FOR A CLASS OF SYSTEM OF PARTIAL DIFFERENTIAL EQUATIONS AND ITS APPLICATION IN THE THEORY OF SHELLS[J]. Chinese Journal of Theoretical and Applied Mechanics, 1992, 24(6): 700-707. DOI: 10.6052/0459-1879-1992-6-1995-793
[9]	SINGULAR ANALYSIS TO THE SOLUTION OF THE RICCATIDIFFERENTIAL EQUAT[J]. Chinese Journal of Theoretical and Applied Mechanics, 1992, 24(2): 216-222. DOI: 10.6052/0459-1879-1992-2-1995-730
[10]	样条积分方程法分析弹塑性板弯曲[J]. Chinese Journal of Theoretical and Applied Mechanics, 1990, 22(2): 241-245. DOI: 10.6052/0459-1879-1990-2-1995-940

Cited By

Cited by

Periodical cited type(2)

1.	陈豪龙，唐欣越，王润华，周焕林，柳占立. 基于物理信息神经网络的多介质非线性瞬态热传导问题研究. 力学学报. 2025(01): 89-102 . 本站查看
2.	韦昌，樊昱晨，周永清，刘欣，李驰，王赫阳. 基于时间权重的物理信息神经网络求解非稳态偏微分方程. 力学学报. 2025(03): 755-766 . 本站查看

Other cited types(0)

Get Citation

PDF

XML

Article Metrics

Article views (535) PDF downloads (184) Cited by(2)

SELF-REGRESSIVE PHYSICS-INFORMED NEURAL NETWORK BASED ON RUNGE-KUTTA METHOD FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS

Abstract

References

Related Articles

Cited by

Periodical cited type(2)

Other cited types(0)

Catalog

Article Metrics

Related

Download Center

Author Center

SELF-REGRESSIVE PHYSICS-INFORMED NEURAL NETWORK BASED ON RUNGE-KUTTA METHOD FOR SOLVING PARTIAL DIFFERENTIAL EQUATIONS

Abstract

References

Related Articles

Cited by

Periodical cited type(2)

Other cited types(0)

Catalog

Article Metrics

Related

Download Center

Author Center

Export File

Citation

Format

Content