A FULLY IMPLICIT AND MONOLITHIC PARALLEL DECOMPOSITION METHOD FOR 3D FLUID-SOLID INTERACTION PROBLEMS
-
摘要: 三维流固耦合问题的非结构网格数值算法在很多工程领域都有重要应用, 目前现有的数值方法主要基于分区算法, 即流体和固体区域分别进行求解, 因此存在收敛速度较慢以及附加质量导致的稳定性问题, 此外, 该类算法的并行可扩展性不高, 在大规模应用计算方面也受到一定限制.本文针对三维非定常流固耦合问题, 提出一种基于区域分解的全隐全耦合可扩展并行算法.首先基于任意拉格朗日−欧拉框架建立流固耦合控制方程, 然后时间方向采用二阶向后差分隐式格式、空间方向采用非结构稳定化有限元方法进行离散.对于大规模非线性离散系统, 构造一种结合非精确Newton法、Krylov子空间迭代法与区域分解Schwarz预条件子的Newton-Krylov-Schwarz (NKS) 并行求解算法, 实现流体、固体和动网格方程的一次性整体求解.采用弹性障碍物绕流的标准测试算例对数值方法的准确性进行了验证, 数值性能测试结果显示本文构造的全隐全耦合算法具有良好的稳定性, 在不同的物理参数下具有良好的鲁棒性, 在“天河二号”超级计算机上, 当并行规模从192增加到3072个处理器核时获得了91%的并行效率.性能测试结果表明本文构造的NKS算法有望应用于复杂区域流固耦合问题的大规模数值模拟研究中.Abstract: Numerical methods based on unstructured meshes for the three-dimensional fluid-solid interaction problems have many applications in science and engineering. Most of the existing algorithms are based on the partitioned approach that the equations for the fluid and solid are solved separately using existing solvers by enabling them to share interface data with one another. The convergence of the partitioned approach is sometimes difficult to achieve because the method is basically a Gauss-Seidel type process and it may encounter the instability problem of the so-called added mass effect. Moreover, the parallel scalability of the solution algorithm is also an important issue when solving the large-scale problem. In contrast, the monolithic approach shows a more robust convergence and also eliminates the added mass effect even for complicated problems. In this work, a fully implicit and monolithic scalable parallel algorithm based on domain decomposition method is developed for the three-dimensional unsteady fluid-solid interaction problem. The governing equations are established based on the arbitrary Lagrangian-Eulerian framework, and a stabilized unstructured finite element method is employed for the discretization in space and a second-order fully implicit backward differentiation formula in time. An inexact Newton-Krylov method together with a restricted additive Schwarz preconditioner is constructed to solve the large, sparse system of nonlinear algebraic equations resulted from the discretization. The accuracy of the numerical method is verified by a benchmark problem of flows around an elastic obstacle. The numerical performance tests show that the fully implicit and monolithic method has good stability with large time step sizes and good robustness under different physical parameters, and a parallel efficiency of 91% was achieved for 3072 processor cores on the “Tianhe 2” supercomputer. The experimental results show that the proposed numerical method is expected to be applied for the numerical simulation of large-scale fluid-structure interaction problems in complex regions.
-
表 1 计算网格
Table 1. Computational meshes
Mesh 1 2 3 4 cells 1.89 × 104 1.30 × 105 1.08 × 106 6.03 × 106 DoF 2.90 × 104 1.78 × 105 1.47 × 106 8.11 × 106 np 24 48 96 192 memory/MB 51.86 141.5 453.8 1140 Newton 2.0 2.0 2.0 2.0 GMRES 11.6 17.0 28.2 40.4 time step/s 1.18 4.15 19.88 77.72 表 2 观测点
${{P}}1\;(0.4,\;0.2,\; - 0.2)$ 处的位移Table 2. Displacements at point
${P}1\;(0.4,\;0.2,\; - 0.2)$ $d_x^{}$/m $d_y^{}$/m $d_z^{}$/m mesh 1 1.984 × 10−3 5.152 × 10−4 −9.431 × 10−5 mesh 2 1.729 × 10−3 5.052 × 10−4 −6.124 × 10−5 mesh 3 1.602 × 10−3 4.832 × 10−4 −5.775 × 10−5 mesh 4 1.598 × 10−3 4.847 × 10−4 −6.785 × 10−5 Ref. [22] 1.63 × 10−3 5.05 × 10−4 −1.22 × 10−4 表 3 观测点
${{P2}}\;(0.5,\;0.2,\; - 0.2)$ 处的位移Table 3. Displacements at point
${{P2}}\;(0.5,\;0.2,\; - 0.2)$ $d_x^{}$/m $d_y^{}$/m $d_z^{}$/m mesh 1 1.964 × 10−3 −5.377 × 10−4 −4.855 × 10−5 mesh 2 1.700 × 10−3 −4.683 × 10−4 −7.854 × 10−5 mesh 3 1.563 × 10−3 −4.251 × 10−4 −7.185 × 10−5 mesh 4 1.550 × 10−3 −4.240 × 10−4 −7.216 × 10−5 Ref. [22] 1.54 × 10−3 −4.65 × 10−4 −3.13 × 10−5 表 4 子区域的重叠层数及子问题求解器对算法性能的影响
Table 4. Impact of the overlapping size and the subsolver on the NKS algorithm
$\delta $ Subsolver Newton GMRES Time/s 1 ILU(0) 2.0 62.3 18.07 1 ILU(1) 2.0 40.0 18.15 1 ILU(2) 2.0 31.2 19.34 1 ILU(3) 2.0 29.2 34.98 2 ILU(0) 2.0 54.3 17.40 2 ILU(1) 2.0 34.4 18.23 2 ILU(2) 2.0 28.2 19.88 2 ILU(3) 2.0 24.5 37.50 表 5 不同时间步长对NKS算法收敛性能的影响
Table 5. Performance of NKS with respect to
$\Delta t$ $\Delta t$ Newton GMRES Time/s 0.001 2.0 19.5 21.34 0.002 2.0 22.6 21.54 0.004 2.0 26.2 21.97 0.008 2.0 39.0 22.96 0.016 2.0 47.2 23.99 0.032 2.0 55.9 24.25 0.064 2.6 71.8 32.98 0.128 3.0 70.4 38.17 表 6 关于流体黏性系数的算法鲁棒性测试
Table 6. Robustness of the algorithm with respect to the fluid viscosity
${\mu ^f}$ Newton GMRES Time/s 1.0 2.0 26.2 21.97 0.1 2.0 26.6 22.04 0.01 2.0 26.8 21.98 表 7 关于固体密度的算法鲁棒性测试
Table 7. Robustness of the algorithm with respect to the solid density
${\rho ^s}$ Newton GMRES Time/s 1 2.0 60.8 25.29 10 2.0 53.2 24.41 100 2.0 45.6 23.75 1000 2.0 26.2 21.97 2000 2.0 22.4 21.13 4000 2.0 14.0 20.85 表 8 关于杨氏模量和Poisson比的算法鲁棒性测试
Table 8. Robustness of the algorithm with respect to the Young’s modulus and Poisson’s ratio
${E^s}$ ${\nu ^s}$ Newton GMRES Time/s 1.4 × 106 0.1 2.0 25.0 21.62 1.4 × 106 0.2 2.0 25.7 21.87 1.4 × 106 0.3 2.0 25.8 21.96 1.4 × 106 0.4 2.0 26.2 21.97 1.4 × 106 0.48 2.0 41.4 23.39 1.4 × 106 0.4 2.0 26.2 21.97 7.0 × 105 0.4 2.0 22.9 21.67 3.5 × 105 0.4 2.0 20.2 21.44 1.75 × 105 0.4 2.0 18.0 21.21 8.75 × 104 0.4 2.0 16.8 21.05 表 9 算法并行可扩展性测试
Table 9. Parallel performance and scalability of the algorithm
${n_p}$ 192 384 768 1536 3072 Newton 2.0 2.0 2.0 2.0 2.0 GMRES 45.7 47.0 49.3 52.4 53.5 time 83.41 35.07 17.20 9.65 5.73 speedup 1 2.38 4.85 8.64 14.56 ideal 1 2 4 8 16 efficiency 100% 119% 121% 108% 91% 表 10 本文算法与其他文献算法的性能比较
Table 10. Comparison of different numerical approaches found in literature
Solver Dofs Newton steps Ratio* CPU cores Parallel efficiency Source Ref. [22] Newton-Block LDU 14 000 000 6 ~ 9 109.9 16 ~ 256 41% Fig.14, Fig.15 Ref. [28] Newton-Krylov-
Multigrid-Richardson120 902 4 1139 1 − Tab. 1, Tab. 12 Ref. [29] Newton-Multigrid 3 531 304 5.15 450.3 1 ~ 32 31% Tab. 6, Tab. 9, Fig. 7 this work Newton-Krylov-
Schwarz8 110 000 2 543.5 192 ~ 3072 91% Tab. 1, Tab. 9 *Note: Here the “Ratio” indicates the degree of freedom persecond computed with one CPU core for each time step -
[1] 孙茂. 动物飞行的空气动力学. 空气动力学报, 2018, 36(1): 122-128 (Sun Mao. Aerodynamics of animal flight. Acta Aerodynamica Sinica, 2018, 36(1): 122-128 (in Chinese) [2] Bantwal A, Singh A, Menon AR, et al. Hemodynamic study of blood flow in the carotid artery with a focus on carotid sinus using fluid-structure interaction. Journal of Fluids Engineering, 2022, 144(2): 021403 doi: 10.1115/1.4051902 [3] Gao D, Deng Z, Yang W, et al. Review of the excitation mechanism and aerodynamic flow control of vortex-induced vibration of the main girder for long-span bridges: A vortex-dynamics approach. Journal of Fluids and Structures, 2021, 105: 103348 doi: 10.1016/j.jfluidstructs.2021.103348 [4] Wijesooriya K, Mohotti D, Amin A, et al, An uncoupled fluid structure interaction method in the assessment of structural responses of tall buildings, Structures, 2020, 25: 448-462 [5] Sayed M, Bucher P, Guma G, et al. Aeroelastic simulations based on high-fidelity CFD and CSD models//Handbook of Wind Energy Aerodynamics. Cham: Springer International Publishing, 2022: 1-76 [6] Xu F, Morganti S, Zakerzadeh R, et al. A framework for designing patient- specific bioprosthetic heart valves using immersogeometric fluid-structure interaction analysis. International Journal for Numerical Methods in Biomedical Engineering, 2018, 34(4): e2938 doi: 10.1002/cnm.2938 [7] Totorean AF, Bernad SI, Ciocan T, et al. computational fluid dynamics applications in cardiovascular medicine- from medical image-based modeling to simulation: numerical analysis of blood flow in abdominal aorta//Advances in Fluid Mechanics, Singapore: Springer, 2022: 1-42 [8] Bazilevs Y, Takizawa K, Tezduyar TE. Challenges and directions in computational fluid-structure interaction. Mathematical Models and Methods in Applied Sciences, 2013, 23(2): 215-221 doi: 10.1142/S0218202513400010 [9] Hou G, Wang J, Layton A. Numerical methods for fluid-structure interaction-a review. Communications in Computational Physics, 2012, 12(2): 337-377 doi: 10.4208/cicp.291210.290411s [10] 何涛. 流固耦合数值方法研究概述与浅析. 振动与冲击, 2018, 37(4): 184-190 (He Tao. Numerical solution techniques for fluid-structure interaction simulations: A brief review and discussion. Journal of Vibration and Shock, 2018, 37(4): 184-190 (in Chinese) doi: 10.13465/j.cnki.jvs.2018.04.028 [11] Farhat C, Van der Zee KG, Geuzaine P. Provably second-order time-accurate loosely-coupled solution algorithms for transient nonlinear computational aeroelasticity. Computer Methods in Applied Mechanics and Engineering, 2006, 195(17-18): 1973-2001 doi: 10.1016/j.cma.2004.11.031 [12] Burman E, Durst R, Fernández MA, et al. Fully discrete loosely coupled Robin-Robin scheme for incompressible fluid- structure interaction: stability and error analysis. Numerische Mathematik, 2022, 151(4): 807-840 doi: 10.1007/s00211-022-01295-y [13] Lorentzon J, Revstedt J. A numerical study of partitioned fluid-structure interaction applied to a cantilever in incompressible turbulent flow. International Journal for Numerical Methods in Engineering, 2020, 121(5): 806-827 doi: 10.1002/nme.6245 [14] Bukač M, Trenchea C. Adaptive, second-order, unconditionally stable partitioned method for fluid-structure interaction. Computer Methods in Applied Mechanics and Engineering, 2022, 393: 114847 doi: 10.1016/j.cma.2022.114847 [15] Förster C, Wall WA, Ramm E. Artificial added mass instabilities in sequential staggered coupling of nonlinear structures and incompressible viscous flows. Computer Methods in Applied Mechanics and Engineering, 2007, 196(7): 1278-1293 doi: 10.1016/j.cma.2006.09.002 [16] 何涛. 基于ALE 有限元法的流固耦合强耦合数值模拟. 力学学报, 2018, 50(2): 395-404 (He Tao. A partitioned strong coupling algorithm for fluid-structure interaction using arbitrary Lagrangian-Eulerian finite element formulation. Chinese Journal of Theoretical and Applied Mechanics, 2018, 50(2): 395-404 (in Chinese) doi: 10.6052/0459-1879-17-197 [17] Dettmer W, Perić D. A computational framework for fluid-structure interaction: finite element formulation and applications. Computer Methods in Applied Mechanics and Engineering, 2006, 195(41-43): 5754-5779 doi: 10.1016/j.cma.2005.10.019 [18] Kuberry P, Lee H. A decoupling algorithm for fluid-structure interaction problems based on optimization. Computer Methods in Applied Mechanics and Engineering, 2013, 267: 594-605 doi: 10.1016/j.cma.2013.10.006 [19] Baek H, Karniadakis GE. A convergence study of a new partitioned fluid-structure interaction algorithm based on fictitious mass and damping. Journal of Computational Physics, 2012, 231(2): 629-652 doi: 10.1016/j.jcp.2011.09.025 [20] Dettmer WG, Lovrić A, Kadapa C, et al. New iterative and staggered solution schemes for incompressible fluid-structure interaction based on Dirichlet-Neumann coupling. International Journal for Numerical Methods in Engineering, 2021, 122(19): 5204-5235 doi: 10.1002/nme.6494 [21] Schott B, Ager C, Wall WA. A monolithic approach to fluid-structure interaction based on a hybrid Eulerian-ALE fluid domain decomposition involving cut elements. International Journal for Numerical Methods in Engineering, 2019, 119(3): 208-237 doi: 10.1002/nme.6047 [22] Jodlbauer D, Langer U, Wick T. Parallel block-preconditioned monolithic solvers for fluid‐structure interaction problems. International Journal for Numerical Methods in Engineering, 2019, 117(6): 623-643 doi: 10.1002/nme.5970 [23] Wang Y, Jimack PK, Walkley MA, et al. An energy stable one-field monolithic arbitrary Lagrangian-Eulerian formulation for fluid-structure interaction. Journal of Fluids and Structures, 2020, 98: 103117 doi: 10.1016/j.jfluidstructs.2020.103117 [24] Wick T, Wollner W. Optimization with nonstationary, nonlinear monolithic fluid-structure interaction. International Journal for Numerical Methods in Engineering, 2021, 122(19): 5430-5449 doi: 10.1002/nme.6372 [25] Dutta S, Jog CS. A monolithic arbitrary Lagrangian-Eulerian-based finite element strategy for fluid-structure interaction problems involving a compressible fluid. International Journal for Numerical Methods in Engineering, 2021, 122(21): 6037-6102 doi: 10.1002/nme.6783 [26] Degroote J, Bathe KJ, Vierendeels J. Performance of a new partitioned procedure versus a monolithic procedure in fluid-structure interaction. Computers & Structures, 2009, 87(11-12): 793-801 [27] Ha ST, Ngo LC, Saeed M, et al. A comparative study between partitioned and monolithic methods for the problems with 3D fluid-structure interaction of blood vessels. Journal of Mechanical Science and Technology, 2017, 31(1): 281-287 doi: 10.1007/s12206-016-1230-2 [28] Aulisa E, Bna S, Bornia G. A monolithic ALE Newton-Krylov solver with Multigrid-Richardson-Schwarz preconditioning for incompressible fluid-structure interaction. Computers & Fluids, 2018, 174: 213-228 [29] Failer L, Richter T. A parallel Newton multigrid framework for monolithic fluid-structure interactions. Journal of Scientific Computing, 2020, 82(2): 1-27 [30] Wick T. Fluid-structure interactions using different mesh motion techniques. Computers & Structures, 2011, 89(13-14): 1456-1467 [31] Cai XC, Sarkis M. A restricted additive Schwarz preconditioner for general sparse linear systems. SIAM Journal on Scientific Computing, 1999, 21(2): 792-797 doi: 10.1137/S106482759732678X [32] Liao ZJ, Chen R, Yan Z, et al. A parallel implicit domain decomposition algorithm for the large eddy simulation of incompressible turbulent flows on 3 D unstructured meshes. International Journal for Numerical Methods in Fluids, 2019, 89(9): 343-361 doi: 10.1002/fld.4695 [33] Liao ZJ, Qin S, Chen R, et al. A parallel domain decomposition method for large eddy simulation of blood flow in human artery with resistive boundary condition. Computers & Fluids, 2022, 232: 105201 [34] Tezduyar TE, Mittal S, Ray SE, et al. Incompressible flow computations with stabilized bilinear and linear equal-order-interpolation velocity-pressure elements. Computer Methods in Applied Mechanics and Engineering, 1992, 95(2): 221-242 doi: 10.1016/0045-7825(92)90141-6 [35] Whiting CH, Jansen KE. A stabilized finite element method for the incompressible Navier-Stokes equations using a hierarchical basis. International Journal for Numerical Methods in Fluids, 2001, 35(1): 93-116 doi: 10.1002/1097-0363(20010115)35:1<93::AID-FLD85>3.0.CO;2-G [36] Dennis Jr JE, Schnabel RB. Numerical methods for unconstrained optimization and nonlinear equations. Society for Industrial and Applied Mathematics, 1996 [37] Knoll DA, Keyes DE. Jacobian-free Newton-Krylov methods: A survey of approaches and applications. Journal of Computational Physics, 2004, 193(2): 357-397 doi: 10.1016/j.jcp.2003.08.010 [38] Balay S, Abhyankar S, Adams M, et al. PETSc Users Manual. Argonne National Laboratory, 2022 [39] Blacker TD, Owen SJ, Staten ML, et al. CUBIT Geometry and Mesh Generation Toolkit 15.1 User Documentation. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States), 2016 [40] Karypis G, Schloegel K, Kumar V. ParMETIS: Parallel Graph Partitioning and Sparse Matrix Ordering Library Version 4.0, University of Minnesota -