EI、Scopus 收录
中文核心期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

复杂流动中的智能颗粒游动策略研究进展

邱敬然 赵立豪

邱敬然, 赵立豪. 复杂流动中的智能颗粒游动策略研究进展. 力学学报, 2021, 53(10): 2630-2639 doi: 10.6052/0459-1879-21-402
引用本文: 邱敬然, 赵立豪. 复杂流动中的智能颗粒游动策略研究进展. 力学学报, 2021, 53(10): 2630-2639 doi: 10.6052/0459-1879-21-402
Qiu Jingran, Zhao Lihao. Progresses in swimming strategy of smart particles in complex flows. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(10): 2630-2639 doi: 10.6052/0459-1879-21-402
Citation: Qiu Jingran, Zhao Lihao. Progresses in swimming strategy of smart particles in complex flows. Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(10): 2630-2639 doi: 10.6052/0459-1879-21-402

复杂流动中的智能颗粒游动策略研究进展

doi: 10.6052/0459-1879-21-402
基金项目: 国家自然科学基金(11911530141)和清华大学国强研究院(2019GQG1012)资助项目
详细信息
    作者简介:

    赵立豪, 教授, 主要研究方向: 颗粒湍流两相流, 智能体与强化学习. E-mail: zhaolihao@tsinghua.edu.cn

  • 中图分类号: V211.1+7

PROGRESSES IN SWIMMING STRATEGY OF SMART PARTICLES IN COMPLEX FLOWS

  • 摘要: 智能颗粒定义为可以根据环境的变化而主动调整自身在流场中运动的颗粒, 一般用于描述微小的游动物体, 如微生物、浮游生物和微纳机器人. 由于颗粒运动学特性和流场环境的复杂性, 探索智能颗粒的运动策略是一项具有现实意义与挑战性的研究. 近期强化学习算法被应用于智能颗粒的运动策略研究, 并取得一定进展. 本文将讨论强化学习在智能颗粒研究中的应用, 并介绍浮游生物运动策略的相关研究进展, 包括适用于海洋浮游生物的游动颗粒模型, 以及强化学习的基本原理和此类问题的研究框架. 海洋浮游生物的昼夜垂直迁移对其生存和繁衍至关重要, 生物学研究表明某些浮游生物可感知周围的流场信息, 但能否利用这些信息提高垂直方向游动效率仍是一个未知的问题. 基于这一背景, 相关研究考虑了重力沉降和颗粒形状对浮游生物垂直游动策略的影响. 研究发现细长的颗粒能够更加高效地垂直向上运动, 而重力沉降则导致智能游动策略发生较大变化. 在此基础上, 后续研究进一步考虑了局部流体信号的作用, 并讨论了基于局部信息在全局坐标系中定向运动的可能性. 当颗粒只能感受到局部流体信号时, 必须破坏动力学系统的对称性, 否则颗粒无法学到有效的垂向游动策略. 此外, 研究还发现强化学习能够利用流体信号背后的机制, 得到颗粒在二维定常流动和三维非定常湍流中高效垂直运动的策略. 这些智能游动策略依赖于本质的物理规律, 因此这些策略在更加复杂或真实的流动中也可能有优秀的表现.

     

  • 图  1  (a)智能颗粒示意图. x-y-z为全局坐标系, n-p-q为颗粒局部坐标系. (b)强化学习原理示意图[39]

    Figure  1.  (a) Sketch of smart particle. x-y-z defines the global frame of reference, and n-p-q defines the particle local frame of reference. (b) A diagram of reinforcement learning[39]

    图  2  智能颗粒垂直运动速度vz随形状和重力沉降作用的变化[22]. 颗粒速度以流动的特征速度u0无量纲化. RL: 强化学习得到的智能策略, naive: 简单策略, settle: 考虑沉降作用

    Figure  2.  Vertical velocity of smart particles vz as functions of the effects of gravitational settling and particle shape [22]. Particle velocity is normalized by velocity scale of the background flow u0. RL: smart strategy found by reinforcement learning. Naive: naive strategy. Settle: the cases considering settling effect

    图  3  (a)智能颗粒在二维Taylor-Green旋涡流动中的运动轨迹. 红点表示颗粒的初始位置, 背景为涡量云图. (b)智能颗粒与非智能颗粒当地流体垂直方向速度uy的概率密度分布(PDF)[23]

    Figure  3.  (a) Trajectories of smart particles in two-dimensional Taylor-Green vortex flow. Red dots represent the initial position of particles. Background contour shows the vorticity of fluid. (b) The probability distribution function (PDF) of the vertical velocity of local fluid uy[23]

    4  (a)颗粒在冻结流场(frozen)与随时间变化的流场(DNS)中的垂直方向速度vz, 以及游动速度nzvs, 当地流体速度uz和沉降速度vg,z的贡献. smart 2D: $ \mathrm{\Delta }{u}_{p} $流体信号下的智能策略; naive: 简单策略智能颗粒. (b)不同策略下颗粒在三维各向同性均匀湍流中的运动速度及各部分贡献. naive: 简单策略; 2D: Δup信号下的二维策略; 3D2S: Δup, Δuq信号下的三维策略; 3D4S: Snp, Snq, Δup, Δuq信号下的三维策略. (c)3D4S策略下的颗粒瞬时分布[24]

    4.  (a) Vertical velocity of particles vz in frozen flow (frozen) and time-dependent flow (DNS), with the contributions of swimming velocity nzvs, local fluid velocity uz, and settling velocity vg,z. Smart 2D: smart strategy with signal $ \mathrm{\Delta }{u}_{p} $. Naive: naive strategy. (b) Vertical velocity of particles under different swimming strategies in homogeneous isotropic turbulence. Naive: naive strategy. 2D: two-dimensional strategy with Δup signal. 3D2S: two-dimensional strategy with Δup, Δuq signals. 3D4S: two-dimensional strategy with Snp, Snq, Δup, Δuq signals. (c) Instanueous distribution of particles with 3D4S strategy[24]

    图  4  (a)颗粒在冻结流场(frozen)与随时间变化的流场(DNS)中的垂直方向速度vz, 以及游动速度nzvs, 当地流体速度uz和沉降速度vg,z的贡献. smart 2D: $ \mathrm{\Delta }{u}_{p} $流体信号下的智能策略; naive: 简单策略智能颗粒. (b)不同策略下颗粒在三维各向同性均匀湍流中的运动速度及各部分贡献. naive: 简单策略; 2D: Δup信号下的二维策略; 3D2S: Δup, Δuq信号下的三维策略; 3D4S: Snp, Snq, Δup, Δuq信号下的三维策略. (c)3D4S策略下的颗粒瞬时分布[24] (续)

    Figure  4.  (a) Vertical velocity of particles vz in frozen flow (frozen) and time-dependent flow (DNS), with the contributions of swimming velocity nzvs, local fluid velocity uz, and settling velocity vg,z. Smart 2D: smart strategy with signal $ \mathrm{\Delta }{u}_{p} $. Naive: naive strategy. (b) Vertical velocity of particles under different swimming strategies in homogeneous isotropic turbulence. Naive: naive strategy. 2D: two-dimensional strategy with Δup signal. 3D2S: two-dimensional strategy with Δup, Δuq signals. 3D4S: two-dimensional strategy with Snp, Snq, Δup, Δuq signals. (c) Instanueous distribution of particles with 3D4S strategy[24] (continued)

    表  1  二维与三维流场中独立流体信号分量

    Table  1.   Independent fluid signals in two-dimensional (2D) and three-dimensional (3D) flows

    Flow signals2D3D
    deformation rate${S_{nn}},\;\;{S_{np}}$${S_{nn}},\;{S_{np}},\;{S_{nq}},\;{S_{pp}},\;{S_{pq}}$
    relative angular velocity$\Delta {\varOmega _q}$$\Delta {\varOmega _n},\;\Delta {\varOmega _p},\;\Delta {\varOmega _q}$
    relative velocity$\Delta {u_n},\;\Delta {u_p}$$\Delta {u_n},\;\Delta {u_p},\;\Delta {u_q}$
    下载: 导出CSV
  • [1] Berg HC, Brown DA. Chemotaxis in Escherichia coli analysed by three-dimensional tracking. Nature, 1972, 239(5374): 500-504 doi: 10.1038/239500a0
    [2] Visser A. Small, Wet & Rational: Individual Based Zooplankton Ecology. Technical University of Denmark, 2011
    [3] Nelson BJ, Kaliakatsos IK, Abbott JJ. Microrobots for minimally invasive medicine. Annual Review of Biomedical Engineering, 2010, 12: 55-85 doi: 10.1146/annurev-bioeng-010510-103409
    [4] Reynolds CW. Flocks, herds and schools: A distributed behavioral model. ACM SIGGRAPH Computer Graphics, 1987, 21(4): 25-34
    [5] Li S, Batra R, Brown D, et al. Particle robotics based on statistical mechanics of loosely coupled components. Nature, 2019, 567(7748): 361-365 doi: 10.1038/s41586-019-1022-9
    [6] Durham WM, Climent E, Stocker R. Gyrotaxis in a steady vortical flow. Physical Review Letters, 2011, 106: 238102 doi: 10.1103/PhysRevLett.106.238102
    [7] Kessler JO. Individual and collective fluid dynamics of swimming cells. Journal of Fluid Mechanics, 1986, 173: 191-205 doi: 10.1017/S0022112086001131
    [8] Durham WM, Climent E, Barry M, et al. Turbulence drives microscale patches of motile phytoplankton. Nature Communications, 2013, 4: 2148 doi: 10.1038/ncomms3148
    [9] Borgnino M, Boffetta G, De Lillo F, et al. Gyrotactic swimmers in turbulence: shape effects and role of the large-scale flow. Journal of Fluid Mechanics, 2018, 856: R1 doi: 10.1017/jfm.2018.767
    [10] Gustavsson K, Berglund F, Jonsson PR, et al. Preferential sampling and small-scale clustering of gyrotactic microswimmers in turbulence. Physical Review Letters, 2016, 116(10): 108104 doi: 10.1103/PhysRevLett.116.108104
    [11] Borgnino M, Gustavsson K, De Lillo F, et al. Alignment of nonspherical active particles in chaotic flows. Physical Review Letters, 2019, 123(13): 138003 doi: 10.1103/PhysRevLett.123.138003
    [12] Zhan C, Sardina G, Lushi E, et al. Accumulation of motile elongated micro-organisms in turbulence. Journal of Fluid Mechanics, 2014, 739: 22-36 doi: 10.1017/jfm.2013.608
    [13] Qiu J, Cui Z, Climent E, et al. Fluid inertia is an effective gyrotactic mechanism for settling elongated micro-swimmers. ArXiv Preprint ArXiv: 2011.10253.2020
    [14] Kiørboe T, Saiz E, Visser AE. Hydrodynamic signal perception in the copepod Acartia tonsa. Marine Ecology Progress Series, 1999, 179: 97-111 doi: 10.3354/meps179097
    [15] Kiørboe T, Visser AW. Predator and prey perception in copepods due to hydromechanical signals. Marine Ecology Progress Series, 1999, 179: 81-95 doi: 10.3354/meps179081
    [16] Michalec FCCO, Fouxon I, Souissi S, et al. Zooplankton can actively adjust their motility to turbulent flow. Proceedings of the National Academy of Sciences, 2017, 114(52): E11199-E11207 doi: 10.1073/pnas.1708888114
    [17] Colabrese S, Gustavsson K, Celani A, et al. Flow navigation by smart microswimmers via reinforcement learning. Physical Review Letters, 2017, 118(15): 158004 doi: 10.1103/PhysRevLett.118.158004
    [18] Gustavsson K, Biferale L, Celani A, et al. Finding efficient swimming strategies in a three-dimensional chaotic flow by reinforcement learning. European Physical Journal E, 2017, 40(12): 110 doi: 10.1140/epje/i2017-11602-9
    [19] Biferale L, Bonaccorso F, Buzzicotti M, et al. Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using reinforcement learning. Chaos: An Interdisciplinary Journal of Nonlinear Science, 2019, 29(10): 103138 doi: 10.1063/1.5120370
    [20] Buzzicotti M, Biferale L, Bonaccorso F, et al. Optimal Control of Point-to-Point Navigation in Turbulent Time Dependent Flows using Reinforcement Learning. Springer, 2020: 223-234
    [21] Techy L. Optimal navigation in planar time-varying flow: Zermelo's problem revisited. Intelligent Service Robotics, 2011, 4(4): 271-283 doi: 10.1007/s11370-011-0092-9
    [22] Qiu J, Huang W, Xu C, et al. Swimming strategy of settling elongated micro-swimmers by reinforcement learning. Science China: Physics, Mechanics & Astronomy, 2020, 63(8): 284711
    [23] Qiu J, Mousavi N, Gustavsson K, et al. Navigation of micro-swimmers in steady flow: The importance of symmetries. ArXiv Preprint ArXiv: 2104.11303.2021
    [24] Qiu J, Mousavi N, Zhao L, et al. Active gyrotactic stability of microswimmers using hydromechanical signals. ArXiv Preprint ArXiv: 2105.12232.2021
    [25] Anderson LW, Sweeney BM. Role of inorganic ions in controlling sedimentation rate of a marine centric diatom ditylum brightwelli 1, 2. Journal of Phycology, 1978, 14(2): 204-214 doi: 10.1111/j.1529-8817.1978.tb02450.x
    [26] Walsby AE. Gas vesicles. Microbiological Reviews, 1994, 58(1): 94-144 doi: 10.1128/mr.58.1.94-144.1994
    [27] Arrieta J, Barreira A, Tuval I. Microscale patches of nonmotile phytoplankton. Physical Review Letters, 2015, 114(12): 128102 doi: 10.1103/PhysRevLett.114.128102
    [28] Colabrese S, Gustavsson K, Celani A, et al. Smart inertial particles. Physical Review Fluids, 2018, 3(8): 84301 doi: 10.1103/PhysRevFluids.3.084301
    [29] Gazzola M, Tchieu AA, Alexeev D, et al. Learning to school in the presence of hydrodynamic interactions. Journal of Fluid Mechanics, 2016, 789: 726-749 doi: 10.1017/jfm.2015.686
    [30] Mirzakhanloo M, Esmaeilzadeh S, Alam M. Active cloaking in Stokes flows via reinforcement learning. Journal of Fluid Mechanics, 2020, 903: A34
    [31] Novati G, Mahadevan L, Koumoutsakos P. Controlled gliding and perching through deep-reinforcement-learning. Physical Review Fluids, 2019, 4(9): 93902 doi: 10.1103/PhysRevFluids.4.093902
    [32] Huntley M, Brooks ER. Effects of age and food availability on diel vertical migration of Calanus pacificus. Marine Biology, 1982, 71(1): 23-31 doi: 10.1007/BF00396989
    [33] Bollens SM, Frost BW. Predator-induced diet vertical migration in a planktonic copepod. Journal of Plankton Research, 1989, 11(5): 1047-1065 doi: 10.1093/plankt/11.5.1047
    [34] Titelman J, Kiørboe T. Motility of copepod nauplii and implications for food encounter. Marine Ecology Progress Series, 2003, 247: 123-135 doi: 10.3354/meps247123
    [35] Happel J, Brenner H. Low Reynolds Number Hydrodynamics: With Special Applications to Particulate Media. Springer Science & Business Media, 2012
    [36] Zhao L, Challabotla NR, Andersson HI, et al. Rotation of nonspherical particles in turbulent channel flow. Physical Review Letters, 2015, 115(24): 244501 doi: 10.1103/PhysRevLett.115.244501
    [37] Kim S, Karrila SJ. Microhydrodynamics: Principles and Selected Applications. Courier Corporation, 2013
    [38] Jeffery GB. The motion of ellipsoidal particles immersed in a viscous fluid. Proceedings of the Royal Society of London. Series A, 1922, 102(715): 161-179
    [39] Sutton RS, Barto AG. Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). Cambridge: MIT Press, 1998
    [40] Watkins CJCH, Dayan P. Q-learning. Machine Learning, 1992, 8(3): 279-292
    [41] Durham WM, Kessler JO, Stocker R. Disruption of vertical motility by shear triggers formation of thin phytoplankton layers. Science, 2009, 323(5917): 1067-1070 doi: 10.1126/science.1167334
    [42] Sengupta A, Carrara F, Stocker R. Phytoplankton can actively diversify their migration strategy in response to turbulent cues. Nature, 2017, 543(7646): 555-558 doi: 10.1038/nature21415
    [43] Alageshan JK, Verma AK, Bec J, et al. Machine learning strategies for path-planning microswimmers in turbulent flows. Physical Review E, 2020, 101(4): 43110 doi: 10.1103/PhysRevE.101.043110
    [44] Gunnarson P, Mandralis I, Novati G, et al. Learning efficient navigation in vortical flow fields. ArXiv Preprint ArXiv: 2102.10536.2021
    [45] Yang Y, Bevan MA, Li B. Efficient navigation of colloidal robots in an unknown environment via deep reinforcement learning. Advanced Intelligent Systems, 2020, 2(1): 1900106 doi: 10.1002/aisy.201900106
    [46] Liu Y, Zou Z, Tsang ACH, et al. Mechanical rotation at low Reynolds number via reinforcement learning. Physics of Fluids, 2021, 33(6): 62007 doi: 10.1063/5.0053563
    [47] Tsang ACH, Tong PW, Nallan S, et al. Self-learning how to swim at low Reynolds number. Physical Review Fluids, 2020, 5(7): 74101 doi: 10.1103/PhysRevFluids.5.074101
  • 加载中
图(5) / 表(1)
计量
  • 文章访问数:  790
  • HTML全文浏览量:  259
  • PDF下载量:  207
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-08-19
  • 录用日期:  2021-09-28
  • 网络出版日期:  2021-09-29
  • 刊出日期:  2021-10-26

目录

    /

    返回文章
    返回