基于深度强化学习的磁浮列车悬浮架协同控制研究
RESEARCH ON COOPERATIVE CONTROL OF MAGLEV TRAIN SUSPENSION SYSTEM BASED ON DEEP REINFORCEMENT LEARNING
-
摘要: 由于磁浮列车悬浮架中存在多电磁铁耦合问题, 运行过程中耦合效应会引起悬浮架的不稳定悬浮, 为保证悬浮架中的多电磁铁模块在外界干扰下的稳定运行, 提出一种电磁悬浮(EMS)型磁浮列车悬浮架多电磁铁系统深度强化学习协同控制方法. 首先, 在考虑耦合情况下对磁浮列车悬浮架中的多电磁铁模块进行动力学建模并分析其耦合性; 其次提出基于SAC算法的悬浮架多电磁铁系统协同控制方法(SAC-CC), 构建深度强化学习协同控制算法框架, 将悬浮架多电磁铁模块的动力学模型转换为深度强化学习环境模型并为此模型设计奖励函数; 然后, 在静态起浮环境下进行训练得到SAC-CC控制器, 并分析不同奖励函数下控制器的控制性能和不同起浮位置下控制器的稳定性; 最后将SAC-CC控制器用于不同工况下悬浮架多电磁铁系统的悬浮控制及协同控制, 通过与传统的比例-积分-微分(PID)控制方法进行对比验证所提出的控制器的有效性和鲁棒性. 结果表明: 在不同工况下, 相较于PID控制器, 本文所提出的SAC-CC控制器不仅能够有效控制悬浮架中的多电磁铁模块稳定悬浮在平衡点位置附近, 同时显著减小电磁铁模块之间的耦合作用, 具有更加优秀的悬浮控制性能和协同控制性能, 不同工况下的SAC-CC控制器悬浮控制性能和协同控制性能分别提升30% ~ 99%和30% ~ 75%.Abstract: Due to the multi-magnet coupling problem in the suspension frame of maglev train, the coupling effect during operation will cause the unstable suspension of the suspension frame. In order to ensure the stable operation of the multi-magnet module in the suspension frame under external interference, a deep reinforcement learning collaborative control method for the multi-magnet system of the suspension frame of the electromagnetic suspension (EMS) maglev train is proposed. Firstly, considering the coupling, the dynamic modeling of the multi-electromagnet module in the maglev train suspension frame is carried out and its coupling is analyzed. Secondly, a cooperative control method (SAC-CC) of multi-electromagnet system based on SAC algorithm is proposed, and a framework of deep reinforcement learning cooperative control algorithm is constructed. The dynamic model of multi-electromagnet module of suspension frame is transformed into a deep reinforcement learning environment model and a reward function is designed for this model. Then, the SAC-CC controller is obtained by training in a static floating environment, and the control performance of the controller under different reward functions and the stability of the controller under different floating positions are analyzed. Finally, the SAC-CC controller is applied to the suspension control and cooperative control of the multi-electromagnet system under different working conditions. The effectiveness and robustness of the proposed controller are verified by comparing with the traditional proportional-integral-derivative (PID) control method. The results show that under different working conditions, compared with the PID controller, the SAC-CC controller proposed in this paper can not only effectively control the multi-magnet module in the suspension frame to suspend stably near the equilibrium position, but also significantly reduce the coupling between the electromagnet modules. It has better suspension control performance and cooperative control performance. The suspension control performance and cooperative control performance of the SAC-CC controller under different working conditions are increased by 30%-99% and 30%-75%, respectively.