Abstract: Flexible multibody system is a kind of mechanical system composed of many flexible components and kinematic pairs, such as flexible robot arms, helicopter rotors, deployable antennas of a satellite, and solar sail spacecraft. Flexible multibody systems serve as useful models in aerospace engineering, vehicle engineering, mechanical engineering, weapon engineering and so on. Recently, with the development of the engineering technology, new challenges have arisen to establish an accurate dynamic model of a flexible multibody system, as well as for the dynamic optimization design of such a flexible multibody system, especially of a flexible multibody system with variable-length components. As a matter of fact, when the component gets more and more flexible, the interactions between the component and the flexible multibody system cannot be disregarded when performing optimization design. The component-based structural optimization, hence, should be extended to the flexible multibody system-based structural optimization. In this review, the research background and significance of the dynamic optimization of flexible multibody systems are firstly surveyed. Three methods for investigating flexible multibody dynamics including flexible multibody systems with variable-length components are briefly outlined, i.e., floating frame of reference formulation (FFRF), geometrically exact formulation (GEF), and absolute nodal coordinate formulation (ANCF). Afterwards, the recent advances are systematically reviewed in the dynamic response optimization, the dynamic characteristics optimization, and the dynamic sensitivity analysis of flexible multibody systems, as well as the structural optimization, i.e., size optimization, shape optimization, and topology optimization of the flexible components in a flexible multibody system. Finally, several open problems are addressed for future studies.