Abstract: To improve numerical accuracy, the numerical simulation of fracture mechanics problems needs to use dense meshes in the local area of crack propagation, while sparse meshes can be used in the area far away from the local area where the crack is located. Additionally, for the numerical simulation of crack propagation problems, most numerical methods have the problem of local remeshing in the process of crack growth. In this paper, an improved scaled boundary finite element method based on image quadtree are proposed to simulate crack propagation problems. This method can automatically perform the quadtree mesh generation according to the image of the geometric outer boundary of the structural domain without any artificial intervention, and the mesh generation efficiency is also very high. Due to the inherent advantages of the scaled boundary finite element method, the hanging nodes of the quadtree mesh can be directly regarded as new nodes without any special treatment. By introducing the idea of virtual node, the intersection of crack and the boundary of quadtree element is regarded as virtual nodes, and the degrees of freedom of virtual nodes are treated as additional degrees of freedom. The level set functions are used to characterize the crack surface inside the material. The subdomain with the discontinuous crack surface can be identified by the node level set functions, so that there is no need for remeshing during crack propagation, and the geometric characteristics of the interface can be characterized by the additional degrees of freedom of the scaled boundary finite element method. Finally, the performance of the proposed method is verified by several numerical examples. The results show that the proposed improved scaled boundary finite element method has high accuracy in solving the mode I and II stress intensity factors as well as simulating the internal crack growth path of materials.