Abstract: Structural defect identification is a crucial research area in the field of structural health monitoring, as it plays a significant role in assessing the safety of engineering structures. However, accurately determining the dimensions of structural defects can be extremely challenging. This paper proposes an innovative data-driven algorithm that combines scaled boundary finite element methods (SBFEM) with autoencoders (AE) causal dilated convolutional neural networks (CDCNN) for crack identification in semi-infinite media. In this model, the SBFEM is used to simulate the propagation of waves in semi-infinite media containing different crack-like defects. For different crack-like defects, by simply changing the scale center at the crack tip and the positions of the nodes at the crack opening, complex remeshing processes can be avoided, enabling efficient generation of enough training data. To simulate wave propagation in semi-infinite media, an absorbing boundary model based on Rayleigh damping is established, eliminating the need to compute the entire structural domain. To ensure the temporal order of data and achieve a larger receptive field without increasing the complexity of the neural network, a CDCNN is implemented in the deep learning framework. This allows for capturing more historical information. The AE, with its strong feature extraction capability, is utilized by mapping input samples to a low-dimensional space using its encoder to obtain latent features for training the neural network model, effectively improving the learning efficiency. Numerical examples demonstrate that the proposed model can efficiently and accurately identify quantitative information about cracks in semi-infinite media, and the identification efficiency of the AE-CDCNN model is about 2.7 times higher than that of a single CDCNN model.