Abstract: In numerical simulations using Cartesian grids, regions with sharp variations in physical quantities are typically refined adaptively to ensure computational accuracy. However, conventional gradient-based adaptation criteria rely on predefined thresholds, which may lead to excessive refinement and an undue increase in mesh count, thereby impairing computational efficiency. Furthermore, existing deep reinforcement learning (Deep Reinforcement Learning, DRL) driven adaptive mesh refinement (Adaptive Mesh Refinement, AMR) methods are primarily designed for finite element frameworks, making them difficult to directly transfer to finite difference systems, and the coupling mechanism between adaptive decision-making and mesh smoothing has not yet been incorporated into the reinforcement learning framework. To address this issue, we design an observation space for the reinforcement learning agent, establish a reward function in integral form, and enable the agent to learn an adaptation strategy consistent with the numerical properties of finite difference methods. Additionally, the adjacency level difference is incorporated into the observation space, and a smoothing penalty term is constructed and embedded into the reward function, allowing the agent to actively satisfy smoothing constraints while executing refinement or coarsening operations, thereby achieving synchronized adaptive decision-making and mesh smoothing. The proposed reinforcement learning approach is evaluated on benchmark simulation cases. Results demonstrate that, under a limited mesh budget, the meshes generated by deep reinforcement learning capture physical gradients more effectively, leading to lower error levels. Moreover, the agent satisfies buffer layer conditions while executing refinement operations, significantly improving mesh utilization and computational efficiency. The DRL method is integrated with a developed weighted least squares interpolation algorithm for newly created hanging nodes, and successfully applied to unsteady flow problems with stationary and moving boundaries, achieving favorable results.