Abstract: In numerical simulations using Cartesian grids, adaptive mesh refinement (AMR) is often employed in regions where physical quantities undergo abrupt changes to ensure computational accuracy. However, conventional gradient-based adaptation criteria rely on predefined thresholds, which may lead to excessive refinement and an undue increase in mesh count, thereby impairing computational efficiency. Furthermore, existing deep reinforcement learning (DRL) driven AMR methods are primarily designed for finite element frameworks, making them difficult to directly transfer to finite difference systems, and the coupling mechanism between adaptive decision-making and mesh smoothing has not yet been incorporated into the reinforcement learning framework. To address this issue, we design an observation space for the reinforcement learning agent, establish a reward function in integral form, and enable the agent to learn an adaptation strategy consistent with the numerical properties of finite difference methods. Additionally, the adjacency level difference is incorporated into the observation space, and a smoothing penalty term is constructed and embedded into the reward function, allowing the agent to actively satisfy smoothing constraints while executing refinement or coarsening operations, thereby achieving synchronized adaptive decision-making and mesh smoothing. The proposed reinforcement learning approach is evaluated on benchmark simulation cases. Results demonstrate that, under a limited mesh budget, the meshes generated by DRL capture physical gradients more effectively, leading to lower error levels. Moreover, the agent satisfies buffer layer conditions while executing refinement operations, significantly improving mesh utilization and computational efficiency. The DRL method is integrated with a developed weighted least squares interpolation algorithm for newly created hanging elements, and successfully applied to unsteady flow problems with stationary and moving boundaries, verifying the effectiveness and generalization capability of the proposed method..