Abstract: In addressing the size effect observed in the plastic deformation of confined metallic thin layers, existing theoretical analyses have relied on pure shear assumptions and traditional passivation boundary conditions. However, their theoretical predictions are not in agreement with experimental results. In this paper, the finite element implementation of Gudmundson's theory of higher-order strain gradient plasticity is carried out based on the elasto-viscoplastic constitutive model. The method is then applied to study the plastic deformation mechanism in the shear of confined metallic layers. This study considers the additional compressive stress resulting from the inclined interface, and the combined compression-shear deformations are modeled through the user-defined plane element. In addition, a "soft-hard" boundary condition corresponding to the intermediate state is also introduced according to the physical context of surface unlocking. The results demonstrate that the shear flow stress of the confined layer under combined compressive and shear loads is significantly lower than that of the confined layer under pure shear, indicating that compressive stress dramatically reduces the yielding shear stress. The transition of the boundary condition due to the saturation of geometrically necessary dislocations at the interface is quantitatively characterized using the periodically passivated boundary condition. The theoretical predictions are in agreement with experimental data. The study emphasizes the importance of loading and boundary conditions in the size-dependent plasticity of confined layers.