Abstract: This study explores the modulation of segregation inversion induced by the initial packing mode of binary grain mixtures in a vertically vibrating rectangular container. A binary system composed of size-matched brass grains (higher mass density, called heavy grain) and plastic grains (lower mass density, called light grain) was employed to probe the density-driven segregation. Experiments and simulations show that the initial packing mode can adjust the segregation inversion time. The random-mixed mode exhibits the longest inversion time, succeeded by the RBNE packing mode. In contrast, inclined RBNE and side-by-side packing modes have progressively shorter segregation times. The spatiotemporal analysis of granular temperature shows that similar heavy and light grains have a self-aggregation acceleration effect, promoting faster segregation. Compared to the RBNE mode (heavier grains on top), the inclined RBNE mode enables more efficient vertical-to-horizontal energy conversion. The increased horizontal granular temperature reduces local packing fraction, strengthening the buoyanc effect and accelerating the downward migration of heavy grains coupled with the upward motion of light grains. Grain aggregation triggers cooperative clustering behavior, which further accelerates the RBNE-BNE inversion.