It has been a tough task to determine the bifurcation points of steady state responses such as periodic or quasi-periodic solutions in nonlinear dynamical systems. The calculation and analysis methods have been well developed for periodic responses. Compared to periodic solutions, however, the solution techniques for quasi-periodic responses have only made relative progress in recent years, and the bifurcation analysis methods are urgently needed. To this end, a fast calculation approach will be proposed, based on the incremental harmonic balance (IHB) method, to determine the bifurcation point for symmetry breaking of QP responses. The QP response is described by generalized Fourier series with two irreducible frequencies. As the symmetry breaking happens, the coefficients of even-order (including the zeroth-order) harmonics will change from 0 to small quantities. Based on this feature, the coefficient of the zeroth-order harmonic is priorly given as a small quantity. And the controlling parameter is incorporated into the IHB iteration scheme. The Duffing oscillator subjected to multiple harmonic excitations with irreducible frequencies is investigated as an illustrative example. The symmetry breaking point can be efficiently determined, without any trail and error repeated calculation, as the convergent result can directly provide the controlling parameter close to the bifurcation value.