The masking properties of the human ear have been successfully applied to adapt a speech enhancement system, yielding an improvement in speech quality. The accuracy of estimated speech spectra plays a major role in computing the noise masking threshold. Although traditional methods using the power-spectral-subtraction method to roughly estimate the speech spectra can provide an acceptable performance, the estimated speech spectra can be further improved for computing the noise masking threshold. In this article, we aim at finding a better spectral estimate of speech by the two-step-decision-directed method. In turn, this estimate is employed to compute the noise masking threshold of a perceptual gain factor. Experimental results show that the amounts of residual noise can be efficiently suppressed by embedding the two-step-decision-directed algorithm in the perceptual gain factor.