The time spectral method is a very popular reduced order frequency method for analyzing unsteady flow due to its advantage of being easily extended from an existing steady flow solver. Condition number of the inverse Fourier transform matrix used in the method can affect the solution convergence and stability of the time spectral equation system. This paper aims at evaluating the effect of the condition number of the inverse Fourier transform matrix on the solution stability and convergence of the time spectral method from two aspects. The first aspect is to assess the impact of condition number using a matrix stability analysis based upon the time spectral form of the scalar advection equation. The relationship between the maximum allowable Courant number and the condition number will be derived. Different time instant groups which lead to the same condition number are also considered. Three numerical discretization schemes are provided for the stability analysis. The second aspect is to assess the impact of condition number for real life applications. Two case studies will be provided: one is a flutter case, NASA rotor 67, and the other is a blade row interaction case, NASA stage 35. A series of numerical analyses will be performed for each case using different time instant groups corresponding to different condition numbers. The conclusion drawn from the two real life case studies will corroborate the relationship derived from the matrix stability analysis.