To improve estimating accuracy in the traditional mean rank or median rank estimation method, an improved mean rank is proposed in the correction principle of rank estimation function as the cumulative distribution function of samples by adjusting the applicable points of natural mean rank. Then a least squares estimation is performed by directly fitting the cumulative distribution function. Based on the hypothesis of Weibull distribution under small sample, the parameter estimations under different ranks are calculated by Monte Carlo simulation. The results indicate that the relative error on calculating scale parameter using the improved mean rank method for the Weibull distribution with different parameters is less than 9.5% under the condition of sample size not less than 4. Furthermore, the relative error on calculating mean time between failures using the improved mean rank method is less than 8.7%, while the relative errors using traditional methods are higher than 16%. From calculated results, proposed method can effectively improve the parameter estimation accuracy for Weibull distribution.