A method for optimizing support vector machine（ SVM） parameters was proposed based on isometric feature mapping（ Isomap） and improved genetic algorithm（ IGA）,to solve the problem that parameters are difficult to choose in the process of gearbox fault diagnosis and recognition. Firstly Isomap was used for high-dimensional feature data of gearbox vibration signal under the automatic optimal neighborhood parameter value. Then optimized the castigation parameter and nucleus function parameter of SVM by improved genetic algorithm. Finally the recognition and classification of lower dimensional data were realized by SVM. The proposed method was applied to gearbox fault diagnosis. The result shows that the method has higher diagnosis accuracy,and it has better diagnosis effect compared with traditional SVM method.