The residual compressive stress of shot peening can effectively improve the fatigue resistance of the material, but the residual compressive stress is unstable during service and will relax. In order to further study the relaxation rule of the residual compressive stress, the fatigue test of the standard tensile specimen of the shot peening 2024 aluminum alloy under two kinds of load was carried out. μ-X360 n X-ray diffractometer based on cosα method was used to analyze the variation law of residual stress during fatigue test. The results show that when the load is 10 kN the residual compressive stress shows regular relaxation, that is, the residual compressive stress drops significantly after the first cycle loading and the residual compressive stress basically remains unchanged or slowly decreased. At the load of 15 kN, the residual compressive stress appears to relax in reverse, that is, the residual compressive stress becomes a residual tensile stress after the first cyclic loading, and gradually decreases with the number of fatigue cycles and the residual stress distribution changed from the initial “√”-shaped distribution to an approximate linear distribution, gradually decreasing from the inside to the outside. In order to describe the relaxation behavior and process of residual compressive stress of shot peening, based on the existing experimental models, a new residual stress relaxation model considering the surface state of shot peening was proposed.