At present, in the refined recycling and utilization of asphalt pavement, the utilization rate of fine recycled asphalt aggregates (FRAP) is much lower than that of coarse recycled asphalt aggregates (CRAP). This not only leads to resource waste but also limits the larger-scale application of RAP in recycled asphalt mixtures. Therefore, this study systematically investigated the impact of FRAP content on the long-term performance of recycled asphalt mixtures through the semi-circular tensile (SCT) tests. Based on the energy evolution laws of strain energy (elasticity and plasticity) and surface energy during material fracture processes, corresponding improvement measures were proposed according to the influence patterns identified. Furthermore, the feasibility of applying a large proportion of FRAP in recycled asphalt mixtures was demonstrated. The research results show that at low temperatures, as the FRAP content increased from 0% to 12%, the strain energy of the recycled asphalt mixture decreases by 39.2%. This indicates a sharp decline in the energy required for cracking, making it more susceptible to the formation of cracks. Additionally, the surface energy of the recycled asphalt mixture decreases by 71.3%, resulting in less energy required for crack propagation, which suggests that cracks are more likely to spread to a completely fractured state. Therefore, the maximum allowable FRAP content should be tested before use in extremely cold regions. At medium temperatures, with the increase in FRAP content, the strain energy of the recycled asphalt mixture drops sharply, with the majority of the energy being used for crack propagation. The cracking issue in recycled asphalt mixtures is more serious than that in conventional asphalt mixture. It is recommended to further reduce the grade of new asphalt or increase the content of regenerant while ensuring the high-temperature stability of the recycled asphalt mixture. The increase in FRAP content resulted in a further rise in the content of aged asphalt, leading to a mismatch in the deformation of the recycled asphalt binder and the stress concentration at the interface between the old and new asphalt. As a result, the fatigue life of the recycled asphalt mixture decreases by nearly 53% compared to that of mixtures without FRAP, and the rate of fatigue damage accumulation increases by 26.7%. The deterioration of fatigue performance is directly related to the increase in FRAP content. The use of a stepwise mixing method, where a portion of new asphalt is mixed with FRAP first, enhances the bonding performance between the new and old asphalt within the recycled asphalt mixture, resulting in a certain degree of improvement in the fracture fatigue performance of the recycled asphalt mixture.