Sports stadiums, as high-energy-consuming public buildings, face challenges such as high energy consumption and low waste heat utilization. This paper constructs an optimization model for a heat recovery and storage system coupled with the building's thermal environment. Through key component selection, layout optimization, and multi-operating-condition simulation analysis, the system's energy-saving performance is improved. A 1:1 stadium model was constructed using the TRNSYS and FLUENT co-simulation platform. Verification results show that the deviation between simulated and measured energy consumption is ≤5%, and the temperature deviation is ≤1.2°C. Simulation data demonstrates that the system's heat recovery efficiency reaches 75.4%-78.6% under normal operating conditions, increasing to 79.3%-81.2% under event conditions and remaining above 72.5% under extreme conditions. Daily fluctuations in energy storage are controlled within 5.8%-7.6% under normal operating conditions and approach 8% under extreme conditions. The heat pump COP reaches 4.5 at 25°C, but significantly decreases to 2.3 at -5°C. This efficiency exceeds traditional rotary heat exchangers (≤60%) by 15%-20%, demonstrating significant improvement. The research achieved the goals of heat recovery efficiency ≥75% and energy storage fluctuation ≤8%, providing support for upgrading energy-saving technologies in sports stadiums.