To address the issue of thermal energy supply and demand balance during deep peak regulation of thermal power units with a high proportion of new energy, this paper proposes an automated regulation algorithm that integrates reinforcement learning and adaptive control. Using the Markov decision process as a framework, a high dimensional state space and a multi-constrained action space are constructed, and a multi-objective weighted reward function is designed to balance accuracy, speed and economy. A 600 MW unit simulation platform is built based on MATLAB/SIMULINK and verified under conditions of rapid load rise and fall, low load stability, and variable operating disturbances. The results show that: when the load fluctuates, the main steam pressure overshoot is 2.1%, the adjustment time is 320 seconds, and IAE and ISE are 42.3% and 58.7% lower than PID, respectively; the temperature fluctuation in low-load operation is \pm 0.5%, and the coal consumption is 322 g/kWh, which is 4.7% lower than PID. The recovery time in the anti-disturbance experiment is 28 seconds, and the maximum deviation is 0.4 MPa. The robustness is superior to that of the traditional algorithm, providing technical support for deep peak regulation.