THERMAL SCIENCE

International Scientific Journal

DYNAMIC LOAD FORECASTING OF THERMAL STORAGE SYSTEM BASED ON MULTIMODAL TRANSFORMER

ABSTRACT
Driven by the "dual carbon" goal, this paper proposes a dynamic load forecasting method based on a multimodal transformer to improve the operating efficiency of the thermal storage system. This method constructs a multimodal dataset, designs an adaptive modal weight allocation mechanism, and utilises the transformer structure to capture both long-term trends and short-term fluctuations. The experiment utilises data from a regional heating system as a sample and compares it with models such as ARIMA and LSTM. The results show that the proposed model performs best in the test set, with RMSE of 2.35 kW, MAE of 1.82 kW, MAPE of 2.15%, and MaxAE of 5.27 kW, which are 18.3%, 16.8%, 15.2%, and 19.4% lower than the suboptimal multimodal CNN-LSTM, respectively. The two-stage fusion combines dynamic modal weights and gating, outperforming single-stage by 11% in MAPE. Meteorological data dominates in cold waves due to strong correlation with load (r=0.82). Transformer’s self-attention models global dependencies, avoiding LSTM sequential loss, explaining its three hours RMSE of 7.82 kW vs. LSTM 9.65 kW. It has significant advantages in high load intervals and long-time series predictions, providing support for the intelligent operation of the system.
KEYWORDS
PAPER SUBMITTED: 2025-05-12
PAPER REVISED: 2025-07-25
PAPER ACCEPTED: 2025-08-25
PUBLISHED ONLINE: 2025-11-29
DOI REFERENCE: https://doi.org/10.2298/TSCI2506247L
CITATION EXPORT: view in browser or download as text file
THERMAL SCIENCE YEAR 2025, VOLUME 29, ISSUE No. 6, PAGES [4247 - 4255]
REFERENCES
[1] Fan, J., et al., Optimizing Attention in a Transformer for Multihorizon, Multienergy Load Forecasting In Integrated Energy Systems, IEEE Transactions on Industrial Informatics, 20 (2024), 8, pp. 10238-10248, 10.1109/tii.2024.3392278
[2] Wang, C., et al., A Transformer-Based Method of Multienergy Load Forecasting in Integrated Energy System, IEEE Transactions on Smart Grid, 13 (2022), 4, pp. 2703-2714, 10.1109/tsg.2022.3166600
[3] Zhan, X., et al., Reliable Long-Term Energy Load Trend Prediction Model for Smart Grid Using Hierarchical Decomposition Self-Attention Network, IEEE Transactions on Reliability, 72 (2022), 2, pp. 609-621, 10.1109/tr.2022.3174093
[4] Varshney, R. P., et al., A Multi-Modal Image Encoding and Self-Attention-Based Transformer Framework with Sentiment Analysis for Financial Time Series Prediction, International Journal of Computational Vision and Robotics, 15 (2025), 1, pp. 31-58, 10.1504/ijcvr.2025.142920
[5] Wu, J., et al., Ensembled Traffic-Aware Transformer-Based Predictive Energy Management for Electrified Vehicles, IEEE Transactions on Intelligent Transportation Systems, 25 (2024), 9, pp. 12333-12346, 10.1109/tits.2024.3375331
[6] Simaiya, S., et al., A Transfer Learning-Based Hybrid Model with LightGBM for Smart Grid Short-Term Energy Load Prediction, Energy Exploration & Exploitation, 42 (2024), 5, pp. 1853-1876, 10.1177/01445987241256472
[7] Dalal, S., et al., A Hybrid Model for Short-Term Energy Load Prediction Based on Transfer Learning with LightGBM for smart Grids in Smart Energy Systems, Journal of Urban Technology, 32 (2025), 1, pp. 49-75, 10.1080/10630732.2024.2380639
[8] Hasnain, K. N., Integrating Machine Learning for Real-Time Energy Load Forecasting in US Smart Grids: A Multi-Model Comparative Approach, Journal of Data and Digital Innovation (JDDI), 2 (2025), 2, pp. 1-19
[9] Zhao, S., et al., Short and Long-Term Renewable Electricity Demand Forecasting Based on CNN-Bi-GRU Model, ICCK Transactions on Emerging Topics in Artificial Intelligence, 2 (2025), 1, pp. 1-15, 10.62762/tetai.2024.532253
[10] Hu, J., et al., Dual-Layer Multi-Mode Energy Management Optimization Strategy for Electric Vehicle Hybrid Energy Storage Systems, Journal of Power Electronics, 25 (2025), 1, pp. 115-127, 10.1007/s43236-024-00886-2
[11] Shi, Y., et al., Electric Vehicle Charging Situation Awareness for Ultra-Short-Term Load Forecast of Charging Stations, Journal of Shanghai Jiaotong University (Science), 28 (2023), 1, pp. 28-38, 10.1007/s12204-023-2566-2
[12] Tu, F., et al., The MulTCIM: Digital Computing-in-Memory-Based Multimodal Transformer Accelerator with Attention-Token-Bit Hybrid Sparsity, IEEE Journal of Solid-State Circuits, 59 (2023), 1, pp. 90-101, 10.1109/jssc.2023.3305663

© 2026 Society of Thermal Engineers of Serbia. Published by the Vinča Institute of Nuclear Sciences, National Institute of the Republic of Serbia, Belgrade, Serbia. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International licence