Multi-Agent Reinforcement Learning for Energy-Efficient Resource Scheduling in 5G-A Systems
Keywords:
5G-Advanced systems; multi-agent reinforcement learning; energy-efficient scheduling; intelligent wireless infrastructure; edge intelligence; resource orchestration; sustainable networking; distributed optimization; network slicing; autonomous communicationsAbstract
The evolution of fifth-generation advanced wireless systems has intensified the complexity of resource scheduling across heterogeneous network infrastructures characterized by ultra-dense deployments, distributed edge intelligence, dynamic traffic demands, and stringent sustainability objectives. Traditional optimization-centric scheduling frameworks increasingly struggle to adapt to rapidly fluctuating network states, particularly under the combined pressures of energy efficiency, latency assurance, fairness preservation, and infrastructure scalability. This paper investigates the application of multi-agent reinforcement learning for energy-efficient resource scheduling in 5G-Advanced systems from a systems-oriented and socio-technical perspective. The study explores how distributed intelligent agents can coordinate spectrum allocation, computational orchestration, user association, transmission power adaptation, and edge resource balancing while minimizing operational energy consumption and preserving service reliability. Unlike centralized reinforcement learning architectures that often encounter scalability bottlenecks and delayed convergence under dense deployment conditions, multi-agent frameworks enable localized intelligence and collaborative adaptation across heterogeneous network domains. The paper develops a comprehensive conceptual architecture for distributed scheduling governance in 5G-A environments and evaluates the implications of agent coordination under varying operational constraints. Particular emphasis is placed on sustainability trade-offs, infrastructure interoperability, policy governance, fairness among network participants, and resilience against adversarial and unstable conditions. The analysis further examines how multi-agent learning interacts with edge-cloud convergence, network slicing, digital twin environments, and green communication objectives. The paper concludes that multi-agent reinforcement learning represents a promising foundation for next-generation adaptive wireless infrastructure management, although significant challenges remain regarding explainability, coordination stability, regulatory oversight, and long-term deployment sustainability in large-scale communication ecosystems.
References
[1] Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
[2] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
[3] Zhang, K., Yang, Z., & Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, 321–384.
[4] Busoniu, L., Babuska, R., & De Schutter, B. (2008). A comprehensive survey of multi-agent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 38(2), 156–172.
[5] Wu, J., Zhang, Y., Zukerman, M., & Yung, E. K. N. (2015). Energy-efficient basestation sleep-mode techniques in green cellular networks: A survey. IEEE Communications Surveys & Tutorials, 17(2), 803–826.
[6] Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017). A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys & Tutorials, 19(4), 2322–2358.
[7] Andrews, J. G., Buzzi, S., Choi, W., Hanly, S. V., Lozano, A., Soong, A. C. K., & Zhang, J. C. (2014). What will 5G be? IEEE Journal on Selected Areas in Communications, 32(6), 1065–1082.
[8] Foukas, X., Patounas, G., Elmokashfi, A., & Marina, M. K. (2017). Network slicing in 5G: Survey and challenges. IEEE Communications Magazine, 55(5), 94–100.
[9] Wang, X., Han, Y., Wang, C., Zhao, Q., Chen, X., & Chen, M. (2020). In-edge AI: Intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Network, 33(5), 156–165.
[10] Auer, G., Giannini, V., Desset, C., Godor, I., Skillermark, P., Olsson, M., Imran, M. A., Sabella, D., Gonzalez, M. J., Blume, O., & Fehske, A. (2011). How much energy is needed to run a wireless network? IEEE Wireless Communications, 18(5), 40–49.
[11] Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019). Artificial neural networks-based machine learning for wireless networks: A tutorial. IEEE Communications Surveys & Tutorials, 21(4), 3039–3071.
[12] Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018). Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2974–2982.
[13] Mach, P., & Becvar, Z. (2017). Mobile edge computing: A survey on architecture and computation offloading. IEEE Communications Surveys & Tutorials, 19(3), 1628–1656.
[14] Jiang, J., & Lu, Z. (2019). Learning attentional communication for multi-agent cooperation. Advances in Neural Information Processing Systems, 32, 7254–7264.
[15] Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018). A comprehensive survey on machine learning for networking. Journal of Internet Services and Applications, 9(1), 1–99.
[16] Abbas, N., Zhang, Y., Taherkordi, A., & Skeie, T. (2018). Mobile edge computing: A survey. IEEE Internet of Things Journal, 5(1), 450–465.
[17] Taleb, T., Samdanis, K., Mada, B., Flinck, H., Dutta, S., & Sabella, D. (2017). On multi-access edge computing: A survey of the emerging 5G network edge cloud architecture and orchestration. IEEE Communications Surveys & Tutorials, 19(3), 1657–1681.
[18] Zhang, H., Liu, N., Chu, X., Long, K., Aghvami, A. H., & Leung, V. C. M. (2017). Network slicing based 5G and future mobile networks: Mobility, resource management, and challenges. IEEE Communications Magazine, 55(8), 138–145.
[19] Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., Niyato, D., & Poor, H. V. (2021). 6G Internet of Things: A comprehensive survey. IEEE Internet of Things Journal, 9(1), 359–383.
[20] Nguyen, T. T., & Reddi, V. J. (2021). Deep reinforcement learning for cyber security. IEEE Transactions on Neural Networks and Learning Systems, 34(4), 3779–3795.
[21] Bianzino, A. P., Chaudet, C., Rossi, D., & Rougier, J. L. (2012). A survey of green networking research. IEEE Communications Surveys & Tutorials, 14(1), 3–20.
[22] Sun, Y., Peng, M., Zhou, Y., Huang, Y., & Mao, S. (2019). Application of machine learning in wireless networks: Key techniques and open issues. IEEE Communications Surveys & Tutorials, 21(4), 3072–3108.
[23] Li, Q. (2026). QoS Assurance Mechanism for 5G Network Slicing Based on the Deep Reinforcement Learning PPO Algorithm. arXiv preprint arXiv:2605.03345.
[24] Liang, Y. C., Chen, Y., Li, G. Y., & Mahonen, P. (2021). Cognitive radio networking and communications: An overview. IEEE Transactions on Vehicular Technology, 60(7), 3386–3407.
[25] Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
[26] Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., & others. (2021). Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1–2), 1–210.
[27] Zeng, Y., Zhang, J., & Letaief, K. B. (2016). Energy-efficient UAV communication with trajectory optimization. IEEE Transactions on Wireless Communications, 16(6), 3747–3760.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Computer Science and Engineering Transactions
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
