Development of a Load Balancing Framework for Microservices Architecture Using Reinforcement Learning

Abstract

The increasing adoption of microservices architecture has improved the scalability, modularity, and maintainability of cloud-native applications. However, the dynamic and heterogeneous nature of microservices workloads creates significant challenges for conventional load-balancing mechanisms, which typically rely on predefined routing rules and have limited ability to respond to changing runtime conditions. This study develops and evaluates RL-MicroLB, an adaptive load-balancing framework that uses reinforcement learning to dynamically distribute requests among microservice instances. The framework employs Proximal Policy Optimization (PPO) and integrates Kubernetes, Istio Service Mesh, Prometheus, Locust, and StableBaselines3 within a closed loop traffic management architecture. Runtime performance measurements are collected through Prometheus and supplied as observations to the PPO agent. The agent determines traffic-routing weights, which are dynamically enforced through Istio VirtualService resources, while subsequent system measurements provide feedback for continued decision-making. The framework was implemented using the Sock Shop microservices application and evaluated under varying workload conditions. Experimental results showed that RL-MicroLB reduced P95 latency from 420 ms under Round Robin to 285 ms, representing a 32.1% reduction. Average response time decreased from 240 ms to 185 ms, CPU utilization variance decreased from 0.18 to 0.07, CPU balance score increased from 0.72 to 0.91, and throughput increased from 820 to 845 requests per second. Comparative evaluation against Least Connections, Weighted Round Robin, and Deep Q-Network further showed that PPO achieved the best overall performance among the evaluated approaches under the experimental conditions.