Load Balancing vs Load Leveling vs Load Shedding
Techniques to Optimize and Protect System Performance
They might all deal with 'load,' but balancing it, leveling it, and shedding it are totally different stories when it comes to keeping systems running smoothly.
Load Balancing
Spreads incoming traffic across nodes to avoid bottlenecks.
Load Balancing is the process of distributing incoming network traffic across multiple servers. It ensures no single server becomes overwhelmed.
𝗕𝗲𝗻𝗲𝗳𝗶𝘁𝘀
→ Evenly distributes requests to ensure consistent response times.
→ Scales horizontally during peak hours (e.g., market open/close).
→ Prevents single points of failure with smart routing.
𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀
It distributes traffic across servers to prevent overload and maintain steady performance. Smart routing helps avoid bottlenecks and critical failures. It works best when resources are healthy and scalable.
Load Leveling
Smooths out spikes by queuing work for later processing.
Load Leveling is the process of managing peaks in demand by buffering and smoothing out request flows across time. It helps prevent service overload and ensures stable performance, especially during fluctuating or high-traffic periods.
𝗕𝗲𝗻𝗲𝗳𝗶𝘁𝘀
→ Buffers request peaks to avoid overload.
→ Decouples services for better modularity(order matching from settlement).
→ Helps manage delays in a way that doesn't harm long-term system health.
𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀
It buffers incoming requests to ensure a smooth flow and reduce the risk of system failure during traffic spikes. With appropriate delays, services can handle load efficiently without harming performance.
Load Shedding
Drops non-critical requests to keep the core alive.
Load Shedding is when a system reduces incoming tasks to avoid overload. It helps keep important functions running by delaying or dropping less critical tasks.
𝗕𝗲𝗻𝗲𝗳𝗶𝘁𝘀
→ Protects the system from crashing during peak load times.
→ Keeps critical services available while dropping non-essential tasks.
→ Helps manage resource usage in high-stress scenarios.
𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀
It helps keep the system stable by dropping or delaying less important tasks when it's too busy, avoiding a complete failure. It makes sure to prioritize essential tasks to avoid critical service interruptions.
Where It All Comes Together
In a stock exchange system:
• Balance incoming traffic to keep nodes healthy.
• Shed non-critical updates during volatile spikes.
• Level settlement processing using queues to avoid crash.
Where Teams Often Get It Wrong
• Relying only on balancing - when all nodes are overloaded, it fails.
• Over-shedding - leads to lost revenue and frustrated users.
• Leveling without limits - queues grow endlessly, latency explodes.
Load isn’t the problem. Mismanaging it is.
