Tech Tip #5: Forecast load before you scale

A simple maths model beat neural networks at predicting server load.

Video created by explainapaper.com

The problem

Reactive autoscaling adds capacity after latency has already risen, so teams keep big safety margins running all day. Proactive scaling can size ahead of demand, and it doesn't need a heavy model.

The fix you can use today

  • Forecast CPU from request volume with a lightweight model.
  • Try exponential smoothing before anything neural.
  • Scale ahead of predicted peaks and shrink the standing safety margin.

GreenCode researchers found that a simple maths model predicted load better than neural networks.

Tools that help

  • KEDA: event-driven autoscaling for Kubernetes, including scaling a workload to zero when there is nothing to do
  • Kepler: exports energy estimates per pod and container on Kubernetes, using eBPF and hardware counters

How GreenCode helps

GreenCode helps tackle this by mapping your running systems and tracing their energy and CO2e to the services that cause it. It shows services running hot, and services barely running at all. More on this.

Tags

#kubernetes#autoscaling#cloud

Sources

Transcript

A simple maths model beat neural networks at predicting server load. Most autoscaling waits until your servers are already struggling. It adds machines after the slowdown has started. So teams keep extra machines running all day, just in case. That spare capacity uses power around the clock. The better way is to scale before the busy moment arrives. And you don't need a heavy AI model to see it coming. GreenCode researchers found that a simple maths model predicted load better than neural networks. They compared twenty-nine ways to forecast, on real traffic from working systems. The most accurate was a simple method called exponential smoothing. It beat the big neural network models. Here is the fix you can use today. Forecast processor load from the number of requests you get. Try a simple model like exponential smoothing first. Scale up ahead of the peaks you predict, using tools like KEDA. Then shrink the spare machines you keep running all day. Check each Kubernetes pod's energy afterwards, using tools like Kepler. GreenCode helps tackle this by mapping your running systems and the energy they use. It shows services running hot, and services barely running at all. That was GreenCode Tech Tip number five. Follow for the next tip, and find more on greencode.ai/tech-tips.

All GreenCode Tech Tips · All GreenCode videos

  • Tip #5
  • For Ops and infrastructure
  • Length 1:24
  • Category Containers, Kubernetes and serverless
GreenCode

Want this checked on your own system? Talk to the GreenCode team.

Get in touch