Tech Tip #12: Faster is not always greener

Your speed-up might not save the energy you think.

Video created by explainapaper.com

The problem

Teams treat speed as a stand-in for energy, but the two do not move together. On llama.cpp, GreenCode research cut GPU energy by 4.7% while speed rose only 3%: energy and time told different stories.

The fix you can use today

  • Measure energy directly, not only runtime.
  • Compare before and after on the same machine under the same load.
  • Report both numbers; a change can be faster and hungrier, or slower and leaner.

GreenCode researchers found that speed gains and energy savings are not the same thing, so they measure both.

Tools that help

  • green-languages (GreenCode researchers): a command-line tool that measures the energy of whole programs or marked sections of code, across languages
  • GreenCode Constitution profiler (profile.sh) (GreenCode researchers): a script from the Constitution tooling that measures the energy of a command from processor counters, GPU sensors and an optional smart plug, and writes the result as JSON
  • perf (RAPL counters): Linux perf reads the processor's RAPL energy counters, e.g. `perf stat -e power/energy-pkg/`

How GreenCode helps

GreenCode helps tackle this by running a load test built from your system's own endpoints. It records energy as well as speed, and ranks the hotspots by the energy they use. More on this.

Tags

#measurement#profiling#llm#gpu

Sources

All GreenCode Tech Tips

  • Tip #12
  • For Developers
  • Category Measuring energy
  • Format short video
GreenCode

Want this checked on your own system? Talk to the GreenCode team.

Get in touch