From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

Synopsis

Large language model inference is a growing part of the environmental footprint of AI systems, yet measuring its energy directly usually needs hardware telemetry, power instruments or infrastructure-specific monitoring. That rules direct measurement out for many comparative studies, early design decisions and sustainability reports. This report sets out a way to estimate inference energy on NVIDIA H100-class GPUs without measuring at run time. The estimator combines transformer arithmetic counts scaled by model size, calibrated memory-traffic factors and hardware-specific energy figures for tensor-core computation and high-bandwidth memory access. It treats prompt processing separately from token-by-token generation, so it can estimate energy for input tokens, output tokens and whole requests. It also splits total energy into computation, parameter access, key-value cache writes and attention reads, showing how energy scales with model size, context length and the number of tokens generated. The estimates are not meant to replace physical power measurement; they are transparent, reproducible approximations with explicit assumptions, suited to comparing models, green coding analysis and design-time evaluation of inference workloads.

Cite this work

Vartziotis, T., Kosteli, R., Vartziotis, E., Dasoulas, G., Keckeisen, M., Skianis, K., Kotsopoulos, S., &amp; Dominici, F. (2026). <i>From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs</i> (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2607.26571