Large language model inference is a growing part of the environmental footprint of AI systems, yet measuring its energy directly usually needs hardware telemetry, power instruments or infrastructure-specific monitoring. That rules direct measurement out for many comparative studies, early design decisions and sustainability reports. This report sets out a way to estimate inference energy on NVIDIA H100-class GPUs without measuring at run time. The estimator combines transformer arithmetic counts scaled by model size, calibrated memory-traffic factors and hardware-specific energy figures for tensor-core computation and high-bandwidth memory access. It treats prompt processing separately from token-by-token generation, so it can estimate energy for input tokens, output tokens and whole requests. It also splits total energy into computation, parameter access, key-value cache writes and attention reads, showing how energy scales with model size, context length and the number of tokens generated. The estimates are not meant to replace physical power measurement; they are transparent, reproducible approximations with explicit assumptions, suited to comparing models, green coding analysis and design-time evaluation of inference workloads.
Vartziotis, T., Kosteli, R., Vartziotis, E., Dasoulas, G., Keckeisen, M., Skianis, K., Kotsopoulos, S., & Dominici, F. (2026). <i>From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs</i> (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2607.26571
Vartziotis, T., Kosteli, R., Vartziotis, E., Dasoulas, G., Keckeisen, M., Skianis, K., Kotsopoulos, S., & Dominici, F. (2026). <i>From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs</i> (Version 1). arXiv. https://doi.org/10.48550/ARXIV.2607.26571Vartziotis, T. <i>et al.</i> (2026) ‘From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs’. arXiv. Available at: https://doi.org/10.48550/ARXIV.2607.26571.T. Vartziotis <i>et al.</i>, ‘From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs’, 2026, <i>arXiv</i>. doi: 10.48550/ARXIV.2607.26571.Vartziotis, Tina, Rodopi Kosteli, Elli Vartziotis, et al. 2026. ‘From Tokens to Watt-Hours: Analytical Energy Estimation for LLM Inference on Modern GPUs’. Version 1. Preprint, ArXiv. https://doi.org/10.48550/ARXIV.2607.26571.Vartziotis T, Kosteli R, Vartziotis E, Dasoulas G, Keckeisen M, Skianis K, et al. From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs [Internet]. arXiv; 2026. Available from: https://arxiv.org/abs/2607.26571@misc{https://doi.org/10.48550/arxiv.2607.26571, doi = {10.48550/ARXIV.2607.26571}, url = {https://arxiv.org/abs/2607.26571}, author = {Vartziotis, Tina and Kosteli, Rodopi and Vartziotis, Elli and Dasoulas, George and Keckeisen, Michael and Skianis, Konstantinos and Kotsopoulos, Sotirios and Dominici, Francesca}, keywords = {Machine Learning (cs.LG), Software Engineering (cs.SE), FOS: Computer and information sciences}, title = {From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs}, publisher = {arXiv}, year = {2026}, copyright = {Creative Commons Attribution 4.0 International} }