The problem

General-purpose generative models get used for classification, extraction and routing that smaller or task-specific models do well. A systematic comparison found multi-purpose generative models orders of magnitude more expensive to run than task-specific ones for the same tasks.

The fix you can use today

• Start with the smallest model, or a classical method, that meets the quality bar.
• Route requests: small model by default, large model only for the hard cases.
• Measure energy per request for each option before you choose.

A UNESCO report found that small models made for one task can cut energy use by up to 90%. GreenCode's own AI uses small, tuned models for the same reason.

Tools that help

• AI Energy Score (Hugging Face): comparable energy ratings for AI models on common tasks
• CodeCarbon: a Python package that estimates the energy and CO2 of a run from hardware power and regional grid intensity
• nvidia-smi: reports the power draw of NVIDIA GPUs from the card's own sensors

How GreenCode helps

GreenCode helps tackle this by routing each request to the smallest component that can handle it, using small fine-tuned models. Retrieval means a model reads only the data it needs. More on this.

Tags

#llm#model-efficiency#inference

Sources

• What policy makers can do (Luccioni, Jernite, Strubell, FAccT 2024)
• GreenCode State of the Art Review, D2.1 (2025), PDF p. 53: small-first model selection and routing; reserve larger models for genuinely hard cases
• UNESCO, small task-specific models can cut energy use by up to 90%
• Green Software Foundation, Green Software Patterns: Right-sized, energy-efficient AI models

All GreenCode Tech Tips