You're using a giant AI model for a tiny job.
General-purpose generative models get used for classification, extraction and routing that smaller or task-specific models do well. A systematic comparison found multi-purpose generative models orders of magnitude more expensive to run than task-specific ones for the same tasks.
A UNESCO report found that small models made for one task can cut energy use by up to 90%. GreenCode's own AI uses small, tuned models for the same reason.
GreenCode helps tackle this by routing each request to the smallest component that can handle it, using small fine-tuned models. Retrieval means a model reads only the data it needs. More on this.
You are using a giant AI model for a tiny job. Big general AI models often sort, tag and route simple requests. Smaller models made for one task do those jobs well. One careful study compared them on the same tasks. The big general models cost far more to run, often many times more. And that cost repeats for every request, for as long as it runs. Here is the fix you can use today. Start with the smallest model, or a classic method, that is good enough. Send most requests to the small model. Use the large model only for the hard cases. Measure the energy per request for each option before you choose. Use tools like the AI Energy Score to compare models, and CodeCarbon for runs. A UNESCO report found small, single-task models can cut energy by up to ninety percent. GreenCode's own AI uses small, tuned models for the same reason. GreenCode helps tackle this by sending each request to the smallest part that can cope. And its models read only the data they need. That was GreenCode Tech Tip number sixteen. Follow for the next tip, and find more on greencode.ai/tech-tips.
Share this
