Tech Tip #51: Ask AI for less

Every extra word in or out of a chatbot costs energy.

Video created by explainapaper.com

The problem

Staff paste whole documents into prompts and accept long, padded answers by default, and every token is paid for in compute.

The fix you can use today

  • Teach short, structured prompts that ask for exactly what is needed.
  • Set output length limits in applications you build.
  • Summarise or retrieve relevant parts instead of pasting everything.

A UNESCO report found that shorter prompts and answers can cut AI energy use by over 50%. GreenCode's own AI reads only what it needs for the same reason.

Tools that help

  • AI Energy Score (Hugging Face): comparable energy ratings for AI models on common tasks
  • CodeCarbon: a Python package that estimates the energy and CO2 of a run from hardware power and regional grid intensity

How GreenCode helps

GreenCode helps tackle this by keeping its own AI lean: small fine-tuned models, retrieval so a model reads only what it needs, and each request sent to the smallest part that can do it. More on this.

Tags

#llm#training

Sources

Transcript

Every extra word in or out of a chatbot costs energy. Staff paste whole documents into prompts. They accept long, padded answers by default. And every word in and out is paid for in computing power. Energy use rises with longer prompts, longer answers and bigger models. So prompt habits are a cost lever your organisation can teach. Here is the fix you can use today. First, teach short, clear prompts that ask for exactly what is needed. Next, set limits on answer length in the AI tools you build. Then summarise, or fetch only the parts that matter, instead of pasting everything. Compare models and runs, using tools like AI Energy Score and CodeCarbon. A UNESCO report found shorter prompts and answers can cut AI energy by over half. GreenCode's own AI reads only what it needs, for the same reason. GreenCode helps tackle this by keeping its own AI lean. It uses small tuned models, and sends each job to the smallest part that works. That was GreenCode Tech Tip number fifty-one. Follow for the next tip, and find more on greencode.ai/tech-tips.

All GreenCode Tech Tips · All GreenCode videos

  • Tip #51
  • For CTOs and technical leads
  • Length 1:11
  • Category AI and machine learning
GreenCode

Want this checked on your own system? Talk to the GreenCode team.

Get in touch