Tech Tip #68: Start pre-trained, fine-tune the adapter

LoRA trained 10,000 times fewer parameters with comparable quality.

The problem

Teams fine-tune whole models to adapt them to a domain, needing big GPU clusters and long runs.

The fix you can use today

  • Start from a pre-trained model instead of training from scratch.
  • Fine-tune a small adapter with LoRA, or QLoRA with 4-bit weights to fit on one GPU.
  • Compare quality with full fine-tuning on your own evaluation set.

Researchers at Microsoft showed that LoRA trains 10,000 times fewer parameters and still does as well. GreenCode's own AI uses small, tuned models.

Tools that help

  • PEFT (Hugging Face): parameter-efficient fine-tuning methods such as LoRA, so only small adapter weights are trained
  • CodeCarbon: a Python package that estimates the energy and CO2 of a run from hardware power and regional grid intensity

How GreenCode helps

GreenCode helps tackle this by keeping its own AI lean: small fine-tuned models, retrieval so a model reads only what it needs, each request sent to the smallest part that can do it, and heavy jobs held for cleaner power. More on this.

Tags

#llm#model-efficiency#gpu

Sources

All GreenCode Tech Tips

  • Tip #68
  • For CTOs and technical leads
  • Category AI and machine learning
  • Format short video
GreenCode

Want this checked on your own system? Talk to the GreenCode team.

Get in touch