What is Fine-tuning?

Fine-tuning takes a general-purpose pre-trained model (like GPT-4 or Llama) and continues its training on a smaller, curated dataset relevant to a specific domain, task, or style. The result is a model that has adapted its weights to better handle that particular context — producing outputs more consistent with the training examples.

Fine-tuning is often compared to RAG (Retrieval-Augmented Generation) as approaches to making an LLM more useful for a specific business context. The key distinction: fine-tuning changes what the model knows and how it behaves, while RAG changes what information it has access to at query time. Fine-tuning is better for adapting tone, style, or reasoning patterns; RAG is better for keeping knowledge current and grounding responses in specific documents.

For most business use cases, RAG is the right starting point — it's cheaper, faster to iterate, and doesn't require maintaining a fine-tuned model as the underlying LLM is updated. Fine-tuning makes more sense when you need very consistent output formatting, domain-specific reasoning that can't be achieved with prompting alone, or when inference cost at scale makes a smaller fine-tuned model economically attractive.