LLM fine-tuning is continued training of a pre-trained large language model on your own examples, so it learns a specific behaviour — an output format, a classification, a tone or a specialised task. Unlike RAG, the change is stored in the model's weights.
Fine-tuning needs a curated dataset and an evaluation set that proves the tuned model beats the base model plus good prompting. Parameter-efficient methods such as LoRA and QLoRA make it affordable on modest hardware.
Read more about LLM fine-tuning
- LLM Fine-Tuning vs RAG: Which Should You Use?RAG supplies facts at question time; fine-tuning changes how a model behaves. How to choose between them, what fine-tuning needs, and how to prove it worked.
- Fine-Tuning LLMs in Depth: LoRA, QLoRA, Training Data, Evaluation and ServingHow LoRA and QLoRA work, GPU memory maths, building training data, a working SFT pipeline, hyperparameters, DPO preference tuning, evaluation and multi-adapter serving.
How Marquedata helps
Data Science & MLData science, machine learning, model training and LLM fine-tuning — from experiment to monitored production model.
Related terms
Have a Data, AI or Customer Intelligence Challenge?
Let's explore what you can build with your data.
Have an idea for an intelligent product?