Fine-tune a model for consistent behavior
Unit 4 of 7 · Module 5: Optimize generative AI model performance with Microsoft Foundry
Before you start
Short explainers for ideas this lesson builds on. Watch these first if they're new to you.
Go deeper
Unlocks when you've watched the lesson.
Read this lesson
Specializing a generalist
Prompt engineering steers how the model behaves, and RAG grounds it in facts. But sometimes its replies still wobble in style, tone, or format. If the model ignores your instructions, or follows them only some of the time, even with a detailed system message and few-shot examples, fine-tuning may be the next step. Fine-tuning takes a model that's already been trained, and trains it a little more on a smaller dataset built for your task. That nudges the model's internal weights, so its answers follow the patterns in your examples. Foundation models like GPT-4o learned from huge amounts of general data: think of a suit bought off the rack. Fine-tuning is the tailor: same cloth, the model's broad language skills, but a new cut that fits the way your examples answer. Under the hood it uses LoRA, short for low-rank adaptation, which captures the weight changes in a much smaller, compressed form. Rather than retraining every parameter, LoRA adjusts only a small set of important ones, so training is faster and cheaper, and quality holds up. And compared with building a model from scratch, fine-tuning takes less time, less computing power, and far less data.
When to fine-tune
Fine-tuning earns its place when prompts alone can't give you the consistency you need. First: a steady style and tone. Say the travel agency wants every reply warm and encouraging, in short paragraphs, every time. Second: exact output formats, like JSON that must match a schema, when few-shot examples keep slipping. Third: shorter prompts. A long system message packed with examples costs tokens and slows things down; fine-tuning bakes those patterns in, so each prompt can be lean. Fourth: distillation. You gather answers from a big, strong model, and use them to fine-tune a smaller one that gets close in quality, for less money and less waiting. And fifth: better tool use, where training on tool examples helps the model choose the right tool, and fill in its parameters. One firm rule: fine-tuning is an advanced move. Always measure how a standard model performs first. Without that baseline, you can't tell whether fine-tuning helped, or made things worse.
Techniques and training data
Microsoft Foundry gives you several fine-tuning techniques. Supervised fine-tuning, or SFT, learns from labeled pairs of prompts and ideal responses, and works best when a task has clear right answers. Reinforcement fine-tuning, or RFT, improves through rounds of feedback, with a grader that scores the responses and rewards the better ones. It fits complex or open-ended tasks with many possible answers, where you want the model to reason better. Direct preference optimization, or DPO, teaches the model what people prefer, using pairs of responses: one preferred, one not. It needs less computing than classic reinforcement learning, and aligns the model just as well. You can also stack them: SFT first to customize the model, then DPO to bring it closer to what people prefer. Training data lives in a JSONL file, with one example on each line. For chat models, each example is a whole conversation, with system, user, and assistant messages, like a friendly travel advisor suggesting the Algarve, or listing budget hotels in Tokyo. Keep the same system message across your examples. Pick high-quality examples that cover the situations you expect. Aim for hundreds at least; more usually helps. And make every assistant reply show exactly the style, format, and tone you're after. Don't drop the system message from your examples, since that tends to make a less accurate model. And use that same system message when you deploy the fine-tuned model.
The challenges
Fine-tuning brings real cost and complexity, so weigh it up first. Training costs: you pay once to train, then every hour to host your custom model. Data quality: poor or lopsided data leads to overfitting, underfitting, or bias. Maintenance: when your data changes, or a new base model comes out, you may need to train again. Experimentation: settling on hyperparameters, like the number of epochs, batch size, and learning rate, takes trial and error. And model drift: tailor it too tightly, and the model gets worse at general language work outside your niche. For the travel agency, fine-tuning means every reply matches the brand's voice and layout, without long system messages. But the team has to set that against the work of preparing training data, and looking after the model over time.
Facts from Microsoft's documentation: Customize a model with fine-tuning; Microsoft Foundry fine-tuning considerations (CC BY 4.0), rewritten in Loop's own words and examples. Narration, animation, and quiz by Loop. The coding lab is based on code samples in Microsoft's Azure AI documentation (MicrosoftDocs/azure-ai-docs, MIT License; notice). Microsoft, Azure, and Microsoft Foundry are trademarks of Microsoft. Loop is independent and not affiliated with or endorsed by Microsoft.