Compare and combine optimization strategies
Unit 5 of 7 · Module 5: Optimize generative AI model performance with Microsoft Foundry
Read this lesson
Two directions
You've seen prompt engineering, RAG, and fine-tuning on their own. They aren't mutually exclusive. They're complementary, and you can combine them to meet different goals. They address two dimensions of performance. Optimize the context when the model is missing specialist knowledge and accuracy is what counts: RAG handles that, fetching the right data from outside sources, like handing a new hire the company handbook. Or optimize the model itself when format, style, tone, and consistency are the goal: fine-tuning does that by training on examples of the output you want, like coaching that new hire until the house style is second nature. Prompt engineering sits underneath both, like the briefing you give on day one. It tells the model how to act and what to pay attention to; you add RAG or fine-tuning on top only when prompts stop being enough.
The trade-offs
Each strategy trades time, complexity, and cost. Prompt engineering is low on all three, with only per-token costs. It's best for guiding tone, format, and behavior, iterating quickly, and giving instructions and examples. You can start right away, with no infrastructure changes. But longer prompts use more tokens per request, the model might not follow complex instructions consistently, and prompts can't give it information it wasn't trained on. RAG is medium on all three: search infrastructure, storage, and per-token costs. It's best for factual accuracy, domain-specific knowledge, and dynamic or frequently changing data. But you need a search service, an index to create and maintain, and embeddings to process. And the answers are only as good as your index, and how well your data was split into chunks. Fine-tuning is high on all three: training compute, model hosting, and per-token costs. It gives the most consistent behavior, because the patterns live in the model's weights, and shorter prompts can lower per-request costs. It's best for behavioral consistency, style enforcement, reducing prompt length, and distillation. But it has the highest upfront investment, and may need retraining when the base model is updated or your requirements change.
Better together
The most effective apps often combine strategies. Prompt engineering plus RAG is the most common pairing. The system message makes the model a travel advisor and sets the format, and RAG pulls real hotel names and prices from the catalog. Together they cover how the model should act, and what it needs to know. Prompt engineering plus fine-tuning suits a specific, consistent style. The fine-tuned model always speaks in the brand voice, and the system message adds session-specific instructions, like giving priority to a seasonal promotion. RAG plus fine-tuning gives you both factual grounding and consistent behavior: brand voice and structured format from the fine-tuned model, with up-to-date pricing and availability from RAG. And for the most demanding apps, use all three. Fine-tuning ensures consistent style and format. RAG provides accurate, current domain knowledge. And the prompt adds the instructions and guardrails for this particular conversation.
Start simple
To decide, start simple, and add complexity only when you need it. Begin with prompt engineering: try system messages, few-shot examples, and parameter tweaks, then check whether the results are good enough. If accuracy matters, and the model needs specific, current, or private data, add RAG with Azure AI Search. If consistency matters, and the model doesn't reliably keep the style, tone, or format you want despite detailed prompts, fine-tune it with representative examples. Then combine as needed, based on what your app requires. Not every app needs all three. This step-by-step approach avoids unnecessary cost and complexity, while still reaching the level of optimization your app needs.
Facts from Microsoft's documentation: Prompt engineering techniques; Retrieval augmented generation (RAG) and indexes in Microsoft Foundry; Microsoft Foundry fine-tuning considerations (CC BY 4.0), rewritten in Loop's own words and examples. Narration, animation, and quiz by Loop. Microsoft, Azure, and Microsoft Foundry are trademarks of Microsoft. Loop is independent and not affiliated with or endorsed by Microsoft.