Optimize model output with prompt engineering
Unit 2 of 7 · Module 5: Optimize generative AI model performance with Microsoft Foundry
Before you start
Short explainers for ideas this lesson builds on. Watch these first if they're new to you.
Read this lesson
The quickest lever
The easiest way to get more out of a model is prompt engineering: crafting and polishing your prompts so the answers come back better, more accurate, and more on topic. It needs no extra infrastructure and no training data, so you can start tinkering right now. Ask a vague question, and you get a vague answer; a clear, well-built prompt shows the model exactly what you're after. A prompt for a chat model usually has four parts. The system message sets the ground rules: who the model is, how it behaves, and what it must not do. The user message is what the person actually typed. Assistant messages are the model's earlier replies, which keep a multi-turn conversation on track. And examples are sample inputs paired with the outputs you want, to show the format. How you build and mix these four pieces decides how well the model does.
The system message
The system message normally goes first, and it outranks everything else in the prompt. Use it to give the assistant a role and limits, set its tone and style, ask for an output format like JSON or bullet points, and add safety and quality rules. It can be one short line, like: you're a helpful AI assistant. Or it can spell out a whole rulebook. For the travel agency's app, it might read: you're a friendly travel advisor for Wayfarer Trips. Only answer questions about travel, hotels, and planning trips. Keep the tone warm and chatty. If something's missing, ask a follow-up question before answering. And list hotel suggestions as bullets, each with the name, the location, and the price range. Keep in mind that a system message nudges the model; it can't force it to obey. So test it, adjust it, and back it up with other safeguards, like content filters and evaluations. A quick checklist: open with the assistant's role, and what a good result looks like. Draw the lines: which topics, actions, and content are off-limits. Describe the output format clearly, and stick to it. And include a plan for when it's unsure, covering requests that are vague, off-topic, or missing details.
Prompt patterns
A few proven patterns help the model respond better. The persona pattern gives the model a point of view to speak from. Ask for a one-sentence description of a CRM product with no persona, and you get a dry, dictionary-style answer. Now add a system message saying it's a seasoned marketing pro writing for technical buyers, and the same request comes back polished and persuasive. A format template spells out the exact shape you want: for each hotel, its name, location, star rating, and nightly price range. Answers then look the same every time, and your app can read them easily. Chain of thought asks the model to work through a problem step by step. For instance: which hotel suits a family of four best, weighing room size, things for kids to do, location, and price? Showing its steps cuts down on wrong answers, and lets you check its logic. A close cousin splits the job into explicit stages, like first pull out the key facts, then answer from them, which helps on complicated tasks. Chain of thought is for non-reasoning models; reasoning models, like the GPT five family and the o-series, already think step by step on their own. With few-shot learning, you show a few examples of input and output, like customer messages labeled booking change or travel information, and the model completes the next one in the same style. One example is called one-shot, and none at all is zero-shot. When a prompt has several parts, fence them off with delimiters, like three dashes, Markdown headings, or XML tags. And beware of recency bias: whatever comes last in a prompt can weigh more, so if the model starts to drift, repeat the key instruction at the end.
Parameters, and when it's enough
You can also adjust the model's parameters. Temperature sets how much randomness goes into the wording. Turn it higher, say to 0.7, and replies get more creative and varied; turn it lower, like 0.2, and they get more focused and predictable. Top p is another randomness dial: it limits the model to the likeliest next tokens, so at 0.9, it only picks from the likeliest tokens that together make up ninety percent of the probability. Change one or the other, but not both at once, and only on models that accept them: reasoning models, like the GPT five family, take a reasoning effort setting instead. In the travel app, you might use 0.2 for factual questions, like whether a hotel has a pool, and 0.7 for dreaming up itineraries. Prompt engineering is always the place to start. It's great for shaping tone, format, and behavior, giving the model specific task instructions, trying ideas quickly with no new infrastructure, and keeping costs low, since there's no training and nothing to store. But it can only go so far. If the model can't access what it needs to know, like your hotel catalog, or it keeps failing to follow a rule no matter how clearly you write it, it's time to consider other strategies.
Facts from Microsoft's documentation: Prompt engineering techniques; System message design for Azure OpenAI (CC BY 4.0), rewritten in Loop's own words and examples. Narration, animation, and quiz by Loop. Microsoft, Azure, and Microsoft Foundry are trademarks of Microsoft. Loop is independent and not affiliated with or endorsed by Microsoft.