Configure retrieval with Foundry IQ
Unit 5 of 7 · Module 10: Build knowledge-enhanced AI agents with Foundry IQ
Before you start
Short explainers for ideas this lesson builds on. Watch these first if they're new to you.
Read this lesson
Connected isn't the same as grounded
Connecting an agent to a knowledge base doesn't guarantee it uses it well. Picture a mountain rescue station on a snowy night, with a dispatcher at the radio. A caller asks where the nearest shelter is, and the dispatcher answers from memory instead of checking the map. Sometimes the answer's right, sometimes it's years out of date, and there's no way to tell which. Agents slip in the same ways: they skip the knowledge base, answer from their training data, forget to say where an answer came from, or guess when the knowledge base has nothing on the topic. The fix isn't more data. It's clear rules for how retrieval should happen, and then testing that the rules actually work.
Rules at two levels
You write those rules in two places. First, the agent's own instructions, the dispatcher's checklist: always search the knowledge base before answering, cite the sources you used, and if the knowledge base doesn't have the answer, say I don't know. Second, the knowledge base itself can carry retrieval instructions, which tell the retrieval planner which source to use for which kind of question, like: use the trail maps for route questions, and the weather feed for conditions. It can also carry answer instructions, which shape the answer it puts together, like: answer in two short sentences. Keep every rule specific enough to check: always cite, is testable, while be helpful, isn't. And when you change a rule, the knowledge base uses it on the very next question.
How hard should it search?
Next, decide how much thinking goes into each search, with the retrieval reasoning effort. Minimal skips the language model entirely: the question goes straight to the sources, which is fastest and cheapest, like checking one map. Low lets a model plan the search and pick which sources to ask. Medium adds another pass, looking again where the first results were thin, like sending out a second search party. You also choose what comes back: extractive data, the raw passages for your agent to reason over, or, in preview, a synthesized answer with its sources. And you can cap each retrieval, with a time limit, a maximum number of documents, and a token budget, so one hard question can't run all night.
Test before you trust
Then run drills before the real calls come in. Open a test conversation in the playground, and ask questions you already know the answers to. Test the easy ones, where one document clearly holds the answer. Test questions that need two sources combined, questions that use different words than the documents do, and vague ones, like, what about benefits, where a good agent asks what you mean. Most important, test questions the knowledge base can't answer, and check that the agent says I don't know instead of inventing something. For every answer, check four things: is it grounded in the retrieved text, does each claim cite a real document that says it, does it actually answer the question, and is it complete, with nothing important left out. When something fails, change one thing at a time, an instruction, a source description, or the reasoning effort, and run the same drill again.
One size doesn't fit every agent
Different agents need different settings, just like different rescue teams. A customer-facing support agent should be strict: answer only from the knowledge base, always cite, and say I don't know whenever it isn't sure, because a wrong answer to a customer costs trust. An internal research assistant can range wider: several sources, a higher reasoning effort, and longer summaries, still with citations, so colleagues can dig further. A specialized expert, say for medical or legal documents, needs precision: a focused set of trusted sources, answers that quote the exact wording, and a clear handoff to a human when a question goes beyond them. When you move to production, keep watching: review real conversations, and track four signals: how often answers cite sources, how often the agent says I don't know, which kinds of questions come in, and whether the retrieved documents really hold the answers. Use what you find to refresh the documents, the instructions, and the search settings. Retrieval isn't set once; it's tuned over time.
Facts from Microsoft's documentation: Create a Knowledge Base; Connect Agents to Foundry IQ Knowledge Bases; What is Foundry IQ? (CC BY 4.0), rewritten in Loop's own words and examples. Narration, animation, and quiz by Loop. Microsoft, Azure, and Microsoft Foundry are trademarks of Microsoft. Loop is independent and not affiliated with or endorsed by Microsoft.