Map potential harms
Unit 3 of 8 · Module 6: Implement a responsible generative AI solution in Microsoft Foundry
Read this lesson
Identify
The first stage is to map the potential harms that could affect your planned solution. It has four steps: identify potential harms, prioritize them, test and verify the prioritized harms, and document and share the verified ones. Which harms are relevant depends on several factors: the specific services and models that generate the output, and any fine-tuning or grounding data used to customize it. Common types include content that's offensive, pejorative, or discriminatory; content with factual inaccuracies; and content that encourages or supports illegal or unethical behavior. To understand the known limitations of your services and models, read their documentation, like the transparency note for Azure OpenAI, or a model developer's system card, such as OpenAI's card for GPT-4. The Microsoft Responsible AI Impact Assessment Guide, and its template, help you document potential harms. And review the guidelines for every resource you use.
Prioritize
Next, for each harm, assess how likely it is to occur, and how big the impact would be if it does. Then put the most likely and most impactful harms first, so you focus on the most harmful risks. Prioritizing must consider the intended use of the solution and the potential for misuse, and it can be subjective. Imagine a smart kitchen copilot that helps chefs and home cooks with recipes. It might give inaccurate cooking times, leaving food undercooked and making people ill. Or, when prompted, it might give a recipe for a lethal poison made from everyday ingredients. The poison has the higher impact. But inaccurate cooking times are likely to happen far more often than someone explicitly asking for poison. So the final priority is a team discussion, which may involve policy or legal experts.
Test, then share
With a prioritized list, test your solution to verify whether the harms occur, and under what conditions. Testing may also reveal harms you hadn't identified, so add them to the list. A common approach is red teaming: a team deliberately probes the solution for weaknesses, and tries to produce harmful results. For the kitchen copilot, testers might ask for poison recipes, or quick recipes with ingredients that must be thoroughly cooked. Document and review the red team's successes, to judge how likely harmful output really is. Red teaming comes from security testing, so extending it to harmful content builds on, and complements, your existing cybersecurity practices. Finally, document the evidence for each harm, share it with stakeholders, and keep the prioritized list up to date as you find new harms.
Facts from Microsoft's documentation: Overview of Responsible AI practices for Azure OpenAI in Foundry Models; Planning red teaming for large language models (LLMs) and their applications (CC BY 4.0), rewritten in Loop's own words and examples. Narration, animation, and quiz by Loop. Microsoft, Azure, and Microsoft Foundry are trademarks of Microsoft. Loop is independent and not affiliated with or endorsed by Microsoft.