Extract personally identifiable information (PII)
Unit 5 of 7 · Module 16: Analyze text with Azure Language in Foundry Tools
Before you start
Short explainers for ideas this lesson builds on. Watch these first if they're new to you.
Read this lesson
Before you share
Some details point straight at one person: a name, a home address, a phone number, an email address, an ID number, a credit card number. These are called personally identifiable information, or PII. Before you share customer feedback, legal papers or medical notes, you often have to take them out. Health details have their own name, PHI, protected health information. Azure Language can find these details for you, and it can also redact them, which means masking them so they can't be read.
Finding the details
Here's a support ticket from a Wayfarer guest. Hi, this is Priya Nair. Call me on five five five, zero one four two, or email priya at nair dot example. I paid with card four one one one, and so on. You send it the usual way, as a list of documents, to recognize underscore PII underscore entities, and you add language equals e n to say the text is in English. If you leave it out, English is assumed. Like entity recognition, it takes at most five documents per request. For each document you loop over the entities and print three things: the text, its category, and the confidence score. Priya Nair comes back as a Person, then a PhoneNumber, an Email, and a CreditCardNumber, each with a score.
The redacted copy
Every result also carries redacted underscore text. It's the same message, with each detail it found covered by asterisks, one per character, so the length stays the same. That's the copy you can safely store or share. Two things to watch. You can pass a categories filter to look only for, say, phone numbers and emails, and leave everything else alone. And the service masks everything it detects, whatever the score, so a low confidence guess gets hidden too. If you need to be pickier, check the scores yourself.
Four ways to mask
Newer versions of the REST API let you choose how the redacted text looks. Character mask is the default: asterisks, or a character you pick. Entity mask swaps each detail for its category, like Person one, or PhoneNumber one. Synthetic replacement, still in preview, swaps in a realistic fake name or number. And no mask skips the redacted text completely, when you only want the list. The newest preview also has a confidence threshold, so hits that score below it are left unmasked. The Python library in this module gives you the asterisk version.
Facts from Microsoft's documentation: What is PII detection in Azure Language?; Identify and extract PII from text; PII and PHI entity categories (CC BY 4.0), rewritten in Loop's own words and examples. Narration, animation, and quiz by Loop. Microsoft, Azure, and Microsoft Foundry are trademarks of Microsoft. Loop is independent and not affiliated with or endorsed by Microsoft.