Data safety for AI: anonymisation, classification labels, redaction, masking and minimisation

Why this matters

Lesson 13 protected data from people who should not be able to reach it. This lesson deals with the harder case: data the model is supposed to reach, and which then comes out somewhere it should not. A support chatbot that can read the customer database will, sooner or later, be asked a question whose honest answer includes another customer's address. A model fine-tuned on support tickets can repeat a phone number from one of them. Encryption does nothing in either case, because the data was decrypted for an authorised path.

The controls here change the data itself before the model, the index or the log sees it. That matters more for AI than for an ordinary application for two reasons. First, a model has no reliable way to keep a secret: whatever is in its context window can be coaxed into its output, and whatever was in its training data may be memorised. Second, AI pipelines copy data enthusiastically — into training sets, embeddings, caches and logs — and every copy inherits the exposure of the original unless something removes it on the way.

Sensitive information disclosure is LLM02 in the 2025 OWASP list, and these controls are the main answer to it. On the exam, the scenarios turn on choosing the right one: whether the data needs to be gone, hidden from some viewers, replaced with something reversible, or never collected at all.

The lesson

Minimisation first: data that never reaches the model cannot leak from it

Every other control in this lesson has a failure rate. Minimisation does not, because there is nothing left to fail. It comes first in the order of preference, and GDPR's data minimisation and data protection by design principles apply directly to AI systems: process only what the purpose needs.

Minimisation for AI happens at four points.

Collection and training. A fine-tuning set for a code assistant needs code, not the commit authors' email addresses. A ticket classifier needs the ticket text, not the customer's account number. Drop columns and fields before the data leaves its source system, not after it has been copied into a notebook.

Retrieval. The RAG app over HR documents should index the policy library, not the HR case files, unless answering questions about cases is the actual purpose. Scope the index to the use case, and scope each retrieval to the user's rights (lesson 11). A retrieval that pulls ten chunks when three suffice puts seven more chunks of content into the context window, the response and the log.

Context assembly. When an application builds a prompt from records, pass the fields the task needs. A chatbot answering "where is my order?" needs the order status and the delivery date; it does not need the full customer record that the developer found convenient to serialise.

Retention. Prompts, responses and conversation history should be kept for a defined period and then deleted. Data kept "in case it is useful for training later" is data kept without a purpose.

The scenario signal for minimisation is a requirement phrased as reducing exposure at source, or a finding that a model, index or log contained data the use case never needed. The answer is to stop collecting it, not to protect it better.

Anonymisation, and why re-identification is easier than teams expect

Anonymisation irreversibly removes the link between data and a person, so that nobody — including the organisation that holds the data — can reasonably identify them. Done properly, anonymised data falls outside data protection law because it is no longer personal data. Pseudonymisation, which replaces identifiers with tokens while keeping a way back, does not: pseudonymised data remains personal data under GDPR for anyone who holds, or can reasonably obtain, the means to re-identify it. Confusing the two is an examinable mistake.

The difficulty is that removing names and identifiers is not anonymisation. A record is often unique on a handful of ordinary attributes — postcode, date of birth and sex together single out many people — so a dataset stripped of names can be re-identified by joining it to another dataset that has them. These quasi-identifiers are what anonymisation has to deal with, and free text is worse still: a support ticket that mentions "the branch manager in our Leeds office" identifies someone without containing a single field anyone would label personal.

Techniques you should recognise:

• Generalisation and suppression: replace exact values with ranges (age band, postcode district) and drop rare combinations.
• k-anonymity: every record is indistinguishable from at least k−1 others on its quasi-identifiers. Useful, but a group whose members share the same sensitive value still discloses it.
• Differential privacy: add calibrated noise, to statistics or during training, so that the result reveals little about whether any one person was included. It gives a measurable guarantee, traded against accuracy.
• Synthetic data: generate records with the statistical shape of the real ones. Helpful for testing and some training, but a generator trained on real data can reproduce real records, so it needs the same scrutiny.

AI adds two re-identification routes. A model trained on supposedly anonymised data can still be probed by membership inference (lesson 19) to reveal whether a given person was in the training set. And a model is itself a re-identification engine: given a few "anonymous" details and access to the web or another dataset, it can do the linking for the attacker. Treat any claim that a training set is anonymised as something to test, not to accept.

Redaction and masking of prompts and of retrieved documents

Both hide sensitive values; they differ in what survives.

Redaction removes the value and replaces it with nothing or a marker: [REDACTED], or better, a typed placeholder such as [EMAIL] so the model still understands the sentence. It is irreversible in the redacted copy.

Masking obscures part or all of a value while keeping its shape: a card number shown as its last four digits, a name replaced with a consistent fake. Static masking produces a masked copy of a dataset, typically for development, testing or training. Dynamic masking applies at read time according to who is asking, so a support agent sees a masked account number and a fraud analyst sees the full one. Tokenisation — replacing a value with a token that a separate vault can reverse — is the reversible variant, and it is pseudonymisation, not anonymisation.

Where these sit in an AI system:

• Inbound prompts. A gateway or prompt firewall (lesson 10) detects personal data, credentials and secrets in what users type and redacts or tokenises it before the prompt reaches a hosted model. With tokenisation, the gateway can swap the real values back into the response for the user, so the provider never sees them.
• Retrieved documents. Redact at indexing time when a value should never be answerable at all, such as national insurance numbers in HR documents. Mask dynamically at retrieval time when some users are entitled to the value and others are not.
• Outbound responses. An output filter catches values that slipped through, including ones the model produced from memorised training data.
• Logs. Lesson 15 treats sanitising prompt logs; the same detectors serve.

The limits are detection limits. Pattern detectors are good at structured values — card numbers, email addresses, keys with a known format — and poor at names, addresses and context in free text. Named-entity recognition models help and still miss things. Redaction should therefore be one layer, not the only one, and its miss rate is worth measuring against a sample with known contents (lesson 16 covers auditing against a known answer set).

Classification labels that follow data into training sets and indexes

A classification label records how sensitive data is and what may be done with it. You know the scheme; the AI problem is propagation. When a confidential document becomes chunks, the chunks become embeddings, the embeddings answer a query, and the answer lands in a log, the label is usually lost at the first step. The vector store then holds confidential content with no marking, and every downstream decision is made blind.

Labels need to travel as metadata:

• At ingestion, copy the source document's label onto every chunk and embedding, along with its owner and access groups. Most vector stores support metadata filters, and the label becomes a filter at query time.
• Into training sets, record the highest classification of any record included. A model inherits the classification of its most sensitive training data, because it may reproduce it.
• Onto outputs, where the response is derived from labelled sources. A summary of three confidential documents is confidential.
• Into routing decisions, so data above a given label can only go to an approved model — an internal model rather than a public hosted one, for example, or a hosted model under a contract with no retention.

Labels also drive the other controls: they tell the redaction layer what to look for, the retention job what to keep, and the access layer who may see it. Unlabelled data in an AI pipeline is the scenario's finding; inheriting the label is the fix.

Choosing between these controls for the scenario in front of you

Ask what the use case needs to keep, and work down:

• The use case does not need the data — minimise. Do not collect, index or pass it.
• It needs the patterns but never the individuals, for training or analytics — anonymise, and test the result for re-identification.
• It needs to link records to people, but the model does not need to see who — pseudonymise or tokenise before the model and restore afterwards.
• Some viewers may see the value and others may not — dynamic masking at retrieval or display, driven by classification labels and the user's rights.
• The value must never appear in this copy — redact.
• You need to know where sensitive data is, so that all of the above can be applied — classify and label first, because none of the others can be aimed without it.

Two traps recur. An answer that masks data in the user interface while the full value remains in the prompt, the vector store and the log has protected the screen, not the data. And an answer that calls pseudonymised or tokenised data "anonymised" has misdescribed a reversible control as an irreversible one, which matters for both law and risk.