Data processing for AI: cleansing, lineage, provenance and the data types a model learns from

Why this matters

A model is a compressed summary of its data. Whatever was in the training set, whether errors, bias, personal data, a competitor's copyrighted text or an attacker's planted examples, is now in the model's behaviour, and it cannot be patched out the way a vulnerable library can. Removing a record's influence from trained weights generally means retraining. So the cheapest place to stop most AI data problems is before training, in the processing stages this lesson covers.

For a security professional this shifts the emphasis from protecting data to being able to vouch for it. Confidentiality still matters, but the question an incident, an auditor or a regulator will ask is different: where did this data come from, who changed it, which models learned from it, and can you prove it? A team that cannot answer cannot scope a poisoning incident, honour a deletion request or defend a licensing complaint.

On the exam, expect scenarios that describe a model behaving badly and ask which data practice would have prevented it, or that describe an investigation and ask which record would let you scope it. The terms in this lesson sound alike; the scenarios usually turn on telling them apart.

The lesson

Cleansing and verification, and what an unverified training set does to a model

Data cleansing makes data fit to train on: removing duplicates, repairing or dropping malformed records, handling missing values, normalising formats, and removing content that should not be there at all, such as personal data the model does not need or text outside the task's scope. Deduplication has a security benefit of its own: records that appear many times are far more likely to be memorised and repeated verbatim.

Data verification checks that data is what it claims to be. That means validating records against the expected schema, auditing a sample of labels against the source, checking hashes or signatures on datasets received from outside, confirming that the licence and consent cover this use, and looking for statistical oddities, such as a cluster of near-identical examples all carrying the same unusual label.

The two are different jobs. Cleansing improves data; verification tests trust in it. A perfectly clean dataset can still be poisoned, because a skilled poisoner submits tidy, well-formed records.

What an unverified training set does to a model:

• Poisoning and backdoors. Planted examples teach a hidden rule, such as "any sample containing this marker is benign", that is invisible in aggregate accuracy. Lesson 18 covers the evidence it leaves.
• Embedded instructions. Text scraped from the web or from support tickets can contain instructions aimed at a model, and a model trained or grounded on it may learn to follow them.
• Leaked secrets and personal data. A code assistant fine-tuned on internal repositories will happily suggest the credentials somebody committed in a configuration file.
• Legal exposure. Data used without a licence or a lawful basis makes the model itself a liability.

Lineage, provenance and integrity: being able to say where every record came from

These three are tested as a set and are easy to blur, so pin each to one question.

Data provenance answers where did this come from? The original source, who created or collected it, when, and under what licence, consent or contract. Provenance is a property of the origin.

Data lineage answers what happened to it since? The path through every transformation: which jobs cleansed and labelled it, which version of the dataset resulted, which training run consumed that version, and which model version came out. Lineage is a property of the journey.

Data integrity answers has it changed without authorisation? It is protected by cryptographic hashes recorded at ingestion, signed manifests for each dataset version, write-once or versioned storage, and tight write access to the data lake and the labelling platform.

A worked example shows why all three are needed. A security vendor discovers that a public malware corpus it trains on was seeded with mislabelled samples. Provenance tells it which datasets drew on that corpus. Lineage tells it which dataset versions included the affected samples and which model versions were trained on them, which is the incident's scope. Integrity records prove that its own copies were not altered after ingestion, so the problem is upstream rather than an internal compromise. Without lineage the only safe response is to distrust every model, which nobody does, so in practice the affected models stay in service.

Practical artefacts carry this information: dataset versions with content hashes, documentation describing each dataset's source and intended use, model cards recording which dataset versions a model was trained on, and an AI bill of materials listing models, datasets and their origins. The same records serve deletion requests and licensing disputes.

Augmentation and balancing, and the bias they can remove or quietly introduce

Data augmentation creates additional training examples from existing ones. For images that means rotation, cropping or noise; for text, paraphrase or translation; and increasingly, wholly synthetic examples produced by another model, as with the GANs of lesson 1.

Data balancing corrects uneven representation. If one class or group is rare, a model can score well by ignoring it. Balancing techniques include oversampling the rare class, undersampling the common one, generating synthetic examples of the rare class and weighting errors on the rare class more heavily.

Security data is unbalanced by nature. Attacks are rare and benign events are plentiful, so an unbalanced intrusion detector can learn that "benign" is almost always the right answer. Balancing is what makes it pay attention to the cases that matter. The same logic applies to fairness: a fraud model trained mostly on one region's customers may perform badly, and unfairly, on another's.

The risk is that both techniques can introduce problems as easily as they remove them:

• Amplified poison. Oversampling copies rare examples, so one planted example in a rare class may be duplicated many times, multiplying its effect.
• Inherited blind spots. Synthetic examples carry the generator's own biases and gaps. A model trained on them learns the generator's idea of an attack, not the attacker's.
• Memorisation. Repeated minority examples are more likely to be memorised, which matters if they are personal records.
• The wrong target. Balancing towards a distribution the real world does not have produces a model miscalibrated in production.

The controls are to record every augmentation and balancing step in the lineage, flag synthetic records as synthetic, and always evaluate on real, untouched test data that no augmentation has seen.

Structured, semi-structured and unstructured data, and why most AI data is the third kind

Structured data follows a fixed schema of rows and columns: a transactions table, an asset inventory, an HR database. Its security tooling is mature. Column-level permissions, row-level security and masking all work because you know exactly where the sensitive field is.

Semi-structured data carries its own labels but not a rigid schema: JSON, XML, YAML, email headers and most log formats. Keys tell you what each value is, but fields vary between records, and free-text fields such as a log message or an error string can hold anything, including tokens, passwords and personal data that nobody intended to log.

Unstructured data has no schema a machine can rely on: documents, PDFs, chat transcripts, emails, images, scanned forms, audio and video. Most of what language models train on and retrieve from is unstructured, and so are the corporate documents behind most internal assistants.

That has four security consequences:

• Classification is harder. Sensitive data can be anywhere in a document, including inside an image of a scanned form, so pattern-based data loss prevention misses much of it.
• Permissions get lost. A document in a file share carries an access list. Extracted into a training set or a retrieval index, the text usually does not, unless someone deliberately carries the permissions along. Lesson 5 returns to this.
• Hidden content travels. Comments, metadata, tracked changes, alt text and text formatted to be invisible all reach the model even though a human reviewer never sees them, which is how instructions get planted in documents.
• Conversion is a step. Turning a PDF or a scan into text is itself a processing stage, with its own errors, and it belongs in the lineage.

In a scenario, identify the data type first. The right control for a database column is rarely the right control for a folder of contracts.

Why data security for AI is a property of the whole pipeline, not of the storage

Encrypting the training bucket is necessary and nowhere near sufficient, because AI data does not stay in the bucket. Follow one HR document through a typical pipeline: collected from the file share, copied to staging, cleansed, sent to a labelling vendor, stored as a dataset version, used for fine-tuning, evaluated, split into chunks and embedded into a vector index, retrieved into prompts, written into prompt logs, and finally fed back as user feedback into the next round of training.

Every one of those is a copy, and every copy needs a classification, an owner, access control and a retention period. Three of them are routinely missed:

• The model weights. A model trained on confidential data is confidential, because it can disclose that data. Weights belong in the same classification as their most sensitive training data.
• The prompt and response logs. Users paste things into assistants. The log becomes a new sensitive dataset the moment the assistant goes live.
• The feedback loop. Ratings, corrections and conversation history that flow back into training are an unverified input channel unless they pass through the same cleansing and verification as everything else.

The security level of the pipeline is set by its weakest stage, which is usually the labelling vendor, the staging area or the logs rather than the encrypted store everyone designed. Lesson 6 walks the same journey as a life cycle, and lessons 13 and 14 cover the encryption and data-minimisation controls applied along it.

What the exam is really asking

Most questions here reduce to picking the right word for the right question. If the scenario asks where a dataset originated and whether it may be used, the answer is provenance. If it asks which models are affected by a bad dataset, it is lineage. If it asks how to show records were not tampered with, it is integrity, delivered by hashes and controlled write access. If a model ignores a rare class, think balancing; if a model is confidently wrong about cases its training set never showed it, think augmentation done badly or not at all. And if the scenario names a document library, a set of scanned forms or a chat archive, it is telling you the data is unstructured, which rules out the controls that only work on database columns. When two answers both sound plausible, prefer the one that acts before training, because after training the only full remedy is to train again.