Metadata analysis, data recovery and root cause

Why this matters

This is the last lesson of the course, and it closes the loop the first lesson opened. Domain 1 built the documents, the register and the frameworks. Domain 2 designed the boundaries. Domain 3 engineered the controls. Domain 4 detected and analysed. This lesson is where an incident becomes a change to all four — or does not, which is the failure most organisations actually have.

Three subjects. Metadata is the evidence that establishes what happened and when, and it is fragile in specific, knowable ways. Recovery is getting data and service back, and the examinable part is knowing when to stop trying. Root cause is the discipline that determines whether the same incident happens again — and it is the one most often performed as a formality.

The connecting idea, and it is the one to carry out of the whole course: an incident that does not change the risk register, a control or a design has been survived rather than learned from.

The lesson

Metadata as evidence: filesystem times, document properties, headers and logs

Metadata is data about data, and in an investigation it is frequently more informative than content.

Filesystem timestamps. Modification, access, creation and metadata-change times, whose exact meanings differ by filesystem. They establish sequence — which file appeared before which — and they are trivially forgeable by an attacker with sufficient privilege, a technique common enough to have its own name. Corroborating filesystem times against a source the attacker did not control — journal entries, log records, a backup's catalogue — is what makes a timeline defensible.

Document properties: author, organisation, creation and revision times, editing duration, software version, and in many formats a revision history and template path. Useful for establishing origin, and a recurring source of accidental disclosure in documents an organisation publishes.

Email headers, which record the path a message took, the servers that handled it, the authentication results from lesson thirty-seven, and timing. The examinable caution is that headers are added by each hop, so those added by systems you control are trustworthy and those added earlier are assertions by parties you do not.

Image and media metadata, which can include capture device, settings and location, and is a disclosure risk in published material.

Log records, which are metadata about activity and are the most reliable category when the logs went somewhere the compromised system's administrator could not alter — which is lesson twenty-three's requirement and lesson forty-one's, arriving where they pay off.

The property that matters across all of them: metadata under the attacker's control is an assertion; metadata outside it is evidence. Building a timeline means preferring the second and treating the first as corroboration.

Order of volatility, and collecting evidence in a sequence that survives scrutiny

Evidence disappears at different rates, and collection order matters because collecting one thing can destroy another.

Order of volatility, most volatile first:

1. CPU registers and cache.
2. Memory, which holds running processes, network connections, decrypted content, injected code and keys — and is lost on power-off.
3. Network state: connections, routing and ARP tables.
4. Running processes and open files.
5. Temporary filesystem contents.
6. Disk.
7. Remote logs and monitoring data, which are comparatively durable.
8. Physical configuration and archived media.

The practical consequences, and each is a decision a scenario may describe:

• Memory first, if memory is needed. Shutting down destroys it, and for many modern intrusions — in-memory payloads, stolen credentials, decrypted data — memory is the most valuable evidence available.
• Isolate rather than power off, where containment is required and evidence matters. Network isolation stops the damage and preserves volatile state; pulling the power is the one irreversible choice available and it should be deliberate.
• Collect before remediating. Reimaging solves the problem and eliminates the investigation — the trade lesson twenty-nine flagged about automated response, arriving here as a human decision.

Chain of custody is what makes collected evidence usable if the matter becomes legal or disciplinary: what was collected, when, by whom, with what tool, its integrity verified by hash at collection, and every transfer of possession recorded. Also: work from copies, verify the copy against the original's hash, and use write-blocking when imaging media.

The judgement to state plainly, because scenarios test it: containment and evidence preservation can conflict, and the resolution depends on what the incident is. Active data exfiltration is contained immediately and evidence comes second. A long-dwell intrusion discovered quietly is frequently better served by careful collection before an attacker learns they are detected. Either way it is a decision with a named owner — which is the RACI from lesson three, and the reason that lesson insisted the matrix covers decisions and not only tasks.

Data recovery from damaged, deleted or encrypted sources, and when to stop

Recovery has two purposes that are often confused: restoring service and recovering evidence. They can conflict, since restoring from backup over a compromised system destroys the evidence on it.

The sources, in order of preference:

• Backups, which is why lesson eight's restore testing matters and why the backup segment in lesson twenty exists — an attacker who reached the backups has removed this option deliberately.
• Filesystem recovery of deleted files, which works while the underlying blocks are unallocated and degrades as the system continues running. Another argument for imaging early.
• Snapshots and replicas, where they were not also affected.
• Reconstruction from other systems — a downstream copy, a report, a partner's records.
• Specialist recovery for physically damaged media, which is expensive and occasionally the only route.

Encrypted data is the case that needs a clear-eyed answer, because the pressure in an incident is toward optimism. Where ransomware has encrypted data and the implementation is sound, recovery without the key is not achievable. What is worth checking: whether a flaw in the specific family has yielded a public decryptor, whether keys remain in memory on a system not yet rebooted — which is another argument for collecting memory before acting — and whether a copy exists somewhere untouched. Beyond that, the honest position is that recovery is from backup or not at all, and payment is a business and legal decision with its own sanctions and regulatory dimensions, not a technical one.

When to stop is a real decision and belongs to the business. The inputs are the recovery point objective from lesson eight, the cost and time of further attempts against the value of what is missing, and whether the data can be reconstructed another way. A recovery effort that continues indefinitely because nobody owns the decision to stop is its own failure, and it delays the restoration of service that people are waiting for.

Root cause analysis that reaches a decision or a design, not a person

The discipline that determines whether the incident recurs, and it is frequently performed as a ceremony.

Root cause is not the immediate technical trigger. "An unpatched server was exploited" is a symptom. The useful question is why it was unpatched, and the answers are usually structural: it was not in the inventory, so it was not in the scan scope; its owner had left; patching it required an outage nobody would authorise; the team had no capacity.

Two properties separate real analysis from a formality:

It reaches something that can be changed. A root cause that is a person's mistake is almost never the root cause — it is the last link in a chain that allowed a mistake to have that consequence. Why was the action possible? Why was it not detected? Why was the blast radius that large? Those reach controls and designs, which can be changed; blame reaches a person, who cannot be redesigned and who will be replaced by someone equally capable of the same mistake.

It is blameless in practice, not only in name. People who expect consequences do not report early, do not volunteer what they did, and do not describe the workaround they had been using for months. A blameless process is not kindness; it is the only way to obtain accurate information about what actually happens, as opposed to what the process says happens.

The technique matters less than the discipline — repeated "why" questions, or a contributing-factor analysis across people, process, technology and environment — provided the output is specific, owned and dated actions, and provided those actions are tracked to completion with the same rigour as any other work. A retrospective producing recommendations nobody owns is where most of this effort is lost.

Two further outputs worth insisting on, because they are the ones that change the organisation rather than the system:

• What detected it, and what should have? If discovery was by a third party or by chance, that is a detection gap and it belongs in the coverage map from lesson forty-two.
• What did our exclusions and suppressions cover? The review lesson forty-two asked for, performed here, on this incident's attack path.

Lessons learned feeding the risk register and the control set from domain 1

The closing section of the lesson, the domain and the course.

An incident is the most reliable information an organisation will ever get about its own security, and it is expensive. Extracting value from it means feeding what was learned back into the artefacts that govern what happens next — which are the ones domain 1 built.

Concretely, an incident should produce updates to:

• The risk register, from lesson eight. A risk that materialised was either absent, scored too low, or treated with a control that did not work. All three are register updates, and a risk that materialised while recorded as "low likelihood" is a calibration finding worth more than the entry itself.
• The control set and the standards, from lesson one. A control that failed needs changing, and a control that worked deserves recording as evidence for its cost.
• The threat model, from lesson nine. A real attack path is the best possible input to a model, and it should be compared against what the model predicted.
• Detection coverage, from lesson forty-two, with new detections for every behaviour observed.
• The asset inventory, from lesson five, if unknown assets were involved — which they frequently are.
• The response plan and the RACI, from lesson three, wherever a decision was slow because nobody was unambiguously empowered to make it.
• Training and communication, if a person's reasonable action had an unreasonable consequence.
• The board report, from lesson three, with what changed and what it cost.

And the measure of whether a programme learns, which is a small number and fits in that report: the proportion of post-incident actions completed by their due date. An organisation with detailed retrospectives and a backlog of incomplete actions is one that conducts reviews rather than one that improves, and the distinction is visible in that single figure.

That is the end of the course. The through-line, stated once: every control in these four domains is a claim about what is true, and a claim nobody has demonstrated — a perimeter never tested by attempting the denies, a detection never seen to fire, a backup never restored, a coverage figure with no denominator, a policy nobody has read since it was approved — is not a control. It is a belief with a document attached. The work of this level is knowing the difference, and insisting on it.