Threat hunting, digital forensics, root cause and the post-incident report

Objective 4.7 covers incident response. The previous lesson took the process, the preparation, negotiation and notification. This one takes the activities around the edges: hunting for what no alert caught, investigating in a way that would survive scrutiny, and the post-incident work that makes sure the same thing does not happen twice. It is the applied lab for 4.7.

Why this matters

These activities sit at three different points in the incident. Threat hunting belongs to identification and often starts before any alert fires. Forensics, e-discovery, preservation and chain of custody belong to investigation. Root cause analysis, lessons learned and the post-incident report close the incident.

Chain of custody is the most reliably examined item here, and it is precise enough that the marks go to whoever learned the actual requirements rather than the general idea. The post-incident material is examined differently: the question usually describes an organisation that keeps suffering the same incident, and asks what it failed to do.

The lesson

Threat hunting, and the hypothesis it starts from

Threat hunting is proactively searching for adversary activity your detections have not alerted on. Its premise is that detection rules cover known patterns, and an attacker using novel or living-off-the-land techniques will not trip them.

It differs from monitoring by who initiates. Monitoring waits for a rule to fire. Hunting starts with a person forming a hypothesis and going to look.

A hunt has a shape:

1. Hypothesis. "If an attacker were using scheduled tasks for persistence, I would see tasks created outside change windows on servers." Hypotheses come from threat intelligence about what actors targeting your sector do, from frameworks such as MITRE ATT&CK, from your own past incidents, and from knowing where your visibility is weakest.
2. Data. Decide what would show it, and confirm you actually collect that. Hunts often stop here, and that is itself a finding: "we cannot answer this question" is a monitoring gap you can fix.
3. Search. Query the data, and expect to be wrong most of the time.
4. Outcome. Either you find something, which becomes an incident, or you do not, in which case the hunt should still leave behind a new detection rule or a documented visibility gap.

A hunt that finds nothing is not wasted if it leaves a rule behind. Hunt for TTPs -- tactics, techniques and procedures -- rather than single indicators. An address or a file hash is cheap for an attacker to change; a technique is not, so hunting for the technique outlasts hunting for the indicator.

Digital forensics, e-discovery and preservation

Digital forensics is the disciplined collection and examination of digital evidence so that conclusions can be trusted, checked and, if necessary, defended in a disciplinary hearing, an insurance claim or a court. Its rules all serve one purpose: proving the evidence examined is the evidence collected.

The core practices:

• Take a bit-level image, not a file copy, so deleted data and slack space are preserved.
• Use a write blocker on physical media so examining it does not change it.
• Hash the image at collection and record the value; re-hashing later shows nothing has changed.
• Work on a copy, never the original.
• Capture memory before powering off, because running processes, network connections, injected code and encryption keys live only there. Isolate the host from the network instead of pulling the plug; EDR isolation (lesson 32) exists for this.

The order in which evidence is collected, most fragile first, and the three kinds of system image -- memory dump, bit-level copy, snapshot -- are the subject of lesson 41, which treats them as investigation data sources.

Preservation means stopping relevant evidence from being changed or destroyed, and being able to show you did. Automatic deletion is the enemy: log retention periods, mailbox policies, backup rotation, and the ordinary decommissioning process in lesson 33 will quietly destroy relevant data unless someone suspends them. So will interference -- an antivirus scan quarantining the sample you needed, an administrator "tidying up", or a reflexive rebuild. Image before you rebuild. In cloud, snapshot volumes and export control-plane logs before the instance is terminated, because recovery's reflex is to terminate it.

E-discovery is identifying, collecting and producing electronic information for legal proceedings. The security team's part is usually finding where the relevant data lives and collecting it in a defensible way. The legal duty to preserve once proceedings are anticipated, the legal hold, sits with compliance in lesson 45.

Chain of custody in practice, and the gap that loses a case

Chain of custody is the documented, unbroken record of every person who has held the evidence, from collection to conclusion.

Each entry records what the item is, with a unique identifier; who took it, from where, and exactly when; every transfer, with both parties signing; where it was stored between transfers and how that storage was secured; and what was done to it.

Why it matters: evidence with a gap in its custody record can be challenged on the basis that it might have been altered, and the challenge does not have to prove it was. An unexplained four hours, a transfer with no receiving signature, or a night in an unlocked drawer is enough.

The practical failures that produce the gap:

• the drive that sat on a desk overnight before anyone documented it;
• a transfer made verbally because it was urgent;
• an image copied to a shared folder several people could reach;
• a hash taken at collection and never taken again, so there is nothing to compare;
• the machine an administrator "just checked" before the investigator arrived, which is interference as well as a custody gap.

The habit that avoids all of them: start the record at the moment of collection, not when you decide it might matter legally. At the start of an incident you rarely know whether it will end in litigation, a regulator's inquiry, an insurance claim or a disciplinary case, and by the time you know, the first hours are already unrecorded.

Root cause analysis that produces a control change rather than a name

Root cause analysis (RCA) asks why the incident was possible, not who did it. Its output should be a change to a control, because that is the only thing that prevents a repeat.

The failure mode is stopping at the first plausible answer. "A user clicked a phishing link" is where many RCAs end, and it is not a root cause; it is the first event in a chain. Keep asking why:

• Why did the email reach the user? The gateway did not flag it.
• Why did the link work? The domain was two days old and nothing blocked newly registered domains.
• Why did the payload run? Macros were allowed and the user had local administrator rights.
• Why did it reach the file server? The workstation had an unrestricted path to it and the account had broad access.
• Why was it not detected for eleven days? No rule covered that behaviour, and endpoint telemetry was not being forwarded.

That produces five control changes -- domain blocking, macro policy, removing local admin, segmentation and a detection rule -- where "the user clicked" produces only more training.

Two habits keep RCA honest. Ask why until the answer is a control you own: "the vendor shipped a flaw" is not actionable, "we had no compensating control and no way to detect exploitation" is. And separate cause from blame: an RCA that names a person produces defensiveness and hidden incidents, while one that names a missing control produces a fix. If a process depended on nobody ever making a mistake, the process was the fault.

Lessons learned and the post-incident report, written so somebody acts on it

Lessons learned is the review meeting held soon after the incident, while memory is fresh, with everyone who took part. It asks what happened, what went well, what went badly, what was missing, and what should change in the plan, the playbooks, the tooling and the training. It is blameless for the same reason RCA is.

The post-incident report (PIR) is the written record that comes out of it. A usable one contains:

• a one-paragraph summary at the top, because many readers stop there;
• the timeline, with timestamps in a stated timezone and the source of each entry;
• scope and impact: what was affected, what was established not to be affected and how, and the business consequences;
• the root cause and contributing factors from the RCA;
• what went well and what did not in the response itself;
• actions, each with an owner and a due date.

The test of a PIR is whether anything changes. Its actions go into the risk register (lesson 43) and through change management (lesson 5), and someone checks that they closed. A report whose findings are not tracked is a document, and the organisation that keeps suffering the same incident is usually one that held the meeting, wrote the report, and assigned nothing.

What to take into the exam

• Threat hunting starts from a hypothesis, not an alert, and a hunt that finds nothing should still leave a detection rule behind. Hunt for TTPs, not indicators.
• Forensics: bit-level image, write blocker, hash at collection, work on the copy, capture memory before power-off.
• Preservation fights automatic deletion and interference; image before you rebuild.
• Chain of custody starts at collection, and an unexplained gap undermines the evidence without anyone proving tampering.
• Root cause analysis produces a changed control. "The user clicked" is the first event, not the cause.
• Lessons learned is the meeting; the post-incident report is the record; both fail if their actions have no owner and date.