Root cause analysis, threat hunting and digital forensics

Objective 4.8 in this course covers incident response. The previous lesson took the process and preparation; this one takes root cause analysis, threat hunting and digital forensics from CompTIA's scope note. It is the applied lab for 4.8.

Why this matters

These three are what happens around the edges of the incident lifecycle: root cause analysis at the end, threat hunting before any alert fires, and forensics throughout.

The forensics material is the most reliably examined. Order of volatility and chain of custody appear in some form on most attempts, and both are precise enough that the marks go to whoever learned the actual order and the actual requirements rather than the general idea.

The lesson

Root cause analysis that produces a control change rather than a name

Root cause analysis (RCA) asks why the incident was possible, not who did it. The output should be a change to a control, because that is the only thing that prevents a repeat.

The failure mode is stopping at the first plausible answer. "A user clicked a phishing link" is where most RCAs end, and it is not a root cause — it is the first event in a chain. Keep asking:

• Why did the email reach the user? The gateway did not flag it.
• Why did the link work? The domain was registered two days earlier and was not blocked.
• Why did the payload run? Macros were enabled by default and the user had local administrator rights.
• Why did it reach the file server? The workstation had an unrestricted path to it and the account had broad access.
• Why was it not detected for eleven days? Nobody had a rule for that behaviour, and endpoint telemetry was not being forwarded.

That produces five control changes — newly registered domain blocking, macro policy, removing local admin, segmentation, and a detection rule — where "the user clicked" produces only more training. Training has its place and it is the weakest of the six.

Two habits that make RCA honest:

• Ask why until the answer is a control you own. "Because the vendor shipped a vulnerability" is not actionable; "because we had no way to detect the exploitation and no compensating control" is.
• Separate cause from blame. An RCA that identifies a person produces defensiveness and hidden incidents. An RCA that identifies a missing control produces a fix. If a process depended on somebody never making a mistake, the process was the fault.

The output goes into the risk register from Domain 5 and into the change process from Domain 1, and somebody owns each item with a date. An RCA whose findings are not tracked is a document.

Threat hunting, and the hypothesis it starts from

Threat hunting is proactively searching for adversary activity that your detections have not alerted on. Its premise is that detection rules cover known patterns, and an attacker using novel or living-off-the-land techniques will not trip them.

It is distinguished from monitoring by who initiates. Monitoring waits for a rule to fire. Hunting starts with a human forming a hypothesis and going to look.

A hunt has a shape:

1. Hypothesis. "If an attacker were using scheduled tasks for persistence, I would see tasks created outside change windows on servers." Hypotheses come from threat intelligence about what actors targeting your sector do, from frameworks such as MITRE ATT&CK, from your own past incidents, and from knowing where your visibility is weakest.
2. Data. Decide what would show it, and confirm you actually collect that — hunts frequently end at this step, which is itself a valuable finding, because "we cannot answer this question" is a monitoring gap you can fix.
3. Search. Query the data, and expect to be wrong most of the time.
4. Outcome. Either you find something, which becomes an incident, or you do not — in which case the hunt should still produce a new detection rule, so the question never has to be asked manually again, or a documented visibility gap.

That last point is what makes hunting cost-effective. A hunt that finds nothing is not wasted if it leaves behind a rule.

Hunting is also where threat intelligence becomes operational: indicators and, more usefully, TTPs — tactics, techniques and procedures. Indicators (an address, a hash) are cheap for an attacker to change; techniques are not, which is why hunting for a technique outlasts hunting for an indicator.

Acquisition, order of volatility, and evidence that evaporates

Evidence disappears at very different rates, and the order of volatility says to collect the most fragile first. From most to least volatile:

1. CPU registers and cache — effectively gone immediately.
2. RAM — running processes, network connections, decryption keys, injected code, anything fileless. Lost at power-off.
3. Network state and running processes — active connections, routing and ARP tables, logged-on sessions.
4. Temporary files and swap/page file — may be cleared on reboot.
5. Disk — files, file system metadata, deleted-but-recoverable data.
6. Remote logging and monitoring data — already forwarded, comparatively safe.
7. Physical configuration and network topology — stable.
8. Archival media — backups and tapes, the most durable.

The single operational instruction that follows: do not power off a compromised machine before capturing memory. Fileless malware, injected code, attacker sessions and encryption keys live only there, and pulling the plug destroys all of it. Isolate it from the network instead — EDR network isolation from the 4.5 lesson exists precisely for this, keeping the machine live for capture while cutting the attacker's access.

Acquisition practices:

• Take a bit-for-bit image, not a file copy, so deleted data and slack space are preserved.
• Use a write blocker on physical media so the act of examining it does not alter it.
• Hash the image immediately and record the hash. Re-hashing later proves nothing has changed.
• Work on a copy, never the original.
• In cloud, acquisition means snapshotting volumes, exporting control-plane logs, and doing it before the resource is deleted — because terminating an instance destroys the evidence and the reflex during recovery is to terminate it.

E-discovery, reporting and preservation

E-discovery is the process of identifying, collecting and producing electronic information for legal proceedings. It is where the incident meets the law, and the security team's part is usually collection and preservation.

Preservation is the practical arm of the legal hold from the previous lesson: identify what is relevant, stop it being deleted, and be able to show that you did. Automatic deletion is the enemy — log retention periods, mailbox retention policies, backup rotation, and the ordinary decommissioning process from 4.2 will all quietly destroy relevant data unless the hold suspends them.

Reporting in a forensic context means writing what you did in a way that another examiner could follow and reproduce: what was collected, when, by whom, with what tool and version, what hash, what was found, and what conclusions follow — with the conclusions clearly separated from the observations. An investigation that cannot be reproduced is an opinion.

The related term interference covers anything that alters the evidence: booting the machine, running tools that write to disk, an antivirus scan quarantining the malware you were about to analyse, or a well-meaning administrator "tidying up".

Chain of custody in practice, and the gap that loses a case

Chain of custody is the documented, unbroken record of every person who has had the evidence, from collection to conclusion.

What each entry records: what the item is, with a unique identifier; who took it, from where, and exactly when; every transfer, with both parties signing; where it was stored between transfers and how that storage was secured; and what was done to it.

Why it matters: evidence with a gap in its custody record can be challenged on the basis that it might have been altered, and the challenge does not have to prove that it was. An unexplained four hours, a transfer with no receiving signature, or a period in an unlocked drawer is enough to undermine it.

The practical failures that produce the gap:

• the drive that sat on a desk overnight before anyone documented it;
• a transfer made verbally because it was urgent;
• an image copied to a shared network folder several people could reach;
• a hash taken at collection and never taken again, so there is nothing to compare;
• the machine that was "just checked" by an administrator before the investigator arrived, which is interference as well as a custody gap.

The habit that avoids all of them: start the record at the moment of collection, not when you decide it might matter legally. You rarely know at the start of an incident whether it will end in litigation, a regulatory process, an insurance claim or a disciplinary case — and by the time you know, the first hours are already unrecorded.

What to take into the exam

• Root cause analysis produces a changed control. "The user clicked" is the first event, not the root cause.
• Threat hunting starts from a hypothesis, not an alert, and a hunt that finds nothing should still leave a detection rule behind.
• Hunt for TTPs rather than indicators — techniques are expensive for an attacker to change.
• Order of volatility: registers, RAM, network state, temp/swap, disk, remote logs, physical, archival. Capture memory before power-off.
• Bit-for-bit image, write blocker, hash immediately, work on the copy.
• Chain of custody starts at collection; an unexplained gap is enough to undermine the evidence without anyone proving tampering.