Directing malware analysis and reverse engineering

Why this matters

The last objective of the exam is incident response, and this lesson covers the analysis half. The scope note belongs at the top rather than at the bottom, because it changes how to read everything below: CAS-005 examines commissioning and interpreting this work, not performing it.

That is a genuine distinction and not a hedge. Malware analysis and reverse engineering are specialist disciplines requiring months of dedicated practice, and CompTIA place them in one bullet of a four-bullet domain worth 22%. What an architect or senior practitioner is expected to do is decide whether analysis is warranted, specify what question it must answer, ensure it is done safely, and read the result well enough to act — including knowing which parts of it are reliable and which are inference.

The most valuable thing in this lesson is therefore the framing of the first question: what do you actually need to know, and what is the cheapest way to learn it? Analysis performed without a question produces a technical document and no decisions.

The lesson

Static and dynamic analysis, and what each answers about an unknown sample

Two complementary approaches, and knowing which answers which question is the examinable part.

Static analysis examines the sample without running it: file type and structure, embedded strings, imported functions, embedded resources, cryptographic constants, compilation artefacts, and — at the deepest level — disassembly of the code.

• Answers: what is this file, what capabilities does it appear to have, what does it reference, does it resemble a known family.
• Advantages: safe, because nothing executes; complete, because all code is present whether or not it would run; and fast at the surface level.
• Limits: packing and obfuscation defeat surface static analysis almost entirely. A packed sample's strings and imports are those of the unpacking stub, and reading meaning into them is a common error.

Dynamic analysis runs the sample in an instrumented environment and observes behaviour: files created, registry or configuration changes, processes spawned, network connections attempted, and memory contents once unpacked.

• Answers: what does it actually do, what does it contact, how does it persist, what would we see in our telemetry.
• Advantages: defeats packing, since the code must unpack itself to run, and produces indicators directly usable for detection and hunting.
• Limits: you observe one execution in one environment. Code paths not taken are unobserved, and the sample may deliberately behave differently — which is the next section.

The practical sequence, and the reason it is worth knowing: triage statically, then observe dynamically, then reverse only if a question remains. Full reverse engineering is expensive and is warranted when you need to understand a capability precisely — an encryption routine, a configuration format, a command protocol — and cannot obtain it otherwise. Most incidents do not need it, and commissioning it reflexively consumes the scarcest skill in the organisation on a question the first two stages already answered.

Sandboxing safely: isolation, evasion, and the sample that behaves when watched

Dynamic analysis requires an environment where execution is safe and observed, and getting that wrong has consequences beyond the analysis.

Isolation requirements, each of which has been the cause of a real incident:

• Network isolation, so the sample cannot reach production, cannot spread, and cannot exfiltrate anything real. Simulated network services are used so behaviour is observable without genuine connectivity.
• Controlled outbound access. Some samples need to contact infrastructure to reveal behaviour, and allowing that from an address attributable to your organisation tells the operator they are being analysed — and tells them who. Attribution-managed egress is the control, and its absence is a scenario.
• Strong host separation. Virtual machine escapes exist; a sandbox on hardware shared with anything sensitive is a risk.
• Discard after use. The environment is destroyed and rebuilt from a clean image for every sample.
• Legal and contractual clarity about handling samples, particularly where they contain data belonging to someone else.

Evasion is the reason a clean sandbox result must never be read as "benign":

• Environment checks — is this a virtual machine, are analysis tools present, is the disk implausibly small, is there a mouse moving, does the machine have a plausible history of use.
• Timing — sleeping past the sandbox's observation window, or requiring user interaction.
• Targeting — executing only on a particular domain, language, or geography, which is common in targeted intrusions and means the sample is genuinely inert in your sandbox.
• Staged payloads that fetch the real functionality from infrastructure that is already offline by the time you analyse.

So the conclusion to carry into an exam: a sandbox report showing no malicious behaviour is not evidence of safety. It is one observation, and the correct reading is that nothing was observed under those conditions — which is the same not-measured-is-not-a-pass rule this course has applied to checks, benchmarks and log sources, arriving in its last domain.

Extracting indicators of compromise that are worth distributing

The output that makes analysis useful to the rest of the response, and the quality bar matters.

Worth extracting and distributing, roughly in order of durability:

• Behavioural characteristics: the persistence mechanism, the parent-child process relationship, the sequence of actions, the technique. Longest shelf life, per lesson forty-seven, and the material for a Sigma rule.
• Structural file characteristics that identify the family rather than the sample — the material for a YARA rule.
• Infrastructure patterns: the shape of the domain names, the request format, the certificate characteristics, the beaconing interval. These survive infrastructure rotation in a way individual addresses do not.
• Specific artefacts: file hashes, file paths, mutex names, registry keys, domains and addresses. Precise, immediately actionable, short-lived.

Not worth distributing, and a report containing these indicates analysis that has not been triaged:

• Hashes of the packer or of common components, which match unrelated software.
• Legitimate infrastructure the sample contacted — a public platform, a content delivery network, an operating system update service — which, blocked, causes an outage.
• Generic strings and paths present in benign software.
• Anything matching an internal tool, which produces false positives forever.

Two practices that make the output usable. Every indicator carries context and an expiry: what it indicates, confidence, when it was observed, when it should be retired. And the indicators are immediately swept retrospectively across retained telemetry — which is the payoff for the retention decision in lesson forty-one, and is frequently how the true scope of an incident is established.

Code stylometry and attribution claims, and how much weight to place on them

Code stylometry analyses stylistic characteristics of code — structural habits, naming conventions, library preferences, compilation artefacts, language and locale settings, comments — to link samples to a common author or development environment.

Its legitimate value is clustering: establishing that two samples probably share an origin, which helps connect campaigns and apply what you learned from one to the other. Used that way, it is a useful analytical technique.

The caution, which is the examinable content, is about attribution — naming who is responsible:

• Stylistic markers are easily faked. An operator can adopt another group's conventions, plant misleading locale settings, embed foreign-language strings and reuse another group's tooling. This is done deliberately, and it is done specifically because analysts look at these signals.
• Tooling is shared, sold and stolen, so a tool associated with one group appears in another's operations.
• Infrastructure is rented, frequently from the same small set of providers, so overlap is weak evidence.
• Confirmation bias is strong in attribution work, because the analyst usually has a hypothesis before looking.

The position to hold, and it is the same one lesson nine took about actor characteristics: attribution rarely changes what you should do. The remediation, the containment, the detections and the architectural improvements are determined by the observed behaviour and capability, not by the name attached to it. Where attribution genuinely matters — legal action, insurance, government notification, sanctions compliance — it is a determination for specialists and authorities with sources beyond your telemetry, and the appropriate output from your analysis is observations with confidence levels, not a conclusion.

Two phrasing habits that keep a report honest: state what was observed separately from what is inferred, and attach a confidence level to every inference. A report that mixes them is the one that gets quoted in a board paper as fact.

Scope note: this course covers commissioning and interpreting this work, not performing it

The closing note, stated explicitly because this is the topic most likely to consume study time disproportionate to its exam weight.

What CAS-005 can reasonably ask:

• When analysis is warranted and what question it should answer.
• The difference between static and dynamic analysis and what each establishes.
• Sandbox isolation requirements and evasion, and why a clean result is not a clean bill of health.
• What makes an indicator worth distributing.
• How much weight attribution claims deserve.
• How analysis output feeds containment, detection and the incident record.

What it will not ask, and what returns nothing for the effort: disassembly, debugger use, unpacking technique, shellcode analysis, or exploit development. Those are the substance of a specialist discipline and a different certification.

The practical framing for the role the exam is written for: you are the person who decides whether to spend a specialist's week on a sample, who specifies the question, who ensures the environment is safe and non-attributable, and who reads the result critically — knowing that a clean sandbox proves nothing, that attribution is usually irrelevant to your decisions, and that the durable output is a behavioural detection rather than a hash. That skill set is what turns analysis from an interesting artefact into a change in what the organisation can see, which is where the final lesson picks up.