AI for code quality, vulnerability analysis, automated testing, incidents and threat models

Why this matters

Most of a security team's time goes on reading: code, scanner output, advisories, logs, tickets, reports in languages nobody on the team speaks. Language models are good at reading and drafting, so these are the tasks they speed up most visibly. A code assistant explains a finding and proposes a fix in seconds. An incident assistant turns ten thousand log lines into a draft timeline. A model summarises a forty-page threat report before the morning meeting.

The risk is the mirror of the benefit. Every one of these outputs is fluent, confident and plausible, and some of it is wrong. A suggested fix that introduces a new flaw, a timeline entry that never happened, a summary that drops the one caveat that mattered, a scan that strays outside the agreed scope: each is worse than no output, because it carries authority it has not earned.

So the lesson has one theme across five tasks. AI does the first draft; a person, a test or the source itself verifies it before it changes anything. Scenario questions usually describe an AI-assisted task that went wrong and ask which verification step was missing.

The lesson

Code quality and linting with AI assistance, and reviewing what it suggests

Traditional linters and static analysis tools apply fixed rules: unused variables, unsafe functions, injection-prone string building, missing input validation. They are deterministic and repeatable, and they are noisy. AI assistance helps in three ways: it explains a finding in plain language and in the context of the actual code; it proposes a fix; and it triages large result sets, separating likely real issues from false positives by reading the surrounding logic.

What it suggests must be reviewed for the failure modes specific to generated code:

• Insecure patterns reproduced from training data. A model has seen a great deal of code, including insecure code, and can produce string-built SQL queries, weak cryptography or disabled certificate checks as readily as correct ones.
• Plausible but non-existent dependencies. A model may suggest importing a package that does not exist. If an attacker has registered that name, the suggestion installs their code. Check every new dependency against the registry and your approved list (lesson 18 covers the supply chain).
• Fixes that silence rather than repair. A "fix" that suppresses the linter warning, catches and ignores the exception or deletes the failing test makes the finding disappear without removing the flaw.
• Outdated or deprecated APIs, and code whose licence terms are unclear.

The controls are ordinary software engineering applied without exception: generated code goes through the same review, the same static analysis, the same tests and the same approval as human code. The deterministic linter remains the gate; the assistant is a helper that makes the gate cheaper to pass. Lesson 27 places this inside the CI/CD pipeline.

Vulnerability analysis and automated penetration testing, within authorised scope

Vulnerability analysis turns scanner output into decisions. AI helps by deduplicating findings across tools, explaining a CVE in terms of your software, checking whether the vulnerable function is actually reachable in your code, and prioritising using exploitability context such as whether a vulnerability appears in CISA's Known Exploited Vulnerabilities catalogue or carries a high exploit prediction score. The result is a shorter, better ordered remediation queue. The verification step is that a person confirms the asset, the version and the exposure before a finding is closed as not applicable, because a model asserting "not reachable" is a hypothesis, not a test.

Automated penetration testing uses AI to plan and sequence testing steps, chain findings, and adapt to what earlier steps reveal. It is legitimate only as authorised, in-scope work, and the controls that keep it there are the same ones that govern a human tester, made enforceable in the tooling:

• Written authorisation from someone with authority over the target, and rules of engagement covering timing, techniques permitted and contacts.
• A technical scope allowlist. The tool may only reach the agreed hosts, ranges and applications. A model that "discovers" a related domain or a third-party service must not follow it, so scope is enforced by the platform rather than left to the model's judgement.
• No destructive or high-impact actions without a human approving them, plus rate limits so testing does not become a denial of service.
• A kill switch and full logging of every action, so the test can be stopped and reconstructed afterwards.
• Human validation of findings before they are reported, since automated tools generate false positives and occasionally misread success.

Automation changes the speed of a test, not who is accountable for it. The engagement lead still owns every action the tool took, which is why the log matters as much as the findings. It also changes the reporting burden: an AI tester can produce a large volume of findings quickly, and each one still needs reproduction steps, evidence and a severity a person has checked before it reaches the system owner. A report padded with unverified findings wastes the remediation effort it was meant to direct.

An exam scenario that describes an automated tester probing a system not named in the engagement is describing a scope failure, and the answer is a technical boundary and a halt, not a better model.

Incident management with AI: triage, timelines and next steps

During an incident AI assists across the response life cycle:

• Triage. Grouping related alerts into one case, enriching them with asset and identity context, and suggesting severity.
• Timelines. Drafting a chronological account from logs, endpoint telemetry and ticket notes, which is slow and error-prone by hand.
• Next steps. Suggesting investigation queries and containment actions drawn from the organisation's own playbooks.
• Communication. Drafting status updates for technical and executive audiences, and handover notes between shifts.

Three cautions are specific to incidents:

• The draft timeline is not evidence. Every entry must link to the underlying log record, and originals are preserved unaltered under the usual chain-of-custody rules. A model can merge two events, invent a plausible intermediate step or get a timezone wrong.
• Attacker-controlled content is in the data. Log lines, email bodies and file names were partly written by the intruder. An assistant reading them is exposed to indirect prompt injection (lesson 17), so it should not have authority to close cases or take containment actions on its own.
• Incident data is sensitive. Personal data, credentials and details of the compromise should only go to an approved service, and redacted where possible.

Containment actions suggested by AI follow the approval design in lesson 26: a person confirms anything that disrupts production.

Threat modelling with AI as a second pair of eyes, not the owner

Threat modelling asks what can go wrong with a system and what to do about it. Given an architecture description or data flow diagram, a model can walk each element and boundary through a structure such as STRIDE, list candidate threats, suggest mitigations and point out ones the team missed. For systems that include a model, it can map the design against MITRE ATLAS techniques (lesson 8).

That is useful as a second pair of eyes, and weak as the owner, because:

• It knows only what it is told. Business context, trust assumptions, the regulator's view and the history of past incidents are rarely in the prompt.
• Its output tends towards the generic: a long list where every common threat appears and none is weighted by likelihood or impact in this organisation.
• It cannot accept risk. Deciding which threats to mitigate, transfer or accept is an accountable decision that needs a named human owner.

The productive pattern is for the team to model first, then ask the AI what they missed, and record which of its suggestions were accepted and why. Giving it better inputs improves the result: the data flow diagram, the trust boundaries, the data classifications and the controls already in place. A model asked to threat-model "our customer portal" with no detail returns a textbook; one given the diagram can question a specific boundary. Remember that the architecture itself is sensitive, so the same rules on approved services apply here as to incident data.

Translation and summarisation, and checking a summary against its source

Translation lets an analyst read foreign-language threat intelligence, criminal forum posts, ransom notes and phishing lures quickly. Its weak points are slang, deliberately obscured wording and technical terms that a general model renders literally. Anything that drives a decision, such as an attribution claim or a threat to a named executive, should be confirmed by a fluent speaker.

Summarisation condenses long reports, advisories, log extracts and incident tickets. The two failure modes are omission, where the caveat, the exception or the affected version range disappears, and fabrication, where the summary states something the source does not.

Checking a summary against its source is a routine, not an act of distrust:

• Ask for the summary to cite where each claim comes from, then spot-check those locations.
• Never take indicators from a summary. IP addresses, hashes, domains and CVE identifiers are copied from the original, because a single wrong character produces a block on the wrong thing or a missed match.
• Check numbers, dates and version ranges against the source every time.
• For a decision that rests on the summary, read the relevant section of the original yourself.

A short scenario

A security operations centre adopts an assistant that summarises overnight alerts and drafts a timeline for any open incident. One morning the summary reports a contained phishing incident; the underlying logs show a second host contacting the same command-and-control domain, which the summary omitted. The missing control is not a better model. It is the rule that the timeline links each claim to its log record and that an analyst checks the source before the incident is closed.

What the exam is really asking

For every task in this lesson the question is the same: what verifies the AI's output before it changes code, scope, evidence or a decision? The correct answer names a test, a boundary, a source check or a human approver.