Validating findings and killing false positives

Objective 3.2 in this course covers result analysis — CompTIA describes it as validating findings, troubleshooting configurations and identifying false positives. This is the applied lab for the objective.

Why this matters

This lesson is the one that separates a penetration test from a scan export, and it is where a tester's professional reputation is actually made or lost.

Every lesson in this domain so far has ended by saying a scanner result is a lead. This is what you do with a lead.

The lesson

Reproducing before reporting, every time

The rule is absolute and it is worth stating without qualification: nothing goes in the report that you have not seen yourself.

Reproducing means you performed the check, observed the result, and captured evidence — not that you read the scanner's description and found it plausible.

What that looks like in practice:

• A different method from the one that found it. If a scanner inferred it from a version, confirm it by behaviour. Agreement between two independent methods is much stronger than either alone.
• Captured as you go. The request and response, the command and its output, with the timestamp. Reconstructing evidence later is how details drift.
• Repeated. A result that happens once may be a coincidence, a cached response, or a transient state.

The cost of skipping it is asymmetric. One finding demonstrated to be false in front of the client does not just remove that finding; it puts every other finding in the report in question, and you will not get to argue each one.

Version banners that lie, and backported patches

This deserves its own treatment because it is the single most common false positive in the trade, and lessons 10, 15 and 16 have each pointed at it.

A scanner sees version 1.2.3, looks up its known issues, reports them. But most distributions backport security fixes: they apply the patch and keep the version number, because changing it would break dependency resolution. So the banner says vulnerable and the software is not.

How to resolve it:

• Authenticated check. Read the actual package version and its distribution revision, which is where the backport shows. Lesson 16's whole point.
• Behavioural check. Test the condition rather than the version, safely — many vulnerabilities can be confirmed by a harmless behaviour rather than by exploitation.
• Ask. The client's engineers know their patching process, and a finding they can immediately disprove should have been checked with them first.

The reverse error exists too and is more dangerous: a current version string does not prove patched, because the running process may have been started before the update landed. Confirming a patched version on disk and a process that has not restarted is a real finding, and one scanners routinely miss.

Proving exploitability without exploiting production

The tension in this job: a finding is much stronger with proof, and proof usually means doing the thing you are being careful about.

The ways through, roughly in order of preference:

1. Observe the condition rather than the consequence. A response header, an error message, a behavioural difference that only the vulnerable state produces.
2. Exploit in your own lab, having reproduced the target's version and configuration. You prove the exploit works against that build, and prove the client is running that build, without touching theirs.
3. A safe subset on the target — a read-only, non-destructive action that demonstrates access without changing anything. A time delay, a single benign record, a value only the vulnerable path could return.
4. A full exploit, with permission, in a window, against a non-production instance — named in the scope, agreed in advance.

Never: destructive proof on production because it was quicker. And remember lesson 4 — the moment you succeed you are holding client data, and the minimum that proves the finding is the right amount.

When a true positive is not a real risk

A finding can be entirely real and still not matter here, and saying so is part of the job rather than a failure of nerve.

Things that legitimately reduce severity:

• Unreachable. The service is not exposed to anyone who could exploit it.
• Compensating controls that genuinely break the attack path.
• The asset does not matter — an isolated system holding nothing, due for decommission next month.
• Exploitation requires access the attacker would not have without already having something worse.

Two disciplines make this honest rather than convenient. Write the reason down, because lesson 5 showed that a stated basis is what survives challenge — and it works in this direction too. And do not delete the finding: record it at its adjusted severity with the reasoning, because compensating controls get removed and the next tester needs to know it was considered.

The inverse also applies. A finding that scores low generically can be critical here, and lesson 40's chain is usually why.

The cost to your credibility of one bad finding

This is the lesson's real argument, and it is professional rather than technical.

A report is trusted as a whole. The client's engineers will check the findings that touch their systems, and they will find any that are wrong. When they do, the conversation stops being about their security and becomes about your competence — and every remaining finding is now something they are motivated to dispute rather than fix.

Which means the incentives point the same way as the ethics: fewer, demonstrated findings beat more, inferred ones. A report of twelve findings you can each prove is worth far more than one of ninety copied from a scanner, and it takes less of everyone's time.

If you are unsure about a finding at the deadline, the options are to spend the time confirming it, or to report it as unconfirmed with the reason. What you do not do is state it as fact and hope.

What to take into the exam

• Nothing is reported that has not been reproduced, ideally by a second independent method, with evidence captured at the time.
• Backported patches make version banners unreliable in both directions — and a patched file with an unrestarted process is a real finding scanners miss.
• Prefer observing the condition, or exploiting a replica in your own lab, over exploiting production.
• A real finding can carry reduced severity for stated reasons; record the reasoning rather than deleting the finding.
• One disproved finding puts the whole report in question.