Analyse output until you can defend every number

applied · 90 min · Objective 2.2

Task

Take a full set of scan results and produce a defensible analysis: duplicates resolved, false positives proven, false negatives acknowledged, and every remaining number traceable to evidence. Objective 2.2 is analysing output, and the test is whether the numbers survive someone asking where they came from.

Steps

  1. Merge every scan you have -- host, web, container, external -- into one dataset with a source column. Count the raw findings.
  2. Deduplicate. The same issue appears from several tools and several scans. Decide the identity key (asset plus vulnerability, not plus plugin) and collapse. Record how many findings the raw count was overstating.
  3. Resolve asset identity. Find the hosts that appear under more than one name or address -- a renamed VM, a changed DHCP lease -- and merge them. Record how many assets your raw data thought you had.
  4. Prove three false positives with host evidence, as in the short lab.
  5. Record two false negatives you know about: the hand-compiled binary and the vendored library from earlier labs.
  6. Compare over time. Put two scans of the same host side by side and classify every difference as fixed, new, or an artefact of the scan itself.
  7. Write the analysis summary: the raw count, the deduplicated count, the verified count, and the caveats -- each with the method that produced it.

Verify

cat /tmp/scan-*.csv | grep -c "^"
awk -F, 'NR>1{print $1"|"$3}' /tmp/scan-*.csv | sort -u | wc -l
awk -F, 'NR>1{print $1}' /tmp/scan-*.csv | sort -u | wc -l
comm -13 <(awk -F, 'NR>1{print $1"|"$3}' /tmp/scan-run1.csv | sort -u) <(awk -F, 'NR>1{print $1"|"$3}' /tmp/scan-run2.csv | sort -u) | wc -l

The first two are the overstatement: raw rows against distinct asset-plus-issue pairs. The third is your asset count as the data believes it. The fourth is findings present in run two and absent from run one -- and every one of them is either genuinely new or an artefact, which is a judgement no tool makes for you.

Notes

The deduplication ratio is worth keeping. Reports built on raw rows inflate by a factor that varies with how many tools you run, which makes cross-period comparison meaningless unless the key is fixed.

Step 6's classification is the habit. A finding that "disappeared" is the most dangerous row in a comparison, because absence is produced by remediation and by failure identically.

This is an independent study companion for CompTIA CySA+ CS0-004 and is not produced by or endorsed by CompTIA.