Watch a scanner miss most of the application

short · 60 min · Objective 2.1

Task

Scan a deliberately vulnerable training application twice -- unauthenticated and authenticated -- and measure the coverage difference. Then find something by hand that the scanner could not reach at all.

Steps

  1. Deploy a well-known deliberately vulnerable web application on the Linux VM and confirm it answers on http://10.10.10.10/.
  2. Unauthenticated scan. Run a web scanner against it with no credentials. Record the finding count and, separately, the count of distinct URLs the scanner reports having visited.
  3. Log in manually and note how much of the application only exists behind the login.
  4. Authenticated scan. Configure the scanner with a session and repeat. Record both counts again.
  5. Compare the URL counts before the finding counts. Coverage explains most of the difference in findings, and reporting findings without coverage is the error this lab exists to prevent.
  6. Now find one issue by hand that neither scan found -- a logic flaw, an authorisation gap between two accounts, a workflow step that can be skipped. Automated scanning cannot reason about intent, and this is where that stops being an abstraction.

Verify

grep -c "^" /tmp/zap-unauth.csv
grep -c "^" /tmp/zap-auth.csv
awk -F, '{print $2}' /tmp/zap-unauth.csv | sort -u | wc -l
awk -F, '{print $2}' /tmp/zap-auth.csv | sort -u | wc -l

The last two are the point: distinct URLs reached. If the authenticated scan visited several times as many URLs, then the unauthenticated scan's clean areas were never examined rather than examined and found clean -- and nothing in its report says so.

Notes

Record the pair "findings / URLs visited" for both scans. That pair is the honest way to report a web scan, and a finding count without it is a number whose denominator is unknown.

The hand-found issue is the deliverable. Write it up with the reason a scanner could not find it, because that sentence is what justifies the budget for manual testing.

This is an independent study companion for CompTIA CySA+ CS0-004 and is not produced by or endorsed by CompTIA.