Scanning web applications and APIs

Why this matters

Web applications are where most organisations' data actually lives, and a network vulnerability scanner barely touches them. It will tell you the web server version and the TLS configuration; it will not tell you that an authenticated user can read another customer's records by changing a number in a URL.

That gap matters for the exam and in practice, because the flaw classes that cause the largest breaches — broken authorisation, injection, exposed data — are application logic problems, not missing patches. Different tool, different method, different limits.

The lesson

Crawling, authentication and scan coverage

A web application scanner works in two phases: discover the application's surface, then test what it found. Both phases fail quietly, and the failure is always the same shape — the scanner tested thoroughly, but only the part it could reach.

Why crawling misses things:

• Authentication. Most of an application is behind a login. An unauthenticated scan tests the marketing pages and reports them clean.
• Multi-step workflows. Pages reachable only after completing a form, adding an item to a basket, or progressing through a wizard.
• Client-side rendering. Single-page applications build their interface in JavaScript; a crawler that only parses returned HTML sees almost nothing.
• Unlinked endpoints. Admin panels, debug routes and API paths with no link pointing at them.
• State destruction. The crawler clicks "log out" early and spends the rest of the scan unauthenticated — a classic, and one reason logout links are usually excluded explicitly.

What good practice looks like:

• Configure authentication properly, with a dedicated test account, and verify the scanner stayed logged in rather than assuming it.
• Exclude destructive actions: logout, delete, submit-payment, send-email.
• Seed the scanner with a list of known endpoints from documentation, a proxy recording of a manual walkthrough, or a sitemap.
• Check coverage afterwards. The count of URLs tested against the count you expected is the single most important number in the report, and it is almost never read.

A scan that covered 12 of 300 endpoints and found nothing is not a clean result. It is an unmeasured application — the third outcome from lesson 14 in another setting.

Where automated scanning stops being useful

Automated tools are reliable for flaws with a mechanical signature and poor at anything requiring understanding of what the application is for.

Found reliably: reflected cross-site scripting, obvious SQL injection, missing security headers, outdated components with known CVEs, directory listing, weak TLS, verbose error messages, and common misconfiguration.

Found unreliably or not at all:

• Broken object-level authorisation. The scanner has one account and no concept of "another user's data", so it cannot tell that object 1002 should be invisible to the account viewing it. This is consistently among the most damaging real-world flaws and is essentially invisible to automation.
• Business logic flaws. Applying a discount twice, negative quantities, skipping a payment step. The scanner does not know what the application is supposed to do.
• Privilege escalation between roles, for the same reason — it requires two accounts and a model of what each should see.
• Chained flaws, where three low-severity issues combine into a serious one.
• Race conditions.

The exam-relevant conclusion: automated scanning is necessary and not sufficient. Manual testing or a penetration test covers the authorisation and logic classes, and a programme that reports "no findings" from automation alone is reporting on a narrow slice.

A cheap partial answer worth knowing: run the scanner twice with two accounts at different privilege levels and compare what each could reach. That finds a meaningful share of authorisation problems without a full manual test.

API scanning and the schema problem

APIs now carry more traffic than browsers, and they break the discovery phase entirely: there is nothing to crawl. No links, no forms, no HTML.

So API testing depends on a specification — OpenAPI, Swagger, a Postman collection, a GraphQL schema. Point the scanner at the spec and it knows the endpoints, methods, parameters and types.

Which creates the characteristic failure: you test what the spec describes, and the spec is usually incomplete. Undocumented endpoints are common — older versions still routed, internal endpoints never meant to be public, debug routes. Those are exactly the ones most likely to be vulnerable, because they were never reviewed.

Ways to close the gap:

• Generate the spec from code rather than maintaining it by hand, so it cannot drift.
• Compare the spec against observed traffic from a gateway or proxy log. Endpoints receiving traffic that appear in no spec are the finding.
• Test old API versions explicitly. /v1/ is frequently still live and unmaintained after everyone moved to /v2/.
• Check the gateway inventory against the spec inventory.

GraphQL deserves a specific mention: a single endpoint, with introspection often enabled, which hands an attacker the entire schema. Introspection should be disabled in production, and query depth and complexity limited, or a single nested query can exhaust the server.

Safe versus intrusive checks on live systems

Web scanning interacts with an application that is doing real work for real people, and some checks change state.

Genuinely risky behaviours:

• Form submission at scale, creating thousands of records, orders, tickets or accounts.
• Email and SMS triggers, potentially sending thousands of messages to real addresses.
• Payment flows, where test transactions may become real ones.
• Injection payloads that modify data rather than just reading it.
• Denial of service, through resource-exhaustion checks or simply through request volume.
• Account lockout, where credential checks lock out real users.

Sensible handling:

• Test in a non-production environment wherever one exists that genuinely mirrors production. The caveat matters: a staging environment with different configuration tests different software.
• Use safe-check modes and exclude destructive test categories on production.
• Use dedicated test accounts and test payment instruments.
• Throttle, and scan outside peak hours.
• Tell the application owner and the SOC, for the same reason as lesson 17 — a scan looks exactly like an attack, and it will be investigated as one.
• Watch the application while scanning and stop if it degrades.

Interpreting results without a developer present

Application scanner output needs more interpretation than network output, and the analyst frequently has to make the first pass alone.

Working method:

1. Confirm it is real. Reproduce the request manually and look at the actual response. Web scanners have a high false-positive rate, particularly on injection detected by error text and on "vulnerable version" findings where the version string is wrong or the fix was backported.
2. Establish whether it is reachable and exploitable in the deployed configuration — is the endpoint authenticated, is it internet-facing, is there a WAF in front.
3. Judge the impact concretely. Reflected XSS on an unauthenticated marketing page is not the same as stored XSS in an admin console, even at the same nominal severity.
4. Group by root cause. Forty findings across one framework version is one upgrade, not forty tickets. Grouping is what makes a report actionable and is covered again in the reporting lesson.
5. Write it for the person who must fix it: the request that triggers it, the response that proves it, the specific line or component, and the remediation in that framework's terms.

The habit worth carrying from this whole domain: the scanner produces evidence, and you produce the finding. A report that forwards raw scanner output is passing your job to a developer who has less context than you do — and it is the fastest way to lose the credibility that makes the next report get acted on.

Topics this lesson owns

• [x] Crawling, authentication and scan coverage
• [x] Where automated scanning stops being useful
• [x] API scanning and the schema problem
• [x] Safe versus intrusive checks on live systems
• [x] Interpreting results without a developer present