Write a detection, then try to slip past it

short · 50 min · Objective 1.3

Task

Write a search that catches something you did, then attack your own rule: find three variations of the same action that it misses. A rule's value is decided by what it misses, and the only way to learn that is to try.

Steps

  1. On the Windows VM, run powershell -EncodedCommand <base64 of Write-Output hi>. Confirm it reaches the collector.
  2. Write a search that matches it. Start with the obvious one: the string -EncodedCommand.
  3. Now defeat your own rule, three ways, each of which does the same thing:
  4. abbreviate the parameter (-enc, -e),
  5. change the case,
  6. insert the parameter after another argument.
  7. Re-run each variant and check whether your rule fired. Fix the rule after each miss, and record what the fix cost you in complexity.
  8. Finally, write the version that matches on something the attacker cannot change -- a base64-looking argument of a certain length passed to a shell -- and compare its false positive rate against the string match by running it over a day of your lab's normal logs.

Verify

grep -icE "encodedcommand" /var/log/collected/windows.log
grep -icE "(-e|-en|-enc|-encodedcommand)[[:space:]]+[A-Za-z0-9+/=]{20,}" /var/log/collected/windows.log
grep -icE "powershell" /var/log/collected/windows.log

The second count must be higher than the first: that is the measure of what your original rule missed. The third is the denominator -- the fraction of all PowerShell activity your final rule flags is its false positive burden, and a rule that flags most of it is not deployable however correct it is.

Notes

Keep both numbers. "This rule caught 4 of 4 variants and fires on 0.3% of PowerShell use" is the sentence that gets a detection deployed; "this rule detects encoded commands" is not.

The general lesson is the one this course keeps making: your rule looked correct after step 2, and three of its four failure modes were invisible until you deliberately tried to cause them.

This is an independent study companion for CompTIA CySA+ CS0-004 and is not produced by or endorsed by CompTIA.