Correlation, prioritisation and behaviour baselines

Why this matters

This lesson closes objective 4.1 with the part that turns collected, parsed, tuned data into decisions: joining events into findings, ranking those findings, and establishing what normal looks like so that abnormal can be recognised.

The reason it matters at the SecurityX level rather than the analyst level is that all three are design problems. Which correlations are possible is decided by what was ingested and how it was normalised. Which prioritisation is possible is decided by whether asset context is available to the detection platform. Whether baselining works is decided by whether the baseline period was representative and whether the population being baselined is coherent.

Get those design decisions wrong and no amount of analyst skill recovers it — which is why CAS-005 asks about them rather than about writing queries.

The lesson

Aggregate analysis: turning many events into one finding worth an analyst's time

The raw material is enormous and mostly uninteresting. Aggregation is what produces something worth a human's attention, and it works in three directions.

Grouping into cases. Fifty alerts about one host in one hour is one incident, not fifty. Grouping by entity — host, user, session — and by time window produces a case with context, and it changes the analyst's question from "is this alert real" to "what is happening on this host", which is the more productive question.

Chaining across stages. Individually unremarkable events form a recognisable sequence: an authentication from an unusual location, followed by enumeration of directory objects, followed by access to a system that account has never touched. No single step warrants an alert; the sequence does. This is where ATT&CK's tactic ordering earns its place — a chain spanning several tactics is far higher confidence than any of its links.

Thresholding and rate analysis. One failed authentication is nothing; several hundred across many accounts from one source is password spraying. The detection is in the aggregate rather than in any event, which means the rule has to be written over a window and the window length is a real design parameter.

Two design points that decide whether this works. The entity must be resolvable — grouping by username fails when the same person appears as three different strings, which is the normalisation work from lesson forty-one paying off or not. And the window must fit the behaviour: a five-minute window misses a patient attacker, and a twenty-four-hour window produces cases too large to reason about. Several windows for different behaviours is the answer, not one compromise.

Correlation rules across sources, and the join keys that make them reliable

Correlation joins events from different sources, and its power is that it is hard for an attacker to look normal in every source at once.

The joins that carry most value:

• Identity across sources. Authentication to a process on a host to an API call in cloud — following one account through the estate.
• Host across sources. Endpoint telemetry to network flow to vulnerability state to asset classification.
• Address and domain across sources, joining network activity to threat intelligence and to DNS.
• Session, where an identifier persists, which is the strongest join when it exists.

The join keys are where reliability is decided, and four problems recur:

• Inconsistent representation. DOMAIN\user, user@domain, and user are the same person and three different keys. Normalising to a canonical form at ingest is the fix and it must be uniform.
• Addresses are not identities. Dynamic addressing, network address translation and shared egress mean an address maps to different hosts at different times, so any address-based join needs a time-bounded lookup against lease or flow data rather than a static mapping.
• Hostnames are not stable. Rebuilt, renamed and reused, particularly in ephemeral infrastructure. A durable identifier from the platform is far better, which is lesson twenty-two's point about keying the record on something the platform can assert.
• Service and shared accounts collapse many actors into one key, which makes behavioural reasoning about them almost useless and is an argument for eliminating shared accounts rather than for better correlation.

The design conclusion: correlation capability is determined at ingest, by what is collected and how it is normalised. A programme that discovers it cannot join authentication to process execution is discovering a decision made when the sources were onboarded, and fixing it is an ingestion project rather than a rule change.

Trend analysis and what a change in a rate means that a threshold cannot say

Thresholds answer "is this value above a line". Trends answer "is this different from how it has been", and a great deal of malicious activity is visible only in the second form.

Where trend beats threshold:

• A gradual increase that never crosses a static line — data egress rising steadily over weeks, which any single day's threshold passes.
• A change in composition at constant volume: the same number of authentications, now including accounts that never authenticated to that system before.
• A decrease, which thresholds almost never watch. Log volume from a host dropping to a fraction is a strong signal — of a failed agent, or of an attacker disabling one. Lesson forty-one made this point about pipeline health; it is equally a detection.
• A change in periodicity. Beaconing to a command-and-control server is regular in a way human-driven traffic is not, and the regularity is the signal rather than the volume.

The design requirements are the same each time: a baseline computed over a representative period, seasonality accounted for — daily, weekly, and the month-end and quarter-end peaks that otherwise generate an alert every cycle — and a defined magnitude of deviation that is worth raising. Statistical deviation alone produces constant noise in a real estate.

The judgement worth stating: trend detections are powerful and expensive in analyst time, because a deviation is rarely self-explanatory. They repay themselves best on a small number of high-value measures — egress volume per host, privileged authentication count, new-service-account creation rate — rather than applied broadly.

Baselining network, system and user behaviour, and how long a baseline needs

A baseline is a model of normal, and its quality is decided by two design choices: the period and the population.

The period must include everything normal. A baseline over two weeks will not contain month-end processing, quarterly reporting, the annual audit, or seasonal patterns. Anything absent from the baseline is an anomaly forever, and the team learns to dismiss the resulting alerts — at which point the detection is worse than nothing, because it has trained its audience to ignore it. Thirty days is a practical minimum for most behaviours; a full cycle is better where the cycle is long.

The population must be coherent. Baselining each entity individually gives precision and needs enough history per entity — which a new user or an ephemeral workload does not have. Baselining a group gives immediate coverage and requires the group to actually be homogeneous. Peer grouping is the usual compromise, and its success depends entirely on how the peer groups are formed: comparing a user against people in the same role is informative, and comparing them against "all employees" is not.

What baselining detects well, by category:

• Users: access to systems never used before, volume anomalies, unusual hours, impossible travel, a sudden change in the breadth of resources touched.
• Systems: new processes, new outbound destinations, new listening ports, changed parent-child relationships, volume changes.
• Network: new pairs communicating, changed protocols, volume and periodicity — which is where the default-deny segmentation of lesson twenty earns a second dividend, because in an enumerated estate a new pair is nearly always meaningful.

Three cautions. A baseline built during a compromise bakes the compromise in, so the attacker's activity becomes normal — an argument for re-baselining after any confirmed incident. Baselines drift as the business changes, so they need continuous update, which creates the opposite risk: a slowly escalating attacker can walk the baseline with them. The usual mitigation is bounding how fast a baseline may move. And anomalous is not malicious — most deviations are a new project, a new tool or a changed role, which is why these detections belong in a hunt queue or a risk score rather than in a queue that must be worked.

Prioritising by asset value and exposure, using the inventory rather than severity alone

The closing point of the objective, and it is the same structural argument lesson thirty-two made about vulnerabilities, applied to alerts.

A detection platform's severity is a property of the rule. It is the same number whether the rule fired on a sandbox host or on the domain controller. Ranking a queue by it means analysts work in an order unrelated to consequence.

What should modify it:

• Asset classification and business criticality, from lesson five's inventory.
• The identity's privilege. The same behaviour on a privileged account is a different finding.
• Exposure. Internet-facing, or reachable from the user estate, or isolated — the segmentation from lesson twenty consumed as triage context.
• Corroboration. Two independent detections on one entity is worth far more than twice one, because the probability of two independent false positives on the same entity is low.
• Known vulnerability state, where the alert relates to something the host is actually exposed to.
• Recent change. An entity that was newly provisioned, newly privileged, or recently involved in another case.

The architectural requirement behind all of that: the detection platform must have asset and identity context available at alert time. That is an integration — the one lesson twenty-one listed as inventory-to-platform — and without it, prioritisation by consequence is impossible no matter how good the analysts are.

The measure that tells you whether it is working: the proportion of true positives found in the top decile of the queue. If serious incidents are routinely found low in the queue, or found by chance, the ranking is not reflecting consequence — and that is a design finding about context availability, not a performance finding about the team.