Allocation, oversubscription and the noisy neighbour

Why this matters

This is where virtualization stops being a diagram and starts explaining things you will actually be asked to diagnose. A workload is slow. Its CPU graph looks fine. Its memory graph looks fine. Nothing in the guest explains it.

The answer is usually that the guest's view is not the whole story: the hypervisor is arbitrating between this guest and others, and the evidence lives outside the guest. Knowing which metric to ask for — and what it means — is the difference between "the cloud is slow today" and a diagnosis.

It also underpins two commercial decisions the exam cares about: why capacity is cheap, and what you are paying for when you buy reserved, dedicated or spot capacity.

The lesson

How providers oversubscribe CPU and memory, and why that is normal rather than cheating

Oversubscription (or overcommit) means allocating more virtual resource than physically exists — more vCPUs than cores, sometimes more memory than installed. It works because workloads are bursty and uncorrelated: most guests are idle most of the time, so the physical resource is shared without anyone noticing.

This is the economic engine of the whole industry. Without it, a provider would need one core per vCPU sold, and the price would look like owning hardware. It is normal, it is expected, and the provider's job is to keep the ratio low enough that contention stays rare.

Two important asymmetries:

• CPU oversubscription is routine and usually harmless. A vCPU that is idle costs nothing to schedule.
• Memory oversubscription is far more dangerous. Memory is not bursty in the same way: a guest that has been given a page tends to keep using it. When a host really runs out of memory the remedies are all bad, which is why providers are conservative here and why memory-optimised instances cost what they do.

What you buy when you pay more is, largely, a guarantee about this ratio.

CPU ready time and steal time: the metrics that reveal contention you do not own

These two metrics are the answer to "the guest looks fine but is slow", and knowing them distinguishes a candidate who has operated a platform from one who has read about one.

• CPU ready time (the hypervisor's term, visible to the platform administrator) is time a vCPU was runnable — it had work to do — but had to wait for a physical core to become free. Rising ready time means the host is oversubscribed relative to actual demand.
• CPU steal time (visible inside a Linux guest, typically as %st in top or vmstat) is the same phenomenon seen from the other side: time the guest wanted the CPU and the hypervisor gave it to someone else.

The diagnostic value is that steal time is visible from inside a guest you control, about a problem you do not control. High steal time with normal guest CPU utilisation is the classic noisy neighbour signature: another tenant on the same host is consuming capacity.

What to do about it, in the order a scenario usually wants:

1. Confirm it is steal and not guest CPU saturation.
2. Move the workload — stop and start the instance, which commonly relocates it to a different host.
3. If it recurs, buy isolation: a larger instance type, a dedicated host, or a family with stronger guarantees.

A counter-intuitive point the exam likes: adding vCPUs can make ready time worse. A guest with many vCPUs may need several physical cores free simultaneously to schedule, so on a busy host a smaller guest can run better than a larger one.

Memory ballooning, swapping and the performance cliff at the end of them

When a host is short of memory, hypervisors reclaim it in a sequence, and the sequence matters because each step is worse than the last.

1. Page sharing / deduplication. Identical pages across guests are stored once. Free, invisible, limited.
2. Ballooning. A driver inside the guest (part of the guest tools) is told to allocate memory, which pressures the guest's own memory manager into releasing its least valuable pages. This is the well-behaved mechanism: the guest decides what to give up, because it knows which pages matter.
3. Compression. Pages are compressed in memory rather than written out. Slower than RAM, much faster than disk.
4. Host swapping. The hypervisor writes guest pages to disk without the guest knowing. This is the cliff: the guest believes it is accessing memory at memory speed and is actually waiting on storage, so performance falls off sharply and the guest's own metrics do not explain why.

Two consequences to carry in. Ballooning is a warning sign, not a failure — it means the host is under pressure and is asking nicely. Host swapping is a failure and the remedy is to reduce memory overcommit, not to tune the guest. And if the guest tools are not installed, ballooning is unavailable and the platform goes straight to the harsher options.

Storage IOPS, throughput and burst credits, and the workload that exhausts them

Storage performance has two distinct dimensions, and conflating them causes misdiagnosis:

• IOPS — operations per second. Small, random reads and writes. Databases and transactional systems care about this.
• Throughput — bytes per second. Large sequential transfers. Backups, analytics and media care about this.

A volume can be starved of one while the other is fine, and the graphs look completely different. Ask which dimension is saturated before tuning anything.

Burst credits are the mechanism that catches people out. Many entry-level volume types deliver a modest baseline and accumulate credits while idle, which can be spent on short bursts of much higher performance. The result is a system that benchmarks beautifully, runs well for an hour, and then collapses to baseline when the credits run out — with no configuration change and no error.

The signature is a performance graph with a flat ceiling that appears part way through a sustained workload. The remedy is a volume type with a provisioned, non-bursting performance level. A scenario describing "fast at first, then consistently slow, same workload" is describing exhausted credits.

The same pattern exists for some small instance families' CPU, which is why the idea is worth generalising: baseline plus credits equals good benchmarks and bad sustained behaviour.

Reserved, dedicated and spot capacity as three answers to the same contention question

These purchasing models are usually taught as cost levers. They are also answers to "how much do I care about contention and availability", which is how the exam frames them.

• On-demand. Pay per unit time, no commitment. Maximum flexibility, highest unit price, and you share hosts with other tenants.
• Reserved / committed use. Commit to a quantity for one or three years for a substantial discount. This is a billing construct: you are buying a price, and in some forms a capacity reservation. The risk is the commitment itself — a three-year reservation for a workload that gets re-architected in twelve months is money spent.
• Dedicated host or dedicated instance. You pay for isolation: no other tenant on your hardware. The reasons are compliance, licensing that is bound to physical cores or sockets, and eliminating noisy neighbours. Expensive, and the right answer when a scenario says "must not share hardware".
• Spot / pre-emptible. Spare capacity at a steep discount, reclaimable by the provider at short notice. Suitable for fault-tolerant, interruptible, stateless work — batch processing, rendering, CI runners. Unsuitable for anything that cannot be killed mid-flight.

The mapping the exam wants:

What to take into the exam

• Oversubscription is normal. CPU overcommit is routine; memory overcommit is the dangerous one.
• Steal time inside a guest with normal guest CPU = noisy neighbour. First remedy is to relocate the instance; lasting remedy is isolation.
• More vCPUs can mean worse scheduling on a contended host.
• Reclaim order: share → balloon (warning) → compress → host swap (failure). Ballooning needs guest tools installed.
• IOPS and throughput are different limits. Identify which is saturated.
• Burst credits produce "fast then permanently slow"; fix with a provisioned performance volume.
• Dedicated = no shared hardware (compliance, licensing). Spot = interruptible only. Reserved = a price commitment and its risk.