Distributions and the shape of data

Why this matters

A distribution describes how often each value occurs: where values cluster, how widely they spread, whether they trail off to one side. Two data sets with the same mean and standard deviation can have completely different shapes, and the shape decides which summaries are honest, which statistical tests are valid, and which values count as unusual.

The exam expects you to recognise common distributions, describe shape, and know why shape must be checked before a method is chosen. This lesson covers the normal distribution, skew and other shape features, frequency tables and histograms, a few other named distributions in plain terms, and checking shape before testing.

The lesson

The normal distribution and the empirical rule

The normal distribution is the symmetric, bell-shaped curve. Its mean, median and mode are equal and sit at the centre, and values become steadily rarer the further they are from it. It is described completely by two numbers: the mean, which sets its centre, and the standard deviation, which sets its width.

Many measurements are approximately normal: heights, measurement errors, the weights of manufactured items. More importantly, averages of many independent values tend towards a normal distribution even when the values themselves are not normal, which is the central limit theorem, and it is why so many statistical methods assume normality for means.

The empirical rule, or 68-95-99.7 rule, gives the shape in numbers. In a normal distribution:

• about 68% of values lie within one standard deviation of the mean;
• about 95% lie within two standard deviations;
• about 99.7% lie within three.

So if delivery times are roughly normal with a mean of 3 days and a standard deviation of 0.5, about 95% of deliveries take between 2 and 4 days, and a delivery of 5 days, four standard deviations out, is extremely unusual. The z-score rule for outliers, which flags values beyond three standard deviations, comes from this.

Skewness, kurtosis and multimodal data

Real data often departs from the normal shape in recognisable ways.

Skewness measures asymmetry. A right (positive) skew has a long tail of high values and a mean above the median; a left (negative) skew is the reverse. The course's order values are strongly right-skewed: a median of £99.21 and a mean of £177.58, pulled up by a few very large orders.

Kurtosis describes the tails: how prone a distribution is to producing extreme values compared with a normal one. Heavy-tailed data produces extremes more often than a normal distribution predicts, which matters in finance and risk, where the rare extreme event is the costly one. Using normal assumptions on heavy-tailed data underestimates how often extremes occur.

A multimodal distribution has more than one peak, which usually means the data mixes different groups. A histogram of customer spend with one peak near £20 and another near £500 probably combines retail and wholesale customers. Averaging across both produces a figure that describes neither, and the right response is to separate the groups and describe each.

Frequency tables and histograms

The two basic tools for seeing a distribution are the frequency table and the histogram.

A frequency table lists each value, or each range of values, with how many records fall in it. It may add the relative frequency, the percentage of the total, and the cumulative frequency, the running total. For the practice survey's satisfaction scores, the table shows 21 responses of 1, 24 of 2, 43 of 3, 43 of 4 and 69 of 5, out of 200: a picture of a left skew that the mean of 3.575 alone does not give.

A histogram draws the frequency table for a continuous variable: the range is divided into bins, and a bar's height shows how many values fall in each. The bin width changes what the histogram shows. Bins too wide hide features such as a second peak; bins too narrow produce a jagged picture dominated by noise. Trying two or three widths is good practice.

A histogram differs from a bar chart: its bars represent ranges of a continuous value and touch each other, while a bar chart's bars represent separate categories.

Uniform, binomial and Poisson in plain terms

A few other distributions are worth recognising by the situations they describe.

A uniform distribution gives every value in a range the same likelihood: a fair die, or a random number between 0 and 1. Its histogram is flat.

A binomial distribution describes the number of successes in a fixed number of independent yes-or-no trials with the same probability. How many of 100 visitors will buy, if each buys with probability 0.03? The answer is a binomial count, centred on 3.

A Poisson distribution describes the number of events in a fixed interval when events happen independently at a steady average rate. How many support calls arrive in an hour, if they average 12 an hour? It is used for counts of arrivals, errors and incidents, and it is right-skewed when the average is small.

The value of knowing these is recognition: a count of events per hour is not normally distributed, and treating it as normal can produce impossible predictions such as a negative number of calls.

Checking a distribution before choosing a test

Many statistical methods assume something about the distribution, most often that data, or the means being compared, are approximately normal. Using a method whose assumptions are badly violated gives results that look precise and are not.

Checking takes a few minutes:

• Plot it: a histogram or box plot shows skew, outliers and multiple peaks at a glance.
• Compare mean and median: a large gap indicates skew.
• Look for outliers and separate groups, and decide whether to treat them separately.
• For a more formal check, a Q-Q plot compares the data with a normal distribution, and a straight line means a good match; formal normality tests exist too.

When data is clearly not normal, the options are to transform it (a logarithm often makes right-skewed data such as incomes or order values roughly normal), to use a non-parametric method that makes no normality assumption, or to rely on large samples, since for means the central limit theorem makes the normality assumption more reasonable as sample size grows.