Presenting findings with their limits

Why this matters

Every analysis rests on assumptions and has limits: data that covers only part of a population, a definition that could have been drawn differently, a model that fits the past and may not fit the future. Presenting results without them invites decisions the evidence cannot support. Presenting nothing but caveats buries the finding. The skill is to state the result clearly and its limits honestly, in proportion.

The exam's communication objective covers how results are delivered and what they may and may not claim. This lesson covers assumptions and limitations, uncertainty, the formats results are delivered in, recommendations, and the claims the data does not support.

The lesson

Stating assumptions and limitations plainly

An assumption is something taken as true for the analysis to work: that the sample represents the customer base, that last year's seasonal pattern will repeat, that returns are recorded in the month they occur. A limitation is a known weakness: missing data for one region, a survey with a low response rate, a short history.

Both should be stated, briefly and specifically, where the reader will see them. "Results may be subject to limitations" says nothing. "The survey covers customers who responded, about one in five of those asked; if dissatisfied customers were less likely to respond, satisfaction is overstated" tells the reader exactly what to watch for and in which direction the error would go.

The limitations that matter most are those that could change the conclusion. A missing week of data that changes a total by 0.1% needs a footnote at most. A missing region that accounts for a third of revenue needs to be in the headline.

Uncertainty: ranges rather than false precision

Most analytical results are estimates, and an estimate without an indication of its uncertainty invites more confidence than it deserves.

Uncertainty can be expressed as a range, such as "between 3.4 and 3.8" or "£1.1 million to £1.4 million"; as a confidence interval, "3.6, with a 95% interval of 3.4 to 3.8"; or in words matched to the evidence, "likely", "probably", "possibly". Forecasts in particular should be presented as ranges or scenarios, because a single number implies a certainty no forecast has.

False precision is the opposite fault: reporting 12.347% from a survey of 200 people, or a forecast to the nearest pound. The extra digits are noise, and they suggest the analysis is more exact than it is. Rounding to what the data can support, usually two or three significant figures, is both clearer and more honest.

A useful discipline is to ask of every number in a report: if the analysis were run again on slightly different data, how much would this change? The answer is the precision it deserves.

Written summaries, presentations and briefings

Results reach their audience in several forms, each with its own conventions.

A written summary or report is the most complete and lasting form. It should open with an executive summary that stands alone, follow with the key findings and supporting evidence, and move method, detail and caveats into later sections or an appendix. It is the record others will return to, so it must be accurate and self-contained.

A presentation supports a spoken explanation. Each slide should make one point, stated in its title as a conclusion ("North churn doubled in Q3") rather than a topic ("Churn by region"), with one clear chart or a few numbers as evidence. Dense tables and paragraphs belong in the handout, not on the screen.

A briefing or verbal update is the shortest form: the conclusion, the one or two numbers that support it, the main caveat and the ask, in a few minutes.

Dashboards and self-service reports deliver results for ongoing monitoring rather than a one-off finding, and are covered in domain 4. Whichever the format, the content should be consistent: the same numbers, the same definitions and the same caveats, so that a reader of the report and a listener to the briefing reach the same understanding.

Recommendations that follow from the evidence

When an analysis includes recommendations, they should follow visibly from the findings. A good recommendation states what should be done, why the evidence supports it, and what result is expected, and it is specific enough to act on: "Offer a retention discount to Business customers in the North whose order frequency has fallen by half" rather than "improve customer retention".

The analyst's role is to make the evidence clear and the options understood. The decision often rests on factors the data does not contain, such as cost, strategy, risk appetite or ethics, and the recommendation should acknowledge them. Where the evidence supports several reasonable options, presenting them with their trade-offs is more useful than presenting one as inevitable.

Recommendations should also say how their effect will be measured, which sets up the next analysis.

What not to claim from the data you have

Some claims are tempting and unsupported, and avoiding them is part of presenting honestly.

• Causation from correlation: "customers who use the app spend more" does not show that the app makes them spend more.
• Generalising beyond the sample: results from one region, one period or one type of customer do not automatically apply to others.
• Extrapolating beyond the data: a trend observed for six months does not predict the next five years.
• Precision the data lacks: small samples and subgroups support broad conclusions, not exact ones.
• Certainty from significance: a statistically significant result can still be small, chance, or unimportant.

When a finding is interesting but not established, it can still be reported, clearly labelled as an indication worth investigating rather than a conclusion. That label is the difference between an analysis that informs a decision and one that misleads it.