Choosing the method for the question and the audience

Why this matters

Before any data is touched, an analysis succeeds or fails on whether it is answering the right question in a way its audience can use. A sophisticated model that answers a slightly different question from the one asked is wasted work, and a correct result presented through a method the audience cannot follow is rarely acted on.

The exam frames communication as selecting methods for different audiences. This lesson covers turning a business question into an analytical one, defining key performance indicators, the four common kinds of analytical question, balancing simplicity against sophistication, and agreeing the question before starting.

The lesson

Turning a business question into an analytical one

Business questions usually arrive vague: "How are we doing with customers?", "Is the new website working?", "Why are sales down?". An analytical question is specific enough to be answered with data.

The translation adds four things:

• A measure: what exactly will be counted or calculated. "Doing with customers" might mean retention rate, satisfaction score or revenue per customer.
• A population: whose data. All customers, or active customers, or customers who joined this year?
• A period and a comparison: over what time, compared with what. This month against last month, or against the same month last year, or against target.
• A decision: what will be done differently depending on the answer.

"Is the new website working?" becomes "Has the checkout conversion rate for visitors on the new website, in its first four weeks, differed from the rate for the old website over the same four weeks last year, and is the difference large enough to justify keeping the new design?" That version can be answered, and everyone can see in advance what the answer will look like.

Key performance indicators and how they are defined

A key performance indicator (KPI) is a measure an organisation has chosen to track progress towards a goal: monthly revenue, customer churn rate, average resolution time, on-time delivery rate. KPIs make performance comparable over time and across teams, and they concentrate attention on what matters.

A KPI is only as useful as its definition, and definitions hide choices. "Churn rate" needs answers to: which customers count as active, how long without an order counts as lost, whether the rate is monthly or annual, and whether the denominator is customers at the start of the period or on average. Two departments calculating churn differently will report two rates and argue about which is right.

A well-defined KPI states its formula, its data source, its population, its period and its owner, and is calculated the same way everywhere, ideally in one governed place. Good KPIs are also actionable: someone can influence them, and a change in them means something. A KPI that nobody can affect, or that moves for reasons unrelated to performance, is a statistic, not an indicator.

Distinguish leading indicators, which change before an outcome, such as website visits before sales, from lagging indicators, which record an outcome after it happens, such as quarterly revenue. Both are useful, for different decisions.

Comparison, composition, distribution and relationship questions

Most analytical questions fall into a few families, and recognising the family points to the method and the chart.

• Comparison: how do values differ across categories or over time? Sales by region, this year against last. Methods include grouped summaries and period-over-period change; charts include bar and line charts.
• Composition: what makes up a whole? Revenue share by product category, the mix of new and returning customers. Methods are proportions and percentages; charts include stacked bars and, for a few parts, pie charts.
• Distribution: how are values spread? The range of order values, the shape of response times. Methods are descriptive statistics and percentiles; charts include histograms and box plots.
• Relationship: do two variables move together? Satisfaction and recommendation, price and demand. Methods include correlation and regression; the chart is the scatter plot.

Naming the family first avoids a common error: choosing a method or chart before deciding what question it is meant to answer.

Simple methods an audience can follow against complex ones it cannot

There is usually more than one valid method for a question, and the most sophisticated is not always the best.

A simple method, such as a comparison of averages, a percentage change or a well-chosen chart, can be understood, checked and trusted by a non-technical audience. A complex method, such as a regression with many variables or a machine learning model, may capture more, and it asks the audience to trust what it cannot inspect.

The choice depends on the stakes and the audience. For an operational decision by managers, a clear comparison that is 90% as informative often beats a model they cannot explain to their own teams. For a high-stakes decision, or where simple methods would be misleading, such as when confounding variables need controlling for, the complex method is justified, and the analyst's job is to explain its result in simple terms and state why it was needed.

A practical rule is to start simple: if a straightforward comparison answers the question convincingly, the complex method adds cost without value. If it does not, the simple result is still a useful check on the complex one.

Agreeing the question before starting the analysis

The most effective way to avoid answering the wrong question is to agree the question in writing before starting. A short statement covering the question, the measure and its definition, the population and period, the comparison, the intended audience and the decision it supports takes minutes to write and can save days of rework.

Agreeing the question also sets expectations. It fixes the definition of success before results are known, which protects against moving the goalposts, and it surfaces disagreements early: two stakeholders who want different answers to what they thought was the same question discover it before the analysis rather than at the presentation.

When the question changes during the work, as it often legitimately does once early results are seen, the change should be acknowledged and agreed again, rather than drifting unnoticed.