Review and quality control before publishing

Why this matters

Once a report is published, its numbers leave the analyst's control. They are quoted in meetings, copied into presentations and used in decisions, and a correction a week later reaches only some of the people who saw the original. The cheapest time to find an error is before release, and a short, consistent review process finds most of them.

The exam's objective includes applying review to solve reporting issues. This lesson covers a pre-publication checklist, peer review of logic, testing filters and edge cases, sign-off and ownership, and correcting a report after release.

The lesson

A pre-publication checklist

A checklist turns review from a vague "have a look" into specific checks that are done every time, regardless of deadline pressure. A useful checklist for a report covers four areas.

Data:

• Is the data complete for the period, and was it refreshed when expected?
• Do row counts and control totals reconcile to the source?
• Have known data quality issues been handled or disclosed?

Logic:

• Are measures calculated with the agreed definitions?
• Are filters, joins and time windows correct?
• Have the headline numbers been recomputed independently?

Presentation:

• Are titles, labels, units and legends present and correct?
• Do charts start their axes appropriately and use suitable types?
• Are numbers formatted consistently, with sensible precision?
• Is the report accessible, with readable text and colour-safe palettes?

Context:

• Are the data date, source, definitions and limitations stated?
• Would the intended audience understand it without explanation?

The checklist should be short enough to be used, and it should grow when an error gets through that a check would have caught.

Peer review of logic, not only appearance

Review often concentrates on what is easiest to see: spelling, colours, layout. Those matter, and they are rarely where the serious errors are. The errors that change decisions are in the logic: a join that duplicates rows, a filter that excludes a region, a measure that averages averages, a date window that is off by a day.

An effective peer reviewer therefore looks behind the report:

• reads the queries and calculations, not only the output;
• checks the definitions against the agreed ones;
• reproduces a headline figure independently;
• asks whether the conclusion follows from the evidence, and whether anything important has been left out.

Review works best when the reviewer is not the author, has enough time and context to do it properly, and is expected to challenge. A reviewer who only confirms is not reviewing.

Testing filters and edge cases

Interactive reports have many possible states, and errors often hide in the ones the author never tried. Testing should deliberately exercise them.

• Each filter value, especially the smallest categories, where empty results, division by zero or misleading percentages appear.
• Combinations of filters that produce no data, to confirm the report shows a clear "no data" state rather than a blank or an error.
• Boundary dates: the first and last day of a period, month ends, year ends, and periods that include a daylight saving change.
• Special values: zero, negative numbers, missing values, very large values.
• Drill paths at every level, to confirm measures stay meaningful.
• Different user roles, to confirm row-level security shows each role the right data and nothing more.

Totals should add up in every state: the sum of the regions equals the total, the filtered view matches a direct query for the same filter.

Sign-off, and who owns a published number

Every published report should have an owner: a named person or team responsible for its accuracy, its maintenance and answering questions about it. Reports without owners decay, because nobody notices when they break.

For important reports, sign-off is the formal confirmation, by someone with authority, that the report has been reviewed and is fit to release. Sign-off matters most for reports that inform significant decisions, go to senior leadership or the board, are shared outside the organisation, or feed regulatory submissions.

Sign-off is only meaningful if the person signing knows what was checked. That is why the validation record and the review checklist matter: they give the approver evidence rather than asking them to take the work on trust.

Correcting a report after release

Despite review, errors will occasionally reach published reports. How they are handled decides whether trust in the reporting survives.

The process follows the same steps as a stale-data incident:

1. Confirm and scope the error: which figures, which periods, which versions of the report.
2. Flag the report as under correction, so no one continues to use the wrong figures.
3. Correct the logic or data, and re-validate the corrected report.
4. Communicate: tell the people who received or used the report what was wrong, what the correct figures are, and whether any decision could be affected.
5. Record the correction in the report's change log, with the date and reason, and keep the previous version for reference.
6. Prevent recurrence by adding a check to the checklist or an automated test that would have caught it.

Correcting openly is uncomfortable, and it is the practice that makes a reporting function trustworthy over time.