Summarise a skewed column, and choose which summary to report

short · 35 min · Objective 3.2

Task

Calculate the centre and spread of the value of every valid order, and its five-number summary, then decide which figures honestly describe a typical order.

Steps

  1. For each order in data/orders.csv with a positive quantity, calculate net value as quantity times unit price (from data/products.csv) times (1 minus discount_pct / 100).
  2. With Python's statistics module, record in lab/describe/summary.txt the lines n: N, mean: N, median: N and stdev: N, each value to two decimal places.
  3. Record the five-number summary as five: min, q1, median, q3, max, using statistics.quantiles(values, n=4, method='inclusive') for the quartiles, each to two decimal places.
  4. Write in lab/describe/choice.txt which two figures you would report as typical and spread, and why.

Verify

These checks run in a POSIX shell with Python 3: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. On Windows, python3 is often a Microsoft Store placeholder that prints “Python was not found”; run alias python3=py in Git Bash first.

grep -E '^(n|mean|median|stdev|five): ' lab/describe/summary.txt
awk -F': ' '/^mean/ {m=$2} /^median/ {d=$2} END {print (m>d) ? "mean above median: right-skewed" : "mean not above median"}' lab/describe/summary.txt
grep -ciE 'median|quartile|IQR' lab/describe/choice.txt

1997 orders with a mean of 177.58, a median of 99.21 and a standard deviation of 735.56 -- larger than the mean itself, a sign of extreme values. The five numbers are 5.51, 53.80, 99.21, 202.26 and 22704.00. The mean sits far above the median, so the data is right-skewed, and the honest summary is the median with the quartiles.

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.