Reproduce Simpson's paradox and explain it

applied · 40 min · Objective 3.3

Task

Build the two support teams' ticket counts from the lesson, show that one team wins in every category and loses overall, and write the explanation a manager would need.

Steps

  1. Write lab/simpson/tickets.csv with header team,difficulty,tickets,resolved and the four rows: A easy 200 180, A hard 50 20, B easy 50 48, B hard 200 90.
  2. Calculate each team's resolution rate per difficulty and overall, and save them to lab/simpson/rates.csv with header team,group,rate, where group is easy, hard or all, rates as percentages to one decimal place.
  3. Write in lab/simpson/explain.txt why the overall comparison reverses, naming the variable responsible.

Verify

These checks run in a POSIX shell with Python 3: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. On Windows, python3 is often a Microsoft Store placeholder that prints “Python was not found”; run alias python3=py in Git Bash first.

awk -F, 'NR>1 {print $1" "$2": "$3"%"}' lab/simpson/rates.csv
grep -ciE 'difficult|mix|hard' lab/simpson/explain.txt

Team B resolves 96.0% of easy tickets against A's 90.0%, and 45.0% of hard against A's 40.0% -- better in both -- yet 55.2% overall against A's 80.0%. The explanation names ticket difficulty: B handles mostly hard tickets. Comparing overall rates compared the workloads, not the teams.

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.