Tabulate a distribution and test the empirical rule on real readings

short · 35 min · Objective 3.2

Task

Build a frequency table of the survey's satisfaction scores, then check how closely the sensor's temperature readings follow the 68-95 rule once the spikes are set aside.

Steps

  1. Count satisfaction scores 1 to 5 in data/survey.json and write lab/shape/frequency.csv with header score,count,percent, percent to one decimal place.
  2. Read temp_c from data/sensor.csv, sort the readings, and set aside the four highest (the spikes).
  3. For the rest, calculate the mean and standard deviation, and the percentage of readings within one and within two standard deviations of the mean. Record them in lab/shape/empirical.txt as readings: N, within 1 sd: N and within 2 sd: N, percentages to one decimal place.

Verify

These checks run in a POSIX shell with Python 3: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. On Windows, python3 is often a Microsoft Store placeholder that prints “Python was not found”; run alias python3=py in Git Bash first.

awk -F, 'NR>1 {print $1": "$2" ("$3"%)"}' lab/shape/frequency.csv
grep -E '^(readings|within 1 sd|within 2 sd): ' lab/shape/empirical.txt

Scores of 1 to 5 were given 21, 24, 43, 43 and 69 times: the commonest answer is the top score, and the shape is skewed left. Of 1424 temperature readings, 64.3% lie within one standard deviation and 97.7% within two -- close to the normal distribution's 68% and 95% but not the same, because these readings follow a daily cycle rather than a bell curve. The rule describes normal data; it is worth checking before relying on it.

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.