Profile the raw customer file before trusting it

short · 40 min · Objective 2.2

Task

Write a profiling script for the raw customer file and record what it finds: the shape, the key, the categories, the missing values and the formats. Each number becomes a test the cleaning labs must pass.

Steps

  1. Write lab/profile/profile.py, which reads data/customers_raw.csv and writes lab/profile/profile.txt.
  2. Record the lines rows: N, distinct ids: N, distinct regions: N, blank emails: N, slash dates: N (signup dates containing a /) and padded names: N (first names with leading or trailing spaces).
  3. Add shared emails: N: how many emails appear more than once when compared in lower case.
  4. Run it twice and confirm the profile does not change.

Verify

These checks run in a POSIX shell with Python 3: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. On Windows, python3 is often a Microsoft Store placeholder that prints “Python was not found”; run alias python3=py in Git Bash first.

cat lab/profile/profile.txt
grep -c ': ' lab/profile/profile.txt

314 rows and 314 distinct ids, yet 14 shared emails: the key is unique and the customers are not. 16 region values for 4 regions, 20 blank emails, 30 slash dates and 25 padded names. Seven lines, each one a number the cleaning must bring to its target.

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.