Classify every column before you share a file

short · 30 min · Objective 5.2

Task

Classify each column of the raw customer file by how directly it identifies a person, then produce the one version of the file that could be shared with a partner without personal data in it.

Steps

  1. Save lab/classify/columns.csv with header column,class, one row per column of data/customers_raw.csv.
  2. Use direct for a column that identifies a person on its own, indirect for one that identifies a person through a key or combined with other columns, and none for one that describes a large group.
  3. Write lab/classify/share.py to write lab/classify/share.csv, keeping only the columns classed none, for every row.

Verify

These checks run in a POSIX shell with Python 3: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. On Windows, python3 is often a Microsoft Store placeholder that prints “Python was not found”; run alias python3=py in Git Bash first.

awk -F, 'NR>1 {c[$2]++} END {for (k in c) print k": "c[k]}' lab/classify/columns.csv | sort
head -1 lab/classify/share.csv
echo "rows: $(($(wc -l < lab/classify/share.csv) - 1))"

Three direct (first_name, last_name, email), three indirect (customer_id, city and signup_date) and two none (region, segment), so the shared file has the header region,segment and 314 rows. customer_id is the one most often misclassed: it names nobody, but anyone holding the customer table can turn it back into a person, and GDPR treats data like that as personal.

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.