Turn free-text comments into word counts

short · 35 min · Objective 1.5

Task

Tokenise the survey's free-text comments, count the words that remain once stop words are removed, and count the comments that were left blank. It is the first step of every text analysis: turning prose into something that can be counted.

Steps

  1. Read the comment field of every response in data/survey.json.
  2. Count the blank comments and record them in lab/text/blank.txt as blank: N.
  3. Lower-case each comment, split it into words of letters only, and drop the stop words to, the, a, and, was, for, of and it.
  4. Save the ten commonest remaining words to lab/text/top-words.csv with header word,count, most frequent first.

Verify

These checks run in a POSIX shell with Python 3: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. On Windows, python3 is often a Microsoft Store placeholder that prints “Python was not found”; run alias python3=py in Git Bash first.

grep -E '^blank: ' lab/text/blank.txt
head -4 lab/text/top-words.csv
grep -c . lab/text/top-words.csv

31 of the 200 comments are blank. The commonest remaining word is "delivery", with 33 mentions, and the file has a header plus ten words. Counting says what people talk about; it does not yet say whether they are pleased, which is the applied lab's question.

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.