Score comment sentiment and check it against the numbers

applied · 50 min · Objective 1.5

Task

Build a simple keyword sentiment classifier for the survey comments, then check it against the satisfaction score the same respondents gave. Where the two disagree, look at why.

Steps

  1. Classify each non-blank comment as positive if it contains any of quick, easy, great, friendly or again, negative if it contains any of late, failing, damaged, hard or expensive, and neutral otherwise.
  2. For each non-blank comment, record response_id, sentiment and satisfaction in lab/sentiment/scored.csv with header response_id,sentiment,satisfaction.
  3. Treat a satisfaction of 4 or 5 as positive and 1 or 2 as negative. For the comments whose satisfaction is not 3, count how often your sentiment matches, and record it in lab/sentiment/agreement.txt as agree: N of M.
  4. Write in lab/sentiment/notes.txt what the respondents who scored 3 wrote, and why a keyword list cannot place them.

Verify

These checks run in a POSIX shell with Python 3: Terminal on macOS or Linux, and on Windows Git Bash (it comes with Git for Windows) or WSL. On Windows, python3 is often a Microsoft Store placeholder that prints “Python was not found”; run alias python3=py in Git Bash first.

awk -F, 'NR>1 {c[$2]++} END {for (s in c) print s": "c[s]}' lab/sentiment/scored.csv | sort
grep -E '^agree: [0-9]+ of [0-9]+$' lab/sentiment/agreement.txt
awk -F, 'NR>1 {n++} END {print n" scored comment(s)"}' lab/sentiment/scored.csv

169 comments are scored, and on every comment whose satisfaction is not 3 the keywords agree with the score. The middle scorers wrote a mix of praise and complaint, which is exactly where keyword counting stops being enough.

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.