A data workbench you can break, and the practice data sets

Listen to this lesson

Episode 1 · 38:33

Every episode of this course is also a podcast: listen on Spotify.

This episode is a study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.

Supplementary

This lesson maps to no DA0-002 objective. CompTIA does not examine "set up a workbench", and nothing in it will be asked of you as such. It exists because every lab in this course asks you to query, clean, reshape and summarise data, and you need somewhere to do that where a mistake costs nothing. Its material still overlaps the exam: the tools are 1.4, the file formats are 1.1, the reproducible environment touches containerization in 1.3, the dirty data sets are what 2.2 and 2.3 are about, and keeping raw data untouched with a record of every step is documentation and versioning in 5.1.

Why this matters

Data+ is a scenario exam. It will describe a report whose totals do not match, a column full of mixed date formats, or a chart that exaggerates a small change, and ask what you should do. You can learn the answers from a list, but you will recognise them much faster if you have caused each of those problems yourself and watched what happened. That needs a place to cause them.

Unlike a server course, this one needs no special hardware. Everything runs on an ordinary laptop with free software. This lesson sets that up, introduces the practice data sets every later lab uses, and establishes two habits that separate a trustworthy analysis from a lucky one.

The lesson

Why an analyst needs a sandbox where a wrong DELETE costs nothing

Real analytical work happens against data that matters to someone: a sales database, a patient register, a finance extract. The most damaging mistakes in data work are quiet ones. A DELETE or UPDATE without the right WHERE clause changes every row instead of one. A join on the wrong key doubles a revenue figure, and the doubled number looks perfectly plausible. A cleaning step overwrites the original values, and nobody can check afterwards what the data said before.

A sandbox is a copy of the data, on your own machine, that nobody else relies on. In a sandbox you can run the wrong query on purpose to see what it does, which is how you learn to recognise it when you run it by accident at work. The labs in this course will regularly ask you to make a mistake first and then find it, because finding errors is a large part of what the exam tests.

Two rules make a sandbox safe. It contains copies, never the only version of anything. And it is isolated: nothing you do there should be able to reach a production database, a shared drive or a live report.

Python, SQLite and a terminal: the whole toolkit, all of it free

The course uses three tools, all free and available on Windows, macOS and Linux.

SQLite is a complete relational database stored in a single file. There is no server to install or run: you open the file and query it with standard SQL. It is the easiest way to practise the SQL the exam expects, and the same SELECT, JOIN and GROUP BY work on the large database servers you will meet at work. Python includes SQLite through its built-in sqlite3 module, so installing Python gives you both.

Python is the general-purpose language the course uses for cleaning, statistics and charts. Its standard library can already read CSV and JSON files and calculate means, medians and standard deviations. Later lessons add the pandas library for table-shaped data and matplotlib for charts, both installed with Python's package manager, pip.

A terminal (Command Prompt or PowerShell on Windows, Terminal on macOS and Linux) is where you run scripts and check results. Many lab verification steps are short commands that count rows or compare files, because counting is the fastest way to notice that data has changed when it should not have.

You will also meet spreadsheets and business intelligence tools in domain 1, and they have their place. They are not the course's main workbench because a spreadsheet hides its steps: a sorted column or a pasted value leaves no record of how it got there.

The course's practice data sets, and why several are dirty on purpose

Every lab uses the same small collection of practice data sets, generated for this course. The people, places and products in them are fictional. They describe a small online shop:

  • products and customers, as tidy CSV files;
  • a dirty copy of the customers, customers_raw, with the faults real exports contain;
  • a year of orders, two thousand rows;
  • shop.db, an SQLite database holding the products, customers and orders as related tables;
  • a web server log of five thousand requests;
  • a survey export in nested JSON, with free-text comments;
  • a sensor series of hourly temperature readings.

Several of these are dirty on purpose, and the faults are counted. The raw customer file contains customers entered twice with slightly different details, a region spelled several ways, missing email addresses, dates in two formats and names padded with spaces. The orders include a few genuine bulk orders that look like outliers but are real, and a few rows that cannot be real at all. The sensor series has a gap and some spikes.

Because every fault was planted deliberately and counted, a lab can tell you exactly what you should find. That is what makes it a lab rather than an exercise: if your cleaning finds fewer duplicates than were planted, you missed some, and if it finds more, it merged customers who were different people.

Keeping raw data read-only and working on copies

The first habit is simple: never edit the raw data. Keep the files exactly as they arrived, mark them read-only if your system allows it, and do every piece of work on a copy or in a new file.

This matters for three reasons. When a result looks wrong, you need the original to check it against. When a cleaning rule turns out to be mistaken, you need to run it again from the start, not undo it by hand. And when someone asks where a number came from, the answer has to lead back to data nobody has altered.

A practical layout is a folder for raw inputs that you never write to, a folder for cleaned or intermediate data that your scripts produce, and a folder for outputs such as reports and charts. Anything in the second and third folders can be deleted and rebuilt, and that is the test of whether the layout is working.

Recording every step in a script, so an analysis can be run again

The second habit is to do the work in a script rather than by hand. A script is a written record of every step: which file was read, which rows were removed and why, how a column was converted. Run it again and you get the same result. Change one step and you can see exactly what changed.

This is reproducibility, and it is the difference between an analysis you can defend and one you can only describe. It also connects to three ideas the exam returns to. Documentation: the script is the most honest record of what was done. Versioning: keeping scripts in a version control system such as Git shows how an analysis changed over time. Lineage: a script that reads from raw files and writes to output files is a small, readable map of where every number came from.

A reproducible environment extends the same idea to the tools themselves. Recording which versions of Python and its libraries you used, often in a requirements.txt file, means the analysis still runs next year. Packaging the whole environment in a container takes that further, and the infrastructure lesson in domain 1 explains how.

Practise what you just read

1. An analyst's cleaning script produces a customer count that looks too low. What makes it possible to find out which step removed the missing customers?

Select one

  1. The cleaned file was saved over the raw one to keep the folder tidy
  2. The raw file was left untouched and every step is recorded in a script
  3. The analyst remembers roughly which filters were applied during the work
  4. The spreadsheet's undo history still holds the last hundred changes
Show answer

B. Finding a faulty step needs the original data to compare against and a record of every step to replay. A script run against untouched raw data provides both; memory and undo history are incomplete and disappear.

2. Why does the course prefer a script to a spreadsheet as the main place where data is cleaned and summarised?

Select one

  1. A script records every step, so the analysis can be read and run again
  2. Spreadsheets stop opening a CSV once it passes a few hundred lines
  3. Scripts store data more compactly, so cleaned files take less disk space
  4. Spreadsheet formulas are not allowed in reports sent outside the company
Show answer

A. A sorted column or a pasted value in a spreadsheet leaves no record of how it got there. A script is a written, repeatable record of each step, which is what reproducibility means and what makes a result defensible.

3. A practice data set's raw customer file has 20 planted blank emails. An analyst's profile reports 14. What does that tell them?

Select one

  1. The generator is random, so a different count on each run is expected
  2. Six emails were repaired automatically when the file was first opened
  3. Blank emails are unimportant, so a partial count is good enough here
  4. Their check missed some blanks, because every planted fault was counted
Show answer

D. The practice faults were planted deliberately and counted, which is what lets a lab state exactly what should be found. Finding fewer than were planted means the check missed some; finding more means it flagged good rows.

7 more questions on this objective are part of the full course.

Practise the full question bank in the exam simulator

Hands-on labs

All hands-on labs

This is an independent study companion for CompTIA Data+ DA0-002 and is not produced by or endorsed by CompTIA.