Write a questionnaire whose answers you could check

short · 35 min · Objective 5.3

Task

Take a set of typical vendor questionnaire items and rewrite each so the answer is verifiable, then mark which ones you could still not check. The unverifiable remainder is what you need independent evidence for.

Steps

  1. Write /tmp/questionnaire-before.md with ten typical items in their usual unhelpful form: 'Do you encrypt data at rest?', 'Do you have a security policy?', 'Do you perform penetration testing?', and so on.
  2. For each, rewrite it so that the answer is specific and checkable: ask for the algorithm and key length, the policy document itself with its approval date, the date and scope of the last test and whether findings were retested.
  3. Write /tmp/questionnaire-after.csv as item,question,evidence_requested,verifiable, marking each yes or no.
  4. For every item still marked unverifiable, name the independent evidence that would settle it — a SOC 2 Type II section, a certificate with its scope statement, a right-to-audit exercise.
  5. Finally, write the sentence that makes a questionnaire enforceable: the clause incorporating the answers into the agreement as representations.

Verify

python3 - <<'PY'
import csv
rows=list(csv.DictReader(open('/tmp/questionnaire-after.csv')))
assert len(rows)>=10, 'fewer than ten items'
for r in rows:
    assert r['evidence_requested'].strip(), 'no evidence requested for item '+r['item']
    q=r['question'].lower()
    assert not q.startswith('do you have'), 'item %s is still a yes/no: %s' % (r['item'], q[:50])
v=[r for r in rows if r['verifiable'].strip().lower().startswith('y')]
print('%d of %d items verifiable from the answer alone' % (len(v),len(rows)))
assert len(v)<len(rows), 'every item claimed verifiable - some genuinely are not, and saying so is the finding'
PY
grep -ciE "representation|incorporat|agreement" /tmp/questionnaire-after.csv /tmp/questionnaire-before.md

The assertion that not everything is verifiable is deliberate. Some questions genuinely cannot be settled by an answer — culture, practice, whether the policy is followed — and marking those honestly is what tells you where independent assessment is required rather than optional.

Notes

The incorporation clause is the cheapest thing in this lab and the most valuable. An answer that is merely collected is paperwork; an answer incorporated into the contract as a representation is something you can act on when it turns out to be false, which is the only mechanism a questionnaire has.

This is an independent study companion for CompTIA Security+ SY0-701 and is not produced by or endorsed by CompTIA.