Backup and restore

Why this matters

The objective is called backup and restore, and the second word is the one that matters. An untested backup is a belief, not a control — and the belief is usually wrong for a specific reason: the backup ran successfully every night while excluding the database, or writing to a full disk, or capturing an inconsistent copy of files that were being written at the time.

Nothing is a backup until it has been restored. Everything below is mechanism; the discipline is testing the restore.

The lesson

tar: the archiver

tar bundles many files into one, preserving permissions, ownership, timestamps and symlinks.

tar -cvf archive.tar /srv/data           # create
tar -czvf archive.tar.gz /srv/data       # create + gzip
tar -cjvf archive.tar.bz2 /srv/data      # bzip2
tar -cJvf archive.tar.xz /srv/data       # xz
tar -tzvf archive.tar.gz                 # LIST without extracting
tar -xzvf archive.tar.gz                 # extract here
tar -xzvf archive.tar.gz -C /restore     # extract somewhere specific
tar -xzvf archive.tar.gz path/to/one/file    # one file only

The flags: c create, x extract, t list, v verbose, f file, z gzip, j bzip2, J xz.

Useful in practice:

tar -czvf backup.tar.gz --exclude='*.log' --exclude='cache' /srv/app
tar -czvf backup.tar.gz -T filelist.txt        # files listed in a file
tar -czvf - /srv/data | ssh backup@host 'cat > backup.tar.gz'   # straight to another host
tar --diff -f archive.tar /srv/data            # compare archive against disk

Always -t a new archive before trusting it. It costs seconds and confirms the archive is readable and contains what you expect — which is the cheapest form of the "test the restore" discipline.

tar preserves ownership only when extracted as root; as an ordinary user everything becomes yours, which matters when restoring a system directory.

cpio

Older, and still present because two things use it: the initramfs is a cpio archive, and find | cpio handles selection more flexibly than tar.

find /srv/data -print | cpio -ov > archive.cpio        # create
cpio -idv < archive.cpio                               # extract
cpio -itv < archive.cpio                               # list
find . -name "*.conf" | cpio -pdmv /backup/            # pass-through copy

-o create, -i extract, -p pass-through, -d create directories, -m preserve times, -v verbose.

You will meet it unpacking an initramfs to see why a system will not boot:

zcat /boot/initramfs-$(uname -r).img | cpio -idv

Compression

gzip file.txt              # REPLACES file.txt with file.txt.gz
gzip -k file.txt           # keep the original
gunzip file.txt.gz
bzip2 file.txt ; bunzip2 file.txt.bz2
xz file.txt ; unxz file.txt.xz
xz -9 file.txt             # maximum compression
7z a archive.7z /srv/data  # 7-Zip: cross-platform, strong ratio
7z x archive.7z
unzip archive.zip          # ZIP, for interoperability with Windows
unzip -l archive.zip

Note gzip, bzip2 and xz replace the original file unless you pass -k, which surprises people.

Choosing: gzip when speed matters or the archive is short-lived; xz for long-term storage where compression time is paid once and space is saved forever; zstd increasingly for both, being nearly as fast as gzip and nearly as small as xz.

Reading compressed files without decompressing

zcat access.log.gz                 # print
zless access.log.gz                # page through
zgrep "404" access.log.gz          # search
zgrep -c "ERROR" /var/log/*.gz     # count across rotated logs
bzcat / bzless / bzgrep            # bzip2 equivalents
xzcat / xzless / xzgrep            # xz equivalents

zgrep across rotated logs is the everyday use. Investigating something that happened last week means searching messages-2026090*.gz, and zgrep does it without unpacking gigabytes to disk first.

rsync: the workhorse

rsync copies only what has changed, which makes it the right tool for repeated backups.

rsync -avz /srv/data/ /backup/data/              # local
rsync -avz /srv/data/ user@host:/backup/data/    # over SSH
rsync -avz --delete /srv/data/ /backup/data/     # mirror: remove what is gone
rsync -avzn --delete /srv/data/ /backup/data/    # -n DRY RUN
rsync -avz --exclude='*.tmp' --exclude='cache/' /srv/ /backup/
rsync -avz --link-dest=/backup/prev /srv/ /backup/current/   # hardlink snapshots
rsync -avzP --partial file.iso user@host:/data/  # resume big transfers

Flags: a archive (recursive, preserving permissions, times, symlinks, ownership), v verbose, z compress in transit, P progress + partial.

The trailing slash changes what happens. rsync -a /srv/data/ /backup/ copies the contents of data into /backup. Without the trailing slash, rsync -a /srv/data /backup/ copies the directory itself, producing /backup/data. This catches everyone once, and combined with --delete it can delete the wrong thing.

Always dry-run --delete first. -n shows exactly what would be removed. --delete with a wrong source path will faithfully mirror an empty directory over your backup.

--link-dest is how you get cheap snapshots. Unchanged files become hard links to the previous backup rather than copies, so thirty daily snapshots of a mostly-static 100 GB tree cost barely more than 100 GB, while each snapshot looks like a complete standalone copy.

dd and ddrescue

dd copies blocks, ignoring filesystems entirely.

dd if=/dev/sda of=/backup/disk.img bs=4M status=progress    # whole-disk image
dd if=/backup/disk.img of=/dev/sda bs=4M status=progress    # restore
dd if=/dev/sda of=/backup/mbr.img bs=512 count=1            # just the MBR
dd if=/dev/zero of=/dev/sdb bs=1M                           # wipe

dd copies everything including free space, so images are the size of the disk unless compressed:

dd if=/dev/sda bs=4M | gzip > disk.img.gz

dd has no confirmation and no undo. Transposing if= and of= overwrites the source with the destination. Read the command twice before pressing enter; the traditional expansion of the name is "disk destroyer" and it is earned.

ddrescue is what you use on failing hardware. Where dd stops at the first read error, ddrescue skips bad areas, copies everything readable first, then returns to retry the difficult parts — and keeps a map file so it can resume:

ddrescue -d -r3 /dev/sdb /backup/rescue.img /backup/rescue.map
ddrescue -d -r3 -R /dev/sdb /backup/rescue.img /backup/rescue.map   # reverse pass

The map file is the important part: interrupt it, come back, and it continues rather than starting again. On a dying disk, copy the readable data first and retry the bad sectors later — every extra read attempt on a failing drive risks the rest of it.

Strategy, briefly

3-2-1: three copies, on two different media, one off-site. It survives disk failure, site loss and — because one copy is elsewhere — a mistake that propagates.

Backup types: full (everything), incremental (changes since the last backup of any kind — small, but a restore needs the full plus every incremental in order), differential (changes since the last full — larger, but a restore needs only two pieces).

Two failure modes worth naming:

Ransomware follows your credentials. A backup share mounted read-write on the machine being backed up gets encrypted along with everything else. Pull backups from the backup server, or use append-only or immutable storage.

Live files produce inconsistent copies. A database copied while running is a copy of a moving target and may not restore. Use the application's own dump (pg_dump, mysqldump), or an LVM or filesystem snapshot taken with the application quiesced, so the copy is of a consistent instant.

And test the restore on a schedule. A restore first attempted during an incident is being tested at the worst possible moment.

On the exam

• tar flags: c create, x extract, t list, f file, z gzip, j bzip2, J xz. -t before trusting an archive.
• gzip, bzip2, xz replace the original unless given -k.
• zcat, zless, zgrep read compressed files in place — zgrep across rotated logs is the common case.
• rsync's trailing slash decides contents-versus-directory, and --delete should always be dry-run with -n first.
• --link-dest gives hard-linked snapshots at almost no extra space.
• dd has no undo; ddrescue handles failing disks and keeps a resumable map.
• cpio appears in initramfs work and with find for flexible selection.
• 3-2-1: three copies, two media, one off-site.
• Incremental restores need the full plus every increment; differential needs the full plus one.
• A read-write backup share is encrypted by ransomware along with the host.