Bash tests, operators and variables

Why this matters

The previous lesson gave you the shapes — if, case, for, functions. This one is the detail inside the brackets, and it is where scripts go quietly wrong rather than loudly wrong.

A script with a syntax error stops and tells you. A script that compares two numbers as strings runs perfectly and gives the wrong answer, every time, until somebody notices that the disk-space alert never fired because "9" was considered greater than "10".

This is also the densest source of exam questions in the whole domain, because operators are unambiguous. You either know that -gt is numeric and > is not, or you do not.

The lesson

Return codes

Every command that finishes leaves a return code — also called an exit status or exit code — and the shell keeps the most recent one in $?.

ls /etc >/dev/null
echo $?          # 0

ls /nonexistent 2>/dev/null
echo $?          # 2

Zero means success. Anything else means failure. That inversion of the usual convention trips people up, and it exists because there is one way to succeed and many ways to fail, so the non-zero values can carry meaning.

The conventional values:

137 is worth committing to memory. A container or process that keeps exiting 137 was not killed by your script; it ran out of memory and the kernel killed it, exactly as in the process-management lesson.

$? is overwritten by the very next command, which is the trap:

mysqldump appdb > dump.sql
echo "dump finished"
if [[ $? -ne 0 ]]; then ...    # tests ECHO, which always succeeds

Capture it immediately, or test the command directly:

mysqldump appdb > dump.sql
rc=$?
if [[ $rc -ne 0 ]]; then echo "dump failed with $rc" >&2; exit "$rc"; fi

# or, better, no $? at all:
if ! mysqldump appdb > dump.sql; then
    echo "dump failed" >&2
    exit 1
fi

The short-circuit operators use the same status, and read well for one-liners:

mkdir -p /srv/app && cd /srv/app          # AND: run if the first succeeded
systemctl start nginx || echo "start failed" >&2   # OR: run if it failed

Set your own script's status with exit. A script that ends without one exits with the status of its last command, which is rarely what you meant.

exit 0        # say so explicitly

Numerical comparison

Inside [ ] and [[ ]], numbers are compared with letter operators:

used=$(df --output=pcent / | tail -1 | tr -dc '0-9')

if [[ $used -ge 90 ]]; then
    echo "root filesystem is ${used}% full" >&2
fi

[[ $# -eq 0 ]] && { echo "usage: $0 <file>" >&2; exit 1; }

They are letters rather than symbols for a historical reason that still bites: [ is a command, and > on a command line is redirection. [ 5 > 3 ] does not compare anything — it runs [ 5 ], which is true because "5" is a non-empty string, and creates a file named 3. No error, wrong answer, and a stray file.

String comparison

String comparison uses symbols:

if [[ $USER == "root" ]]; then
    echo "do not run this as root" >&2
    exit 1
fi

[[ $answer != "yes" ]] && exit 0
[[ -z $BACKUP_DIR ]] && BACKUP_DIR=/backups

Two notes on CompTIA's published list, which contains typographical errors worth recognising so they do not confuse you in the exam room. The objectives print the regex operator as = ~ and the inequality as ! =, with spaces. A space in either breaks it — they are the two-character operators =~ and !=. This is the same class of slip as vit-manager in objective 1.7.

Three real traps:

Quote the left-hand side. If answer is unset, [ $answer = yes ] expands to [ = yes ], which is a syntax error. [[ $answer == yes ]] is safe because [[ ]] does not word-split — one more reason to prefer it.

Inside [[ ]], the right-hand side of == is a glob pattern, not a plain string:

[[ $file == *.log ]]        # true for anything ending .log -- pattern match
[[ $file == "*.log" ]]      # true only for the literal name *.log

That is useful when you want it and surprising when you do not. Quoting turns it off.

< and > need escaping in [ ] — [ "$a" \< "$b" ] — because the shell would otherwise treat them as redirection. In [[ ]] they work unescaped, and they compare by the shell's collation order, which under a non-C locale is not simple ASCII order.

The two families do not mix

This is the single most important sentence in the lesson: -gt compares numbers, > compares strings, and using the wrong one produces a wrong answer rather than an error.

[[ "10" -gt "9" ]]     # TRUE  -- numeric: ten is more than nine
[[ "10" > "9" ]]       # FALSE -- string: "1" sorts before "9"

A disk-usage check written with > works fine at 95% and fails silently the moment usage reaches 100%, because "100" > "90" is false as a string. Nothing alerts, nothing errors, and the filesystem fills.

The reverse fails loudly, at least: [[ "abc" -eq "def" ]] complains about an invalid arithmetic expression under most shells — though bash will treat unset or non-numeric strings as 0, so [[ $notanumber -eq 0 ]] is quietly true.

Where <= and >= actually live

CompTIA lists <= and >= among the operators, and they are real — but not in [[ ]] for strings. [[ "a" <= "b" ]] is a syntax error. They are arithmetic operators, and they belong inside (( )) or $(( )):

(( count <= 10 ))  && echo "within limit"
(( used >= 90 ))   && echo "disk is filling"
(( a == b ))       && echo "equal"
(( retries != 0 )) && echo "retried"

The arithmetic context is worth using for numbers generally. Inside (( )) you write the operators the way every other language writes them — <, <=, >, >=, ==, != — variables need no $, and the whole expression is evaluated numerically:

if (( free_mb < 500 )); then
    echo "low memory" >&2
fi

Note the one wrinkle: (( )) returns false when the expression evaluates to zero, following C rather than the shell. (( 0 )) is a failure status, so (( count )) is a neat way of asking "is count non-zero" and a trap if you meant something else.

So the mapping the exam wants:

• [[ ]] with -eq, -lt, -ge for numbers in a test
• [[ ]] with ==, !=, =~ for strings
• (( )) with <=, >=, == for arithmetic

Combining tests

if [[ -f $config && -r $config ]]; then ...          # AND
if [[ $env == "prod" || $force == "yes" ]]; then ... # OR
if [[ ! -d $dir ]]; then mkdir -p "$dir"; fi         # NOT

&& and || work inside [[ ]]. Inside [ ] you need -a and -o, which are deprecated and parse ambiguously — another reason [[ ]] is the default in bash.

Variables and assignments

Assignments in bash have one rule that catches everybody:

name=web01           # correct
name = web01         # WRONG: runs the command "name" with two arguments
name= web01          # WRONG: runs "web01" with name set to empty

No spaces around =. The error message — name: command not found — does not mention the space.

More on assignments:

count=0
path="/srv/app data"          # quote anything with spaces
readonly MAX_RETRIES=3        # cannot be changed afterwards
files=(one.txt two.txt)       # an array
echo "${files[1]}"            # arrays are zero-indexed
echo "${#files[@]}"           # element count

Convention, not syntax: lower case for your script's own variables, UPPER CASE for environmental ones. Following it keeps you from overwriting PATH with a loop counter.

Shell variables versus environmental variables

A plain assignment creates a shell variable, visible only to the current shell. Environmental variables are additionally passed to every child process.

greeting=hello                # shell variable
bash -c 'echo $greeting'      # prints nothing -- the child never saw it

export greeting=hello         # now environmental
bash -c 'echo $greeting'      # prints hello

export EDITOR=vim             # export an existing variable
export -n greeting            # demote it back to shell-only

export is the single fact behind a whole class of "the script works when I run it but not from cron" problems: cron gives you a minimal environment, so a variable you rely on from .bashrc is simply not there. Set what you need inside the script rather than assuming the environment.

Inspecting them:

env                # environmental variables only
printenv PATH      # one of them
set                # ALL variables and functions -- shell ones included
declare -p name    # how one variable is actually defined

set with no arguments listing everything is a genuinely useful debugging move, and it is a different set from the one below, which is the same builtin doing a second job.

Arguments

A script receives its arguments as numbered positional parameters:

#!/usr/bin/env bash
echo "script name: $0"
echo "first: $1   second: $2"
echo "count: $#"
echo "all: $@"

"$@" and "$*" differ, and it matters. Quoted, "$@" expands to one word per argument, preserving anything containing spaces; "$*" collapses them into a single word. Use "$@" when passing arguments on — nearly always — and "$*" only when you actually want one string.

for arg in "$@"; do echo "[$arg]"; done     # one line per argument

shift discards $1 and renumbers the rest, which is how you walk an argument list:

while [[ $# -gt 0 ]]; do
    case $1 in
        -v|--verbose) verbose=1; shift ;;
        -f|--file)    file=$2; shift 2 ;;
        *)            echo "unknown option: $1" >&2; exit 1 ;;
    esac
done

Validate before you use them. ${1:?usage: backup.sh <dir>} from the previous lesson is the compact form, and a script taking a path should always check that what arrived is what it expected.

local

Inside a function, local confines a variable to that function:

process() {
    local i
    local file=$1
    for i in {1..3}; do echo "$file pass $i"; done
}

Without it every variable is global, so a function's loop counter overwrites the caller's. This is the second time this lesson has appeared and it is worth the repetition: it is the most common source of action-at-a-distance bugs in shell scripts, and local is one word.

set

set changes how the shell itself behaves:

set -e            # exit on error
set -u            # unset variable is an error
set -o pipefail   # a pipeline fails if any stage fails
set -x            # PRINT EVERY COMMAND BEFORE RUNNING IT
set +x            # turn that back off
set -euo pipefail # the standard opening line

set -x is the debugger. Turn it on, run the script, and every command appears with its variables already expanded — which shows you immediately that the variable you thought held a path is empty. Wrap just the suspect section in set -x / set +x rather than tracing the whole script.

set -- also replaces the positional parameters, which is occasionally the neatest way to re-parse something:

set -- one two three
echo "$2"          # two

unset, alias and unalias

unset removes a variable or function entirely, which is not the same as setting it empty:

unset greeting
unset -f myfunction
[[ -z $greeting ]]           # true for empty OR unset
[[ -v greeting ]]            # true only if SET -- the distinction, when it matters

Under set -u the difference becomes loud: an unset variable aborts the script while an empty one does not.

An alias is a shorthand the interactive shell expands:

alias ll='ls -alF'
alias rm='rm -i'
alias                        # list all of them
unalias rm                   # remove one
unalias -a                   # remove all

Two things the exam likes. Aliases are not expanded in non-interactive shells, so an alias defined in .bashrc does not exist inside a script — which is why scripts use functions instead. And an alias only substitutes the first word of a command, so it takes no arguments in the middle; anything needing that is a function.

alias rm='rm -i' is also a trap of its own. Getting used to being prompted means that on a machine without the alias — a server you have just logged into, or inside a script — rm deletes silently and you have already pressed Enter.

Bypass an alias for one invocation with a backslash or the full path:

\rm file
/bin/rm file
command rm file

On the exam

• 0 is success; non-zero is failure. $? holds the last one and is overwritten by the next command, so capture it immediately.
• 127 is command not found, 126 is not executable, 130 is Ctrl-C, 137 is SIGKILL — usually the OOM killer.
• Numbers: -eq -ne -lt -le -gt -ge. Strings: = == != < > =~.
• [[ "10" > "9" ]] is false — string comparison. The wrong family gives a wrong answer, not an error.
• [ 5 > 3 ] creates a file called 3.
• <= and >= are arithmetic — (( )), not [[ ]].
• CompTIA's list prints = ~ and ! =; the operators are =~ and != with no space.
• No spaces around = in an assignment.
• export makes a variable environmental so children inherit it — the reason scripts break under cron.
• "$@" preserves each argument; "$*" joins them. ${10} needs braces.
• local inside functions, or you are writing to globals.
• set -euo pipefail to be strict, set -x to trace.
• unset removes; empty and unset differ under set -u.
• Aliases do not work in scripts — use a function. unalias removes one, unalias -a removes all.