The shell environment and redirection

Why this matters

Everything else in this course is typed into a shell. Two things about it repay learning properly rather than by accident: the environment, which explains why a command works for you and not for cron, and redirection, which is how you capture output, discard noise, and chain small tools into something useful.

Redirection in particular is where beginners lose data. > silently destroys the file it writes to. Knowing the difference between > and >> before you need it is worth more than any other five seconds in this lesson.

The lesson

Environment variables

The environment is a set of name/value pairs every process inherits from its parent. Read one with $NAME, list them all with env or printenv.

echo $PATH                    # /usr/local/bin:/usr/bin:/bin
export PATH=$PATH:/opt/tools  # append, for this shell and its children
env | sort | less             # everything

Two traps worth internalising now.

export is what makes a variable inheritable. FOO=bar sets it for the current shell only; export FOO=bar passes it to every command the shell launches. A script that "cannot see" a variable you just set is nearly always missing an export.

PATH is why cron jobs fail. Cron runs with a minimal environment — often just /usr/bin:/bin. A script that runs perfectly for you and fails silently from cron is usually calling a command that is not on cron's much shorter PATH. Use absolute paths in cron jobs, or set PATH explicitly at the top.

SHELL reports the login shell from /etc/passwd, not the shell you are typing into. Run bash inside zsh and $SHELL still says zsh. To know what is actually running, check ps -p $$.

Paths: absolute and relative

An absolute path starts at the root and is unambiguous from anywhere: /var/log/messages. A relative path is interpreted from your current directory: log/messages means something different depending on where you stand.

Three shorthands do most of the work:

• . — the current directory
• .. — the parent directory
• ~ — your home directory

cd ..            # up one level
cd ../..         # up two
cp file ../      # copy into the parent
./script.sh      # run a script here — the ./ is required, because . is not on PATH

That last line is a rule people trip over constantly. The current directory is deliberately not in PATH, so a script in front of you will not run by name alone. This is a security decision, not an oversight: if . were on PATH, dropping a malicious file named ls into a shared directory would be enough to attack anyone who cd'd there.

Use absolute paths in anything automated. A script that assumes its working directory works until the day something runs it from elsewhere.

Shell configuration files

Which file your settings belong in depends on which kind of shell reads it, and this is the most confusing corner of the shell for newcomers.

• ~/.bash_profile — read by login shells. Runs once when you log in.
• ~/.bashrc — read by interactive non-login shells. Runs for every new terminal window.
• ~/.profile — the older, shell-agnostic equivalent of .bash_profile. Bash reads it only if .bash_profile does not exist.

The practical rule: aliases and prompt settings go in .bashrc; environment variables and one-time setup go in .bash_profile. Because most systems have .bash_profile source .bashrc, putting everything in .bashrc usually works — which is exactly why the distinction is confusing until it bites you over SSH.

System-wide equivalents live in /etc/profile and /etc/bash.bashrc, and apply to every user.

Changes take effect on the next shell — or immediately with:

source ~/.bashrc     # run it in the current shell
. ~/.bashrc          # identical, and what you will see in scripts

source matters because running ./~/.bashrc would execute it in a child shell, whose variables vanish when it exits. source runs the file in the current shell, which is the entire point.

The three channels

Every process opens three streams:

Errors go to a separate channel so you can keep them apart from results — the whole reason the split exists.

Redirection

command > file      # stdout to file, OVERWRITING it
command >> file     # stdout to file, appending
command 2> file     # stderr to file
command > file 2>&1 # both to the same file
command &> file     # both — shorthand, bash-specific
command < file      # read stdin from file
command 2>/dev/null # discard errors

> truncates its target the moment the command starts — before it has produced any output, and even if the command then fails. cat missing.txt > important.log destroys important.log and gives you nothing back.

The ordering of > file 2>&1 matters and is a classic exam item. It reads left to right: send stdout to the file, then point stderr at wherever stdout currently goes. Reverse it — 2>&1 > file — and stderr is aimed at the terminal before stdout is redirected, so errors still appear on screen.

Here documents and here strings

A here document feeds a literal block into a command's stdin:

cat <<EOF > /etc/motd
Welcome to the server.
Unauthorised access is prohibited.
EOF

Everything up to the terminator becomes input. Quote the delimiter — <<'EOF' — to stop the shell expanding $variables and backticks inside the block, which is essential when writing out scripts or configuration containing literal $.

A here string is the one-line version:

grep root <<< "$USER_LIST"

Pipes, and re-running commands

A pipe connects one command's stdout to the next command's stdin:

ps aux | grep nginx | wc -l

Note that a pipe carries stdout only. Errors still go to your terminal unless you redirect them explicitly — which is often what you want, and occasionally a nasty surprise when a silent failure means the pipeline processes nothing.

The shell keeps a history, and two shortcuts save real time:

!!            # the previous command
sudo !!       # re-run the last command with sudo — the classic use
!ssh          # the most recent command starting with "ssh"

!! after a permission-denied error is probably the single most-used shortcut in Linux administration.

On the exam

• > overwrites, >> appends. Expect a scenario where the wrong one destroys a log.
• > file 2>&1 captures both streams; 2>&1 > file does not. Order is the whole question.
• 2>/dev/null discards errors — recognise it in a command line you are asked to explain.
• export makes a variable available to child processes. Without it, the variable exists only in the current shell.
• .bashrc for interactive shells, .bash_profile for login shells.
• source file and . file are the same command, and both differ from executing the file.
• ./script.sh needs the ./ because the current directory is not on PATH, and know that this is deliberate.