Automating common server tasks

Why this matters

The previous lesson covered how scripts are written. This one covers what they are used for on servers and how to run them without anyone having to remember. A backup script that someone must start by hand will eventually not be started. A cleanup job that fails silently will be discovered only when the disk it was meant to protect fills up.

Automation is only an improvement if it is reliable, and reliability comes from three things this lesson covers: running tasks on a schedule or at the right moment, having each script record what it did and complain when it fails, and keeping scripts where changes to them can be tracked.

The lesson

Scheduling with cron and Task Scheduler

Both main server platforms have a built-in scheduler.

On Linux, cron runs commands at set times. Each user has a crontab, edited with crontab -e and listed with crontab -l, and system-wide jobs live in /etc/crontab and /etc/cron.d. Each line has five time fields followed by the command:

• minute (0-59), hour (0-23), day of the month (1-31), month (1-12), and day of the week (0-7, where both 0 and 7 are Sunday);
• an asterisk means "every", so 30 2 * * * runs at 02:30 every day, and 0 * * * 1 runs on the hour, every hour, on Mondays;
• a step value such as */15 in the minute field means every fifteen minutes.

Modern Linux systems also offer systemd timers, which can run missed jobs after a server was powered off at the scheduled time, and log to the journal.

On Windows, Task Scheduler runs tasks on triggers: a time or schedule, at system startup, at user logon, or when a particular event appears in the event log. Tasks can be created in the graphical console, with the schtasks command, or with PowerShell.

Scheduled jobs have two common traps. They run with a minimal environment, so a script that works when run by hand may fail under cron because a path or variable it relied on is missing; use full paths in scheduled scripts. And they run as a particular account, which needs exactly the permissions the task requires. On Windows, a task set to run only when a user is logged on will not run on a server where nobody is.

Scripts for user creation, log rotation and cleanup

Some tasks are automated on almost every server.

User creation. Creating accounts by hand invites inconsistency: a missed group, a mistyped department, a forgotten home folder. A script reads new users from a list, often a CSV file, and creates each one the same way: New-ADUser in PowerShell for Active Directory, or useradd on Linux, followed by group memberships and folders. The matching deprovisioning script, disabling accounts and removing access when people leave, matters even more for security.

Log rotation. Logs grow for ever unless something stops them. Rotation closes the current log, renames it, starts a fresh one, compresses older files and deletes the oldest once they pass the retention period. On Linux, logrotate does this from a configuration describing each log's schedule and retention. Windows event logs have maximum sizes and overwrite rules instead, while application logs often need a scheduled cleanup script.

Cleanup. Temporary files, old installers, stale exports and expired backups accumulate. A scheduled cleanup deletes files older than a set age from known locations. It is also the most dangerous kind of automation, because it deletes things unattended, so it should work on explicit paths, never on a path built from a variable that might be empty, and log everything it removes.

Startup and shutdown scripts

Some tasks belong at a particular moment rather than a particular time.

Startup scripts run when a server boots: mapping resources, starting an application in the right order, or checking that a dependency is available before a service starts. On Linux, this is usually done with a systemd unit, which can declare dependencies so it starts only after the network or a database is up. On Windows, startup scripts can be assigned through Group Policy, or run by a scheduled task triggered at startup; logon scripts, by contrast, run when a user signs in.

Shutdown scripts run as a server stops, to close an application cleanly, flush data to disk or notify other systems. They matter because a service killed abruptly can leave corrupted data or locked files. They also have a limit: a power failure or crash skips them entirely, so nothing essential should depend on a shutdown script running.

The usual rule is to prefer the platform's own service management over a script where possible. A service configured to start automatically, with recovery options to restart it if it fails, is more reliable than a script that starts it.

Logging what a script did, and failing loudly when it cannot

A script running unattended at 02:00 has nobody watching it. The only record of what happened is what it writes down, and the only way anyone learns it failed is if it says so.

Logging. Each run should record when it started and finished, what it did, and any errors, with timestamps, in a known location. That turns "did the cleanup run last night?" into a question with an answer.

Checking for errors. Every command reports success or failure through an exit code: 0 means success, and anything else means failure. Scripts should check it rather than carrying on regardless. In Bash, $? holds the last exit code, and set -e makes the script stop at the first failing command; in PowerShell, try and catch blocks handle errors, and -ErrorAction Stop turns errors that would otherwise be reported and ignored into ones that can be caught. A script that ignores a failed step and continues can do far more harm than one that stops.

Failing loudly. When a script cannot do its job, it should exit with a non-zero code of its own, so the scheduler records the failure, and send an alert, by email or into the monitoring system, so a person finds out. The worst automation is the kind that fails silently for weeks, such as a backup job whose errors go unread until the day a restore is needed.

Keeping scripts in version control

Scripts are code, and they deserve the same care as any other code. A folder of files called cleanup.ps1, cleanup-new.ps1 and cleanup-final-fixed.ps1 is a sign that nobody knows which version is running or what changed.

Version control, usually Git, records every change to a script: what changed, when, who made it and, in the commit message, why. It provides:

• history, so you can see exactly what a script looked like when something went wrong;
• rollback, returning to the last working version in seconds;
• review, where a second person checks a change before it is used;
• a single source, so every server runs the same version rather than a hand-edited copy.

One rule matters above all: never commit passwords or other secrets to version control. Once a secret is in the history it is very hard to remove, and anyone who can read the repository can read it. Keep credentials in a secure store, such as a password vault or the platform's credential manager, and have scripts retrieve them when they run.