Automated Website Backup Script Using Bash and SSH

Welcome back! 👋 Day 10 of the 100 Days Cloud DevOps Challenge, and today we're leveling up - creating our first production automation script! This is where DevOps gets exciting - writing code that runs itself and saves hours of manual work! 🚀

🤔 What Are We Building?

The xFusionCorp Industries production support team needs to automate their website backup process. Currently, someone manually:

Logs into App Server 2
Zips up the website files
Copies the zip to the backup server
Verifies everything worked
Repeats this EVERY. SINGLE. DAY. 😅

Our mission: Automate this entire process with a single bash script that anyone can run!

🎯 Why Backup Automation Matters

Let me paint you a picture of why this is critical:

💀 The Manual Backup Nightmare

Day 1: Junior admin forgets to backup. No one notices.
Day 2: Different admin on duty. Also forgets.
Day 3: Server crashes. Data from last 3 days... GONE! 💀
Cost: Thousands of dollars + customer trust + countless hours

😴 The Human Factor

Humans are amazing, but we:

Forget things when busy
Make typos under pressure
Can't work 24/7
Get sick or go on vacation
Have "more important" tasks

🤖 The Automation Solution

Automated backups:

✅ Never forget
✅ Run at 3 AM without complaining
✅ Work weekends and holidays
✅ Execute consistently every time
✅ Log everything for audit trails

Real-World Impact:

A mid-sized e-commerce company automated their backups and:

Saved 2 hours daily of manual work
Eliminated 3 data loss incidents per year
Passed compliance audits effortlessly
Sleep better at night 😴

💼 Today's Requirements

Let's break down what the production team needs:

The Infrastructure:

App Server 2 (steve@stapp02 - 172.16.238.11)
- Runs the static website at /var/www/html/beta
- Needs local backup at /backup/
Backup Server (clint@stbkp01 - 172.16.238.16)
- Remote backup location at /backup/
- Provides redundancy

The Requirements:

✅ Script name: beta_backup.sh
✅ Location: /scripts/ on App Server 2
✅ Create zip archive: xfusioncorp_beta.zip
✅ Source: /var/www/html/beta
✅ Local save: /backup/
✅ Copy to: clint@stbkp01:/backup/
✅ No password prompts
✅ steve user must be able to run it
✅ No sudo inside script
✅ zip package pre-installed

Why these requirements?

No sudo = Security (least privilege principle)
No password = Automation-friendly
Specific user = Accountability
Local + remote = Redundancy
Standard location = Easy to find

This is real-world production stuff! 🎯

🧩 Understanding the Building Blocks

Before we build, let's understand the components:

Bash Scripting Basics

Bash is the glue that holds Linux automation together:

#!/bin/bash              # Shebang - tells system to use bash
VARIABLE="value"         # Variables store data
command                  # Execute commands
if [ condition ]; then   # Conditional logic
    do_something
fi

Why bash for backups?

✅ Native to all Linux systems
✅ No compilation needed
✅ Can call any system command
✅ Easy to read and maintain
✅ Powerful for file operations

SSH Key-Based Authentication

Remember Day 7? We're using that knowledge!

┌──────────────────┐         ┌──────────────────┐
│  App Server 2    │         │  Backup Server   │
│  (steve)         │         │  (clint)         │
│                  │         │                  │
│  Private Key     │◄───────►│  Public Key      │
│  ~/.ssh/id_rsa   │   SCP   │  authorized_keys │
└──────────────────┘         └──────────────────┘

Without keys:

scp file.zip clint@stbkp01:/backup/
# Password: ▯▯▯▯▯▯▯  ← Can't automate this!

With keys:

scp file.zip clint@stbkp01:/backup/
# ✓ Copied silently  ← Perfect for automation!

The Zip Command

Creating archives:

zip -r archive.zip directory/

-r = Recursive (include subdirectories)
Preserves permissions and structure
Cross-platform compatible
Easy to extract anywhere

🛠️ Let's Build This Script!

Phase 1: Setup SSH Passwordless Authentication

This is the foundation. Without this, automation is impossible!

Step 1: SSH to App Server 2

ssh steve@stapp02
# Password: Am3ric@

We're now on App Server 2 as steve! 🎯

Step 2: Generate SSH Keys (if not exists)

# Check if keys already exist
ls -la ~/.ssh/id_rsa*

If they don't exist:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -N ""

Breakdown:

ssh-keygen - Key generation tool
-t rsa - Use RSA algorithm
-b 4096 - 4096-bit key (strong!)
-f ~/.ssh/id_rsa - File location
-N "" - No passphrase (for automation)

Output:

Generating public/private rsa key pair.
Your identification has been saved in /home/steve/.ssh/id_rsa
Your public key has been saved in /home/steve/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx steve@stapp02

Perfect! Keys generated! ✅

Step 3: Copy Public Key to Backup Server

ssh-copy-id clint@stbkp01
# Password: H@wk3y3 (enter clint's password once)

What just happened:

1. Connected to stbkp01 as clint (with password)
2. Created ~/.ssh/ directory on stbkp01
3. Copied steve's public key to ~/.ssh/authorized_keys
4. Set proper permissions (600)
5. This was the LAST time you'll need clint's password!

Step 4: Test Passwordless SSH

ssh clint@stbkp01 "echo 'SSH connection successful'"

Expected output:

SSH connection successful

No password prompt? SUCCESS! 🎉

Now steve can SSH/SCP to stbkp01 without passwords!

Phase 2: Install Required Packages

The script needs zip to create archives:

# Switch to root temporarily
sudo su -

# Install zip
yum install -y zip

# Verify installation
which zip
zip --version

Output:

Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
This is Zip 3.0 (July 5th 2008), by Info-ZIP.

Exit root:

exit

Now back as steve! ✅

Phase 3: Create Required Directories

Both /scripts and /backup need to exist:

# Create directories (as root)
sudo mkdir -p /scripts
sudo mkdir -p /backup

# Give steve ownership
sudo chown steve:steve /scripts
sudo chown steve:steve /backup

# Verify
ls -ld /scripts
ls -ld /backup

Output:

drwxr-xr-x 2 steve steve 4096 Nov 15 10:00 /scripts
drwxr-xr-x 2 steve steve 4096 Nov 15 10:00 /backup

Perfect! steve can now write to these directories! ✅

Also ensure backup directory exists on remote server:

ssh clint@stbkp01 "sudo mkdir -p /backup && sudo chown clint:clint /backup"

Phase 4: Create the Backup Script

Here's where the magic happens! ✨

vi /scripts/beta_backup.sh

The Script (Simple Version):

#!/bin/bash

##############################################
# Website Backup Script for xFusionCorp Beta
# Description: Creates zip archive and copies to backup server
# Author: Production Support Team
# Date: 2025-11-15
##############################################

# Variables
SOURCE_DIR="/var/www/html/beta"
BACKUP_DIR="/backup"
ARCHIVE_NAME="xfusioncorp_beta.zip"
BACKUP_SERVER="clint@stbkp01"
REMOTE_BACKUP_DIR="/backup"

# Create backup directory if it doesn't exist
if [ ! -d "$BACKUP_DIR" ]; then
    mkdir -p "$BACKUP_DIR"
fi

# Remove old archive if exists
if [ -f "$BACKUP_DIR/$ARCHIVE_NAME" ]; then
    rm -f "$BACKUP_DIR/$ARCHIVE_NAME"
fi

# Create zip archive
echo "Creating backup archive..."
cd /var/www/html
zip -r "$BACKUP_DIR/$ARCHIVE_NAME" beta/

# Check if archive was created successfully
if [ $? -eq 0 ]; then
    echo "Backup archive created successfully: $BACKUP_DIR/$ARCHIVE_NAME"

    # Get archive size
    ARCHIVE_SIZE=$(du -h "$BACKUP_DIR/$ARCHIVE_NAME" | cut -f1)
    echo "Archive size: $ARCHIVE_SIZE"
else
    echo "Error: Failed to create backup archive"
    exit 1
fi

# Copy archive to backup server
echo "Copying archive to backup server..."
scp "$BACKUP_DIR/$ARCHIVE_NAME" "$BACKUP_SERVER:$REMOTE_BACKUP_DIR/"

# Check if copy was successful
if [ $? -eq 0 ]; then
    echo "Backup copied successfully to $BACKUP_SERVER:$REMOTE_BACKUP_DIR/"
    echo "Backup completed at: $(date)"
else
    echo "Error: Failed to copy backup to remote server"
    exit 1
fi

# Display completion message
echo "========================================="
echo "Backup Process Completed Successfully!"
echo "Local backup: $BACKUP_DIR/$ARCHIVE_NAME"
echo "Remote backup: $BACKUP_SERVER:$REMOTE_BACKUP_DIR/$ARCHIVE_NAME"
echo "========================================="

exit 0

Let me explain each section:

1. Variables Section:

SOURCE_DIR="/var/www/html/beta"
BACKUP_DIR="/backup"
ARCHIVE_NAME="xfusioncorp_beta.zip"
BACKUP_SERVER="clint@stbkp01"
REMOTE_BACKUP_DIR="/backup"

Why use variables?

Easy to modify (change once, affects everywhere)
Self-documenting code
Prevents typos
Makes testing easier

2. Directory Check:

if [ ! -d "$BACKUP_DIR" ]; then
    mkdir -p "$BACKUP_DIR"
fi

What this does:

[ ! -d "$BACKUP_DIR" ] - Checks if directory doesn't exist
-d = directory test
! = NOT operator
If directory missing, creates it

3. Remove Old Archive:

if [ -f "$BACKUP_DIR/$ARCHIVE_NAME" ]; then
    rm -f "$BACKUP_DIR/$ARCHIVE_NAME"
fi

Why remove first?

Prevents zip from asking to overwrite
Ensures fresh backup
Avoids file conflicts

4. Create Zip Archive:

cd /var/www/html
zip -r "$BACKUP_DIR/$ARCHIVE_NAME" beta/

Why cd first?

Zip archives include the path from current directory
Ensures beta/ is the top level in the zip
Makes extraction cleaner

5. Error Checking:

if [ $? -eq 0 ]; then
    # Success actions
else
    # Error actions
    exit 1
fi

Understanding $?:

Contains exit code of last command
0 = success
Non-zero = error
Critical for automation!

6. Copy to Remote:

scp "$BACKUP_DIR/$ARCHIVE_NAME" "$BACKUP_SERVER:$REMOTE_BACKUP_DIR/"

SCP explained:

Secure Copy Protocol
Uses SSH for transfer
Syntax: scp local_file user@host:remote_path
With SSH keys = no password needed!

Phase 5: Make Script Executable

chmod +x /scripts/beta_backup.sh

Verify:

ls -l /scripts/beta_backup.sh

Output:

-rwxr-xr-x 1 steve steve 1589 Nov 15 10:30 /scripts/beta_backup.sh

The x means executable! ✅

Phase 6: Test the Script

The moment of truth! 🎬

/scripts/beta_backup.sh

Expected output:

Creating backup archive...
  adding: beta/ (stored 0%)
  adding: beta/index.html (deflated 65%)
  adding: beta/css/ (stored 0%)
  adding: beta/css/style.css (deflated 73%)
  adding: beta/images/ (stored 0%)
  adding: beta/images/logo.png (deflated 2%)
Backup archive created successfully: /backup/xfusioncorp_beta.zip
Archive size: 2.3M
Copying archive to backup server...
xfusioncorp_beta.zip                    100% 2345KB  25.3MB/s   00:00
Backup copied successfully to clint@stbkp01:/backup/
Backup completed at: Fri Nov 15 10:35:22 UTC 2025
=========================================
Backup Process Completed Successfully!
Local backup: /backup/xfusioncorp_beta.zip
Remote backup: clint@stbkp01:/backup/xfusioncorp_beta.zip
=========================================

Beautiful! Everything worked! 🎉

✅ Verification & Testing

Test 1: Local Backup Exists

ls -lh /backup/xfusioncorp_beta.zip

Output:

-rw-r--r-- 1 steve steve 2.3M Nov 15 10:35 /backup/xfusioncorp_beta.zip

File exists! ✅

Test 2: Zip Integrity

unzip -t /backup/xfusioncorp_beta.zip

Output:

Archive:  /backup/xfusioncorp_beta.zip
    testing: beta/                    OK
    testing: beta/index.html          OK
    testing: beta/css/                OK
    testing: beta/css/style.css       OK
    testing: beta/images/             OK
    testing: beta/images/logo.png     OK
No errors detected in compressed data of /backup/xfusioncorp_beta.zip.

Perfect! No corruption! ✅

Test 3: Remote Backup Exists

ssh clint@stbkp01 "ls -lh /backup/xfusioncorp_beta.zip"

Output:

-rw-r--r-- 1 clint clint 2.3M Nov 15 10:35 /backup/xfusioncorp_beta.zip

File on backup server! ✅

Test 4: File Sizes Match

# Local file size
ls -l /backup/xfusioncorp_beta.zip | awk '{print $5}'

# Remote file size
ssh clint@stbkp01 "ls -l /backup/xfusioncorp_beta.zip" | awk '{print $5}'

Both should show same byte count! If they match, transfer was successful! ✅

Test 5: Multiple Runs (Idempotency)

/scripts/beta_backup.sh
/scripts/beta_backup.sh
/scripts/beta_backup.sh

Each run should:

Complete successfully
Replace previous backup
No errors about existing files

This proves the script is idempotent - can run multiple times safely! ✅

Test 6: Contents Verification

unzip -l /backup/xfusioncorp_beta.zip

Output:

Archive:  /backup/xfusioncorp_beta.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  11-15-2025 10:00   beta/
     3456  11-15-2025 09:45   beta/index.html
        0  11-15-2025 10:00   beta/css/
     1234  11-15-2025 09:30   beta/css/style.css
        0  11-15-2025 10:00   beta/images/
   234567  11-15-2025 08:15   beta/images/logo.png
---------                     -------
   239257                     6 files

All website files included! ✅

🎨 Advanced Script Version (Production-Ready)

Want to make it even better? Here's an enhanced version:

#!/bin/bash

##############################################
# Website Backup Script for xFusionCorp Beta
# Description: Creates zip archive and copies to backup server
# Features: Logging, timestamps, verification, error handling
# Author: Production Support Team
# Date: 2025-11-15
##############################################

# Variables
SOURCE_DIR="/var/www/html/beta"
BACKUP_DIR="/backup"
ARCHIVE_NAME="xfusioncorp_beta.zip"
BACKUP_SERVER="clint@stbkp01"
REMOTE_BACKUP_DIR="/backup"
TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")
LOG_FILE="/var/log/beta_backup.log"

# Function to log messages
log_message() {
    echo "[$TIMESTAMP] $1" | tee -a "$LOG_FILE"
}

# Function to check exit status
check_status() {
    if [ $? -eq 0 ]; then
        log_message "SUCCESS: $1"
    else
        log_message "ERROR: $1"
        exit 1
    fi
}

# Start backup process
log_message "========== Backup Process Started =========="

# Check if source directory exists
if [ ! -d "$SOURCE_DIR" ]; then
    log_message "ERROR: Source directory $SOURCE_DIR does not exist"
    exit 1
fi
log_message "Source directory verified: $SOURCE_DIR"

# Create backup directory if needed
if [ ! -d "$BACKUP_DIR" ]; then
    mkdir -p "$BACKUP_DIR"
    check_status "Created backup directory"
fi

# Remove old archive
if [ -f "$BACKUP_DIR/$ARCHIVE_NAME" ]; then
    rm -f "$BACKUP_DIR/$ARCHIVE_NAME"
    log_message "Removed old archive"
fi

# Create zip archive
log_message "Creating backup archive..."
cd /var/www/html
zip -rq "$BACKUP_DIR/$ARCHIVE_NAME" beta/
check_status "Backup archive created"

# Get and log archive size
if [ -f "$BACKUP_DIR/$ARCHIVE_NAME" ]; then
    ARCHIVE_SIZE=$(du -h "$BACKUP_DIR/$ARCHIVE_NAME" | cut -f1)
    log_message "Archive size: $ARCHIVE_SIZE"
else
    log_message "ERROR: Archive file not found after creation"
    exit 1
fi

# Test archive integrity
unzip -t "$BACKUP_DIR/$ARCHIVE_NAME" > /dev/null 2>&1
check_status "Archive integrity verified"

# Copy to backup server
log_message "Copying to backup server $BACKUP_SERVER..."
scp -q "$BACKUP_DIR/$ARCHIVE_NAME" "$BACKUP_SERVER:$REMOTE_BACKUP_DIR/"
check_status "Copied to remote backup server"

# Verify remote file exists
ssh -q "$BACKUP_SERVER" "test -f $REMOTE_BACKUP_DIR/$ARCHIVE_NAME"
check_status "Remote backup verified"

# Get remote file size
REMOTE_SIZE=$(ssh -q "$BACKUP_SERVER" "du -h $REMOTE_BACKUP_DIR/$ARCHIVE_NAME" | cut -f1)
log_message "Remote file size: $REMOTE_SIZE"

# Compare sizes
if [ "$ARCHIVE_SIZE" == "$REMOTE_SIZE" ]; then
    log_message "SUCCESS: Local and remote file sizes match"
else
    log_message "WARNING: File size mismatch (Local: $ARCHIVE_SIZE, Remote: $REMOTE_SIZE)"
fi

# Completion summary
log_message "========== Backup Process Completed =========="
log_message "Local backup:  $BACKUP_DIR/$ARCHIVE_NAME ($ARCHIVE_SIZE)"
log_message "Remote backup: $BACKUP_SERVER:$REMOTE_BACKUP_DIR/$ARCHIVE_NAME ($REMOTE_SIZE)"

exit 0

Enhancements in this version:

✅ Logging to file
✅ Timestamps on all messages
✅ Functions for code reuse
✅ Archive integrity checking
✅ Remote verification
✅ Size comparison
✅ Better error messages
✅ Audit trail

💡 Key Takeaways

✨ Bash scripts automate repetitive tasks

✨ SSH keys enable passwordless automation

✨ Variables make scripts maintainable

✨ Error checking prevents silent failures

✨ Exit codes indicate success/failure

✨ zip -r creates recursive archives

✨ scp transfers files securely over SSH

✨ $? contains last command's exit code

✨ Idempotent scripts can run repeatedly safely

✨ Logging provides audit trails

✨ Local + remote backups provide redundancy

✨ Testing proves reliability

✨ No sudo needed with proper permissions

✨ Scripts should be executable (chmod +x)

✨ Always verify backups after creation

🎓 Interview Questions to Master

Q1: Why is it important to check exit codes ($?) in backup scripts?

Answer: Exit codes are critical for automation and error handling. In bash, $? contains the exit status of the last executed command (0 = success, non-zero = failure). Without checking exit codes, a backup script could fail silently - the zip command might error due to permissions or disk space, but the script continues and reports success! This creates a false sense of security where you think you have backups but actually don't. In production, this can be catastrophic during disaster recovery. Checking $? after critical operations (zip, scp, mkdir) allows the script to: 1) Detect failures immediately, 2) Log specific errors for troubleshooting, 3) Exit with proper status for monitoring systems, 4) Prevent cascading failures (don't try to copy a file that wasn't created). Best practice is wrapping critical commands with if [ $? -eq 0 ]; then success_action; else error_action; exit 1; fi. This makes scripts production-ready and suitable for automated monitoring.

Q2: Explain the difference between using passwords vs SSH keys for automation scripts.

Answer: Password-based authentication fundamentally breaks automation because it requires interactive input. When a script runs scp file user@server:/path/ with password auth, it prompts for password and waits indefinitely for human input - this makes automation impossible. SSH key-based authentication solves this: the private key acts as automatic authentication without prompts. Advantages of SSH keys for automation: 1) Non-interactive - scripts run unattended (cron jobs, CI/CD), 2) More secure - 4096-bit keys are cryptographically stronger than passwords, 3) Revocable - remove public key from authorized_keys to revoke access without changing passwords, 4) Auditable - can track which key accessed what, 5) Rotation-friendly - easier to rotate keys systematically. Implementation: Generate keypair on source server (ssh-keygen), copy public key to destination (ssh-copy-id), script then uses scp/ssh without password prompts. For production automation (backups, deployments, monitoring), SSH keys aren't just preferred - they're the only viable option.

Q3: Why should we avoid using sudo inside automation scripts?

Answer: Using sudo in automation scripts creates multiple problems: 1) Security risk - Scripts with sudo have elevated privileges; if compromised, attackers get root access. Principle of least privilege says grant minimum necessary permissions. 2) Password prompts - sudo typically requires password (unless NOPASSWD configured), breaking automation. 3) Audit trail complications - Actions appear as root, losing individual accountability. 4) Dependency on sudoers configuration - Script breaks if sudoers rules change. Better approaches: a) Proper file ownership - chown user:user /backup so user can write without sudo, b) Directory permissions - chmod 755 /scripts allows user execution, c) Group permissions - Add user to appropriate groups (docker, www-data), d) Capabilities - Use Linux capabilities for specific privileges without full root. When sudo is absolutely needed: Configure NOPASSWD for specific commands in sudoers: steve ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart nginx. This allows automation while limiting scope. For backup scripts specifically, proper directory ownership eliminates sudo need entirely.

Q4: How would you schedule this backup script to run automatically every day?

Answer: Use cron for scheduled automation. Step 1: Edit crontab as steve: crontab -e. Step 2: Add cron entry: 0 2 * * * /scripts/beta_backup.sh >> /var/log/beta_backup_cron.log 2>&1. Breakdown: 0 2 * * * = every day at 2 AM (minute hour day month weekday), /scripts/beta_backup.sh = script to run, >> /var/log/beta_backup_cron.log 2>&1 = append output to log file, redirect stderr to stdout. Why 2 AM? Low traffic time, less resource contention. Best practices: 1) Use absolute paths - cron has limited PATH, 2) Redirect output - capture logs for troubleshooting, 3) Test first - run script manually before scheduling, 4) Monitor execution - set up alerts if backup fails, 5) Log rotation - prevent log files from filling disk. Advanced: Use anacron for systems that aren't always on, or systemd timers for more complex scheduling. Verification: crontab -l to list jobs, grep CRON /var/log/cron to see execution logs. For critical backups, add monitoring: 0 3 * * * test -f /backup/xfusioncorp_beta.zip || echo "Backup failed!" | mail -s "Alert" admin@company.com.

Q5: What are the advantages of creating both local and remote backups?

Answer: This implements the 3-2-1 backup rule (3 copies, 2 different media, 1 offsite). Local backup advantages: 1) Fast recovery - restore from /backup/ in seconds vs network transfer time, 2) No network dependency - works even if network is down, 3) Quick verification - check backup integrity immediately, 4) Lower cost - uses existing server disk. Remote backup advantages: 1) Disaster recovery - if App Server 2 disk fails or server crashes, backup survives, 2) Physical separation - fire, flood, hardware failure on one server doesn't affect backup, 3) Compliance - many regulations require offsite backups, 4) Ransomware protection - if App Server 2 is compromised, backup server remains clean. Real scenario: Server hard drive fails at 3 AM. With only local backup, ALL data lost. With remote backup, restore from stbkp01 and business continues. Cost-benefit: Network transfer adds 30 seconds to backup time but provides disaster recovery insurance. Best practice: Local for quick restores, remote for disaster recovery, both for production peace of mind. Consider adding third copy to cloud (S3/Azure Blob) for ultimate protection.

Q6: How would you verify that a backup script is working correctly in production?

Answer: Multi-layered verification approach: 1) Immediate verification (in script): Check exit codes after each operation ($?), verify file exists (test -f), check file size is non-zero ([ -s file ]), test archive integrity (unzip -t), compare local and remote sizes. 2) Monitoring: Set up cron monitoring that checks if backup file timestamp is recent: find /backup -name "*.zip" -mtime -1 (modified within last day). Alert if no recent backup found. Use monitoring tools (Nagios, Prometheus, Zabbix) to track backup success/failure. 3) Automated testing: Schedule weekly restore test to separate location: unzip -t /backup/xfusioncorp_beta.zip && echo "Backup valid". Create a separate script that does test restore and verification. 4) Manual audits: Monthly check that can actually extract and use backed up data. Compare backup size trends (sudden size changes indicate issues). Review backup logs for warnings or errors. 5) Disaster recovery drills: Quarterly or biannually, actually restore from backup to verify complete recovery process works. Production example: Set up Jenkins job or cron that runs daily after backup: if [ ! -f /backup/xfusioncorp_beta.zip ] || [ ! $(find /backup/xfusioncorp_beta.zip -mtime -1) ]; then send_alert; fi. Remember: Backups you haven't tested are Schrödinger's backups - simultaneously working and not working until you need them!

🚨 Common Mistakes to Avoid

❌ Not testing SSH before running

# Script runs, gets stuck waiting for password
scp file user@server:/path/
Password: ▯▯▯▯  ← Script hangs forever!

✅ Test first:

ssh user@server "echo test"
# Should work without password before running script

❌ Forgetting to make script executable

/scripts/beta_backup.sh
bash: /scripts/beta_backup.sh: Permission denied

✅ Always chmod +x:

chmod +x /scripts/beta_backup.sh

❌ Not checking exit codes

zip -r backup.zip directory/
scp backup.zip server:/backup/
echo "Success!"  # Even if zip failed!

✅ Check every critical operation:

zip -r backup.zip directory/
if [ $? -ne 0 ]; then
    echo "Zip failed!"
    exit 1
fi

❌ Hardcoding paths without variables

zip -r /backup/xfusioncorp_beta.zip /var/www/html/beta/
scp /backup/xfusioncorp_beta.zip clint@stbkp01:/backup/
# Repeated paths = error-prone

✅ Use variables:

ARCHIVE="/backup/xfusioncorp_beta.zip"
zip -r "$ARCHIVE" /var/www/html/beta/
scp "$ARCHIVE" clint@stbkp01:/backup/
# Change once, affects everywhere

❌ Not removing old backup first

zip -r backup.zip directory/
# Zip asks: replace backup.zip? [y/n]
# Script waits forever for input!

✅ Remove old backup:

rm -f backup.zip
zip -r backup.zip directory/

❌ Running as wrong user

sudo /scripts/beta_backup.sh
# SSH keys are user-specific!
# Root doesn't have steve's keys

✅ Run as intended user:

# As steve user
/scripts/beta_backup.sh

🚀 Taking It Further

Add to Cron for Automation

# Edit crontab as steve
crontab -e

# Add daily backup at 2 AM
0 2 * * * /scripts/beta_backup.sh >> /var/log/beta_backup.log 2>&1

# Or weekly on Sunday at 3 AM
0 3 * * 0 /scripts/beta_backup.sh >> /var/log/beta_backup.log 2>&1

# Or hourly
0 * * * * /scripts/beta_backup.sh >> /var/log/beta_backup.log 2>&1

Add Email Notifications

# In script, add email on completion
ADMIN_EMAIL="admin@xfusioncorp.com"

# At end of script
echo "Backup completed successfully at $(date)" | mail -s "Backup Success" $ADMIN_EMAIL

# On error
echo "Backup failed at $(date)" | mail -s "Backup FAILED!" $ADMIN_EMAIL

Add Retention Policy

Clean up old backups:

# Keep only last 7 days locally
find /backup -name "xfusioncorp_beta*.zip" -mtime +7 -delete

# Keep 30 days on backup server
ssh clint@stbkp01 "find /backup -name 'xfusioncorp_beta*.zip' -mtime +30 -delete"

Add Timestamped Backups

Instead of overwriting, keep multiple versions:

TIMESTAMP=$(date +%Y%m%d_%H%M%S)
ARCHIVE_NAME="xfusioncorp_beta_${TIMESTAMP}.zip"

# Creates: xfusioncorp_beta_20251115_143022.zip

Add Slack/Discord Notifications

# Slack webhook integration
SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"

send_slack() {
    curl -X POST -H 'Content-type: application/json' \
    --data "{\"text\":\"$1\"}" \
    $SLACK_WEBHOOK
}

# Use in script
send_slack "✅ Backup completed: $ARCHIVE_NAME ($ARCHIVE_SIZE)"

Add Database Backup

Extend to backup databases too:

# Backup MySQL database
mysqldump -u backup_user -p'password' database_name > /backup/database.sql

# Add to zip
zip -r "$ARCHIVE_NAME" beta/ database.sql

Add Compression Level

Control zip compression:

# Maximum compression (slower, smaller file)
zip -9 -r backup.zip directory/

# Fastest (less compression, faster)
zip -1 -r backup.zip directory/

# Store only (no compression, fastest)
zip -0 -r backup.zip directory/

Add Encryption

For sensitive data:

# Encrypt with password
zip -er backup.zip directory/
# -e = encrypt

# Or use GPG
gpg --symmetric --cipher-algo AES256 backup.zip

🔧 Troubleshooting Guide

Issue: Script Permission Denied

Symptoms:

/scripts/beta_backup.sh
bash: /scripts/beta_backup.sh: Permission denied

Solution:

chmod +x /scripts/beta_backup.sh

Issue: SSH Password Prompt

Symptoms:

scp backup.zip clint@stbkp01:/backup/
Password: ▯

Solution:

# Setup SSH keys
ssh-keygen -t rsa -b 4096 -N ""
ssh-copy-id clint@stbkp01

# Test
ssh clint@stbkp01 "echo test"

Issue: Zip Command Not Found

Symptoms:

./beta_backup.sh
./beta_backup.sh: line 25: zip: command not found

Solution:

sudo yum install -y zip

Issue: Directory Doesn't Exist

Symptoms:

zip: directory/: No such file or directory

Solution:

# Check if source exists
ls -la /var/www/html/beta

# Create if needed
sudo mkdir -p /var/www/html/beta

Issue: Permission Denied Writing Backup

Symptoms:

zip error: Permission denied

Solution:

# Check directory ownership
ls -ld /backup

# Fix ownership
sudo chown steve:steve /backup

Issue: Disk Space Full

Symptoms:

zip error: Disk full

Solution:

# Check disk space
df -h

# Clean up old backups
rm /backup/*.zip.old

# Check largest files
du -sh /backup/*

📊 Monitoring & Maintenance

Check Backup Status

# List recent backups
ls -lht /backup/ | head -10

# Check backup age
find /backup -name "*.zip" -mtime -1
# Should find files modified within last day

# Compare sizes (should be consistent)
ls -lh /backup/xfusioncorp_beta.zip
ssh clint@stbkp01 "ls -lh /backup/xfusioncorp_beta.zip"

Verify Backup Integrity

# Weekly integrity check (add to cron)
#!/bin/bash
unzip -t /backup/xfusioncorp_beta.zip
if [ $? -eq 0 ]; then
    echo "Backup integrity: OK"
else
    echo "Backup integrity: FAILED!" | mail -s "Backup Alert" admin@company.com
fi

Monitor Backup Timing

# Check how long backup takes
time /scripts/beta_backup.sh

# Log execution time in script
START_TIME=$(date +%s)
# ... backup operations ...
END_TIME=$(date +%s)
DURATION=$((END_TIME - START_TIME))
echo "Backup took $DURATION seconds"

Disk Space Monitoring

# Alert if backup partition > 80% full
USAGE=$(df -h /backup | awk 'NR==2 {print $5}' | sed 's/%//')
if [ $USAGE -gt 80 ]; then
    echo "Backup disk usage: ${USAGE}%" | mail -s "Disk Alert" admin@company.com
fi

🎯 Real-World Production Example

Here's a complete production-ready script with all bells and whistles:

#!/bin/bash
#
# Production Website Backup Script
# Company: xFusionCorp Industries
# Purpose: Automated website backup with monitoring
# Version: 2.0
# Author: DevOps Team
# Last Modified: 2025-11-15
#

# Configuration
SOURCE_DIR="/var/www/html/beta"
BACKUP_DIR="/backup"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
ARCHIVE_NAME="xfusioncorp_beta_${TIMESTAMP}.zip"
BACKUP_SERVER="clint@stbkp01"
REMOTE_BACKUP_DIR="/backup"
LOG_FILE="/var/log/beta_backup.log"
RETENTION_DAYS=7
ADMIN_EMAIL="devops@xfusioncorp.com"
SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/WEBHOOK"

# Functions
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

error_exit() {
    log "ERROR: $1"
    send_notification "❌ Backup Failed: $1"
    exit 1
}

send_notification() {
    # Send to Slack
    curl -s -X POST -H 'Content-type: application/json' \
        --data "{\"text\":\"$1\"}" "$SLACK_WEBHOOK" > /dev/null 2>&1

    # Send email
    echo "$1" | mail -s "Backup Notification" "$ADMIN_EMAIL" 2>/dev/null
}

check_prerequisites() {
    # Check if source exists
    [ -d "$SOURCE_DIR" ] || error_exit "Source directory not found: $SOURCE_DIR"

    # Check if zip is installed
    command -v zip >/dev/null 2>&1 || error_exit "zip command not found"

    # Check disk space (need at least 1GB free)
    AVAILABLE=$(df -BG "$BACKUP_DIR" | awk 'NR==2 {print $4}' | sed 's/G//')
    [ "$AVAILABLE" -lt 1 ] && error_exit "Insufficient disk space: ${AVAILABLE}GB free"

    # Check SSH connectivity
    ssh -q -o ConnectTimeout=5 "$BACKUP_SERVER" "echo test" > /dev/null 2>&1
    [ $? -ne 0 ] && error_exit "Cannot connect to backup server"
}

cleanup_old_backups() {
    log "Cleaning up old backups (older than $RETENTION_DAYS days)..."

    # Local cleanup
    find "$BACKUP_DIR" -name "xfusioncorp_beta_*.zip" -mtime +$RETENTION_DAYS -delete

    # Remote cleanup
    ssh -q "$BACKUP_SERVER" \
        "find $REMOTE_BACKUP_DIR -name 'xfusioncorp_beta_*.zip' -mtime +$RETENTION_DAYS -delete"
}

create_backup() {
    log "Creating backup archive: $ARCHIVE_NAME"

    cd /var/www/html || error_exit "Cannot change to source directory"

    zip -rq "$BACKUP_DIR/$ARCHIVE_NAME" beta/ 2>&1 | tee -a "$LOG_FILE"

    [ $? -ne 0 ] && error_exit "Failed to create zip archive"
    [ ! -f "$BACKUP_DIR/$ARCHIVE_NAME" ] && error_exit "Archive file not created"

    ARCHIVE_SIZE=$(du -h "$BACKUP_DIR/$ARCHIVE_NAME" | cut -f1)
    log "Archive created successfully: $ARCHIVE_SIZE"
}

verify_backup() {
    log "Verifying backup integrity..."

    unzip -t "$BACKUP_DIR/$ARCHIVE_NAME" > /dev/null 2>&1

    [ $? -ne 0 ] && error_exit "Backup integrity check failed"

    log "Backup integrity verified"
}

transfer_backup() {
    log "Transferring backup to remote server..."

    scp -q "$BACKUP_DIR/$ARCHIVE_NAME" "$BACKUP_SERVER:$REMOTE_BACKUP_DIR/"

    [ $? -ne 0 ] && error_exit "Failed to transfer backup to remote server"

    # Verify remote file
    ssh -q "$BACKUP_SERVER" "test -f $REMOTE_BACKUP_DIR/$ARCHIVE_NAME"
    [ $? -ne 0 ] && error_exit "Remote backup verification failed"

    REMOTE_SIZE=$(ssh -q "$BACKUP_SERVER" "du -h $REMOTE_BACKUP_DIR/$ARCHIVE_NAME" | cut -f1)
    log "Remote backup verified: $REMOTE_SIZE"
}

generate_report() {
    log "========================================="
    log "Backup Summary:"
    log "  Archive: $ARCHIVE_NAME"
    log "  Local Size: $ARCHIVE_SIZE"
    log "  Remote Size: $REMOTE_SIZE"
    log "  Local Path: $BACKUP_DIR/$ARCHIVE_NAME"
    log "  Remote Path: $BACKUP_SERVER:$REMOTE_BACKUP_DIR/$ARCHIVE_NAME"
    log "  Duration: $(($(date +%s) - START_TIME)) seconds"
    log "========================================="
}

# Main execution
START_TIME=$(date +%s)
log "========== Backup Process Started =========="

check_prerequisites
cleanup_old_backups
create_backup
verify_backup
transfer_backup
generate_report

send_notification "✅ Backup completed successfully: $ARCHIVE_NAME ($ARCHIVE_SIZE)"

log "========== Backup Process Completed =========="
exit 0

This production script includes:

✅ Timestamped backups
✅ Old backup cleanup
✅ Pre-flight checks
✅ Error handling
✅ Integrity verification
✅ Remote verification
✅ Logging
✅ Email notifications
✅ Slack integration
✅ Execution timing
✅ Detailed reporting
✅ Disk space checks
✅ SSH connectivity tests

💡 Lessons Learned Today

Technical Skills:

✅ Bash scripting fundamentals
✅ SSH key-based authentication
✅ Error handling and exit codes
✅ File operations and archiving
✅ Remote file transfer with SCP
✅ Script variables and functions
✅ Conditional logic in bash

DevOps Principles:

✅ Automation over manual processes
✅ Idempotency (safe to run multiple times)
✅ Logging and audit trails
✅ Error handling and recovery
✅ Testing and verification
✅ Documentation in code
✅ Security best practices

Production Readiness:

✅ No passwords in automation
✅ Proper error handling
✅ Monitoring and alerting
✅ Backup verification
✅ Redundancy (local + remote)
✅ Maintenance considerations

🎯 What's Next?

Day 10 complete! 🎉 We've created our first production automation script - a complete backup solution that runs automatically, handles errors gracefully, and provides redundancy through remote backups!

This is real DevOps work:

Writing code that runs itself
Building reliability into systems
Automating away manual toil
Creating audit trails
Thinking about failure scenarios

Remember: Good automation isn't just about running commands - it's about building reliable, maintainable, and monitorable systems!

Tomorrow, Day 11 awaits with new challenges! Keep automating! 💪

Day: 10/100
Challenge: KodeKloud Cloud DevOps
Date: November 15, 2025
Topic: Bash Scripting & Backup Automation

What's your most creative automation script? Share in the comments - let's inspire each other! 🚀

📚 Additional Resources

Bash Scripting:

Backup Strategies:

3-2-1 Backup Rule
Grandfather-Father-Son rotation
Incremental vs Full backups
Backup testing procedures

Commands Quick Reference:

# Zip operations
zip -r archive.zip directory/     # Create recursive archive
unzip -t archive.zip              # Test integrity
unzip -l archive.zip              # List contents
unzip archive.zip -d /path/       # Extract to path

# SSH operations
ssh-keygen -t rsa -b 4096         # Generate key
ssh-copy-id user@host             # Copy public key
ssh user@host "command"           # Remote command
scp file user@host:/path/         # Secure copy

# Script debugging
bash -x script.sh                 # Trace execution
bash -n script.sh                 # Check syntax
set -x                            # Enable debug mode in script
set -e                            # Exit on error

# Cron
crontab -e                        # Edit cron jobs
crontab -l                        # List cron jobs
crontab -r                        # Remove all cron jobs

Next Steps:

Add this script to cron for daily automation
Implement monitoring and alerting
Create restore procedure documentation
Test disaster recovery process
Add encryption for sensitive data
Implement backup rotation strategies

Command Palette

🤔 What Are We Building?

🎯 Why Backup Automation Matters

💼 Today's Requirements

🧩 Understanding the Building Blocks

Bash Scripting Basics

SSH Key-Based Authentication

The Zip Command

🛠️ Let's Build This Script!

Phase 1: Setup SSH Passwordless Authentication

Phase 2: Install Required Packages

Phase 3: Create Required Directories

Phase 4: Create the Backup Script

Phase 5: Make Script Executable

Phase 6: Test the Script

✅ Verification & Testing

Test 1: Local Backup Exists

Test 2: Zip Integrity

Test 3: Remote Backup Exists

Test 4: File Sizes Match

Test 5: Multiple Runs (Idempotency)

Test 6: Contents Verification

🎨 Advanced Script Version (Production-Ready)

💡 Key Takeaways

🎓 Interview Questions to Master

🚨 Common Mistakes to Avoid

🚀 Taking It Further

Add to Cron for Automation

Add Email Notifications

Add Retention Policy

Add Timestamped Backups

Add Slack/Discord Notifications

Add Database Backup

Add Compression Level

Add Encryption

🔧 Troubleshooting Guide

Issue: Script Permission Denied

Issue: SSH Password Prompt

Issue: Zip Command Not Found

Issue: Directory Doesn't Exist

Issue: Permission Denied Writing Backup

Issue: Disk Space Full

📊 Monitoring & Maintenance

Check Backup Status

Verify Backup Integrity

Monitor Backup Timing

Disk Space Monitoring

🎯 Real-World Production Example

💡 Lessons Learned Today

🎯 What's Next?

📚 Additional Resources

Comments

More from this blog