Key points
- Bash scripts that back up each host and its MySQL databases, encrypt the archives with 7-Zip and upload them to Amazon S3.
- They date from 2014 and 2016 (CentOS 6): install the tools from your distribution and dump databases with
mysqldump --single-transaction. - I later moved these backups from Amazon to Google Cloud.
Automate Backups of Web Hosting Servers with Strong Encryption
Updated October 2026: These scripts that automate backups date from 2014 and 2016 and were written for CentOS 6, which reached its end of life on 30 November 2020. The idea still works, but I would change this today:
- Install the tools from your distribution (the old s3tools and RepoForge repositories are gone): on Debian 13 or Ubuntu,
apt install s3cmd 7zip; elsewherepip install s3cmdworks too. - Dump databases with
mysqldump --single-transaction(mariadb-dumpon MariaDB) instead of stopping MySQL, and keep the password in a root-only~/.my.cnf, not on the command line, where the process list shows it (the same goes for7za -p). - Amazon S3 has encrypted every new object by default (SSE-S3) since January 2023. AWS no longer recommends Reduced Redundancy Storage; use S3 Standard with a lifecycle rule to S3 Glacier Flexible Retrieval or Deep Archive. GnuPG now uses AES by default; do not pick CAST5. restic encrypts, deduplicates and uploads to S3 or Google Cloud Storage in one tool.
- Both scripts end with
python -c "print ...", which is Python 2 and fails on current systems. Use the line below instead, and start the second script with#!/bin/bash, because its[[ ]]tests fail where sh is dash (Debian and Ubuntu).
runtime=$(python3 -c "print(${end} - ${start})")
How I back up each host to the Amazon cloud
The aim of this article is to show how I successfully managed to automate the process of backing up a web hosting Linux server, host by host, including MySQL data. This was achieved by compressing and protecting the archive with a high level of encryption before transferring it to the Amazon cloud. To fully understand the script, you will need basic knowledge of Linux commands, as well as the installation of the s3cmd tool and 7zip. My experiment was conducted on Linux CentOS 6.5, and I had already obtained an Amazon S3 account, so make sure you get one as well.
A good question was raised on the Google+ discussion board (Google+ closed in 2019): Is it safe to upload your backups to the cloud? My answer is yes, as long as you know what you are doing. I took the risk after ensuring that Amazon provides server-side encryption. Furthermore, to enforce the TNO (Trust No One) concept, my script implements two levels of encryption before the upload phase. The first level uses 256-bit AES encryption via 7-Zip, and the second level uses GnuPG with the CAST5 algorithm. With these measures, I believe there is no way anyone can access the backup content unless they can guess my 128-character gibberish random passwords used in the three different encryption methods.
Installation
# First you will need to install the s3cmd tool:
# You will be asked to accept a new GPG key: answer yes (perhaps twice).
yum install epel-release
yum install s3cmd
# To configure s3cmd:
# Make sure you set https on with gpg enabled
s3cmd --configure
# Run s3cmd ls to list all your buckets:
s3cmd ls
# Make a bucket with s3cmd mb s3://my-new-bucket-name:
s3cmd mb s3://digi77backups
# List the contents of the bucket:
s3cmd ls s3://digi77backups
# Upload a file into the bucket (plain):
s3cmd put chatcleaner s3://digi77backups/chatcleaner
# Upload a file into the bucket (PGP encryption):
s3cmd -e put chatcleaner s3://digi77backups/chatcleaner
# Retrieve the file back and verify that it hasn't been corrupted:
s3cmd get s3://digi77backups/chatcleaner chatcleaner-2
md5sum chatcleaner chatcleaner-2
# Clean up: delete the object and remove the bucket:
s3cmd del s3://digi77backups/chatcleaner s3://digi77backups/chatcleaner2 s3://digi77backups/chatcleaner3
s3cmd rb s3://digi77backups
# For more commands see the s3cmd usage page: https://s3tools.org/usage
# Now that you have learned how to control your bucket via your server let us move to the next step.
# Install 7zip tool:
yum -y install epel-release
yum -y install p7zip
# To create an encrypted zip file with 7z archive:
7za a -tzip -pMY_SECRET -mem=AES256 secure.zip doc.pdf doc2.pdf doc3.pdf
# To extract a password protected file:
7za x thefile.zip -pThePasswordShellScriptThe Script
I will be explaining the script through the lines with the comment symbol #.
#!/bin/sh
# Shell script written by Warith AL Maawali
# (c) 2014 Warith AL Maawali
# https://www.digi77.com
# http://www.om77.net
# script starts here:
# Script timer variable
start=$(date +%s.%N)
# get the current time and date
fulldate="`date`"
# Set the backup date in Year - Month - Day format
backupdate="`date +%y%m%d`"
echo "Go to /home/backup and run rm -f * " $fulldate
cd /home/backup
# Remove any left over from previous backups
rm -f *.tar
rm -f *.zip
# Backup all 3 hosts carssite - busisite - personalsite : Those names are just examples
cd /home/backup
fulldate="`date`"
domanvalue="carssite"
echo "tar $domanvalue at: /home/$domanvalue " $fulldate
tar -zcf $domanvalue-$backupdate.tar /home/$domanvalue
echo "Protecting $domanvalue-$backupdate.tar with 7zip" $fulldate
# Compress from tar to zip with AES256 protection password used is : YOUR_ARCHIVE_PASSWORD
7za a -tzip -pYOUR_ARCHIVE_PASSWORD -mem=AES256 $domanvalue-$backupdate.zip $domanvalue-$backupdate.tar
rm -f $domanvalue-$backupdate.tar
echo "Sending $domanvalue-$backupdate.zip to the cloud" $fulldate
# Copy the file to the Amazon s3
s3cmd -e put $domanvalue-$backupdate.zip s3://mybackups/$domanvalue-$backupdate.zip
echo "Calculating md5sum hash" $fulldate
md5sum $domanvalue-$backupdate.zip
rm -f $domanvalue-$backupdate.zip
# Next site
fulldate="`date`"
domanvalue="busisite"
echo "tar $domanvalue at: /home/$domanvalue " $fulldate
tar -zcf $domanvalue-$backupdate.tar /home/$domanvalue
echo "Protecting $domanvalue-$backupdate.tar with 7zip" $fulldate
7za a -tzip -pYOUR_ARCHIVE_PASSWORD -mem=AES256 $domanvalue-$backupdate.zip $domanvalue-$backupdate.tar
rm -f $domanvalue-$backupdate.tar
echo "Sending $domanvalue-$backupdate.zip to the cloud" $fulldate
# Copy the file to the Amazon s3
s3cmd -e put $domanvalue-$backupdate.zip s3://mybackups/$domanvalue-$backupdate.zip
echo "Calculating md5sum hash" $fulldate
md5sum $domanvalue-$backupdate.zip
rm -f $domanvalue-$backupdate.zip
# Final site
fulldate="`date`"
domanvalue="personalsite"
echo "tar $domanvalue at: /home/$domanvalue " $fulldate
tar -zcf $domanvalue-$backupdate.tar /home/$domanvalue
echo "Protecting $domanvalue-$backupdate.tar with 7zip" $fulldate
7za a -tzip -pYOUR_ARCHIVE_PASSWORD -mem=AES256 $domanvalue-$backupdate.zip $domanvalue-$backupdate.tar
rm -f $domanvalue-$backupdate.tar
echo "Sending $domanvalue-$backupdate.zip to the cloud" $fulldate
# Copy the file to the Amazon s3
s3cmd -e put $domanvalue-$backupdate.zip s3://mybackups/$domanvalue-$backupdate.zip
echo "Calculating md5sum hash" $fulldate
md5sum $domanvalue-$backupdate.zip
rm -f $domanvalue-$backupdate.zip
#Backup Mysql
fulldate="`date`"
# Avoid Cpanel restarting the services
echo "Cpanel TailWatch switching off" $fulldate
/usr/local/cpanel/bin/tailwatchd --disable=Cpanel::TailWatch::ChkServd
fulldate="`date`"
echo "Mysql switching off" $fulldate
# Stop Mysql service
/etc/init.d/mysql stop
# Move to my backupfolder
fulldate="`date`"
cd /home/backup
# Compact Mysql folder to tar file
echo "tar mysql folder at: /var/lib/mysql " $fulldate
tar -zcf mysql-$backupdate.tar /var/lib/mysql
fulldate="`date`"
echo "start mysql" $fulldate
# Strat Mysql service
/etc/init.d/mysql start
fulldate="`date`"
echo "start cpanel TailWatch again" $fulldate
# Strat Cpanel watch again
/usr/local/cpanel/bin/tailwatchd --enable=Cpanel::TailWatch::ChkServd
# Preparing to Copy mysql to cloud
cd /home/backup
fulldate="`date`"
domanvalue="mysql"
echo "Protecting $domanvalue-$backupdate.tar with 7zip" $fulldate
7za a -tzip -pYOUR_ARCHIVE_PASSWORD -mem=AES256 $domanvalue-$backupdate.zip $domanvalue-$backupdate.tar
rm -f $domanvalue-$backupdate.tar
echo "Sending $domanvalue-$backupdate.zip to the cloud" $fulldate
# Copy the file to the Amazon s3
s3cmd -e put $domanvalue-$backupdate.zip s3://mybackups/$domanvalue-$backupdate.zip
echo "Calculating md5sum hash" $fulldate
md5sum $domanvalue-$backupdate.zip
rm -f $domanvalue-$backupdate.zip
echo "Backup completed successfully" $fulldate
# Calculate how long the script took to run
end=$(date +%s.%N)
runtime=$(python -c "print ${end} - ${start}")
echo "Runtime was $runtime"ShellScriptHere is another version of my backup script. No Amazon cloud is involved, but I have improved the MySQL backup and the MD5 calculation:
#!/bin/sh
# Shell script written by Warith AL Maawali
# (c) 2016 Warith AL Maawali
# https://www.digi77.com
# http://www.om77.net
# script starts here:
start=$(date +%s.%N)
fulldate="`date`"
bakdate="`date +%y%m%d`"
echo "Go to /home/backup and run rm -f * " $fulldate
# Remove any left over from previous backups
OUTPUT="/home/backup"
cd $OUTPUT
rm -f $OUTPUT/*.tar
rm -f $OUTPUT/*.zip
# Backup all hosts
# Digi77 backup
fulldate="`date`"
domanvalue="digi77"
echo "tar $domanvalue at: /home/$domanvalue " $fulldate
tar -zcf $domanvalue-$bakdate.tar /home/$domanvalue
echo "Protecting $domanvalue-$bakdate.tar with 7zip" $fulldate
7za a -tzip -pYourzippasswordgoeshere -mem=AES256 $domanvalue-$bakdate.zip $domanvalue-$bakdate.tar
rm -f $domanvalue-$bakdate.tar
echo "";
# Prepare mysql and rest
fulldate="`date`"
domanvalue="mysql"
echo "Backing mysql folder at: /home/mysql " $fulldate
# MySQL User
USER='root'
# MySQL Password
PASSWORD='YourDBPasswordGoesHere'
# Backup Directory - NO TAILING SLASH!
start=$(date +%s.%N)
fulldate="`date`"
bakdate="`date +%y%m%d`"
cd $OUTPUT
echo "Starting MySQL Backup";
echo `date`;
databases=`mysql --user=$USER --password=$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
for db in $databases; do
if [[ "$db" != "information_schema" ]] && [[ "$db" != _* ]] ; then
echo "Dumping database: $db"
mysqldump --force --opt --user=$USER --password=$PASSWORD --databases $db > $OUTPUT/$domanvalue-$bakdate-$db.sql
7za a -tzip -pYourzippasswordgoeshere -mem=AES256 $domanvalue-$bakdate-$db.zip $domanvalue-$bakdate-$db.sql
rm -f $OUTPUT/$domanvalue-$bakdate-$db.sql
fi
done
echo "Finished MySQL Backup";
echo "Getting MD5 Hash from the zip file list:" $fulldate
md5sum /home/backup/*.zip
fulldate="`date`"
echo "Backup completed successfully" $fulldate
end=$(date +%s.%N)
runtime=$(python -c "print ${end} - ${start}")
echo "Runtime was $runtime"ShellScriptNotes
# To test the script:
. backup.sh
# For my self I have set a cron job to run it every Sunday as the following:
15 5 * * 0 bash /home/xxx/week
# If the archive is too large then it might not fit into your system /tmp directory which gpg uses to store temporary files therefore you will need to use the following to avoid errors:
s3cmd put $domanvalue-$backupdate.zip s3://mybackups/$domanvalue-$backupdate.zip
# Instead of:
s3cmd -e put $domanvalue-$backupdate.zip s3://mybackups/$domanvalue-$backupdate.zipShellScriptFor cloud backup, I used Amazon S3 RRS and then moved the data to S3 Glacier, along with CloudBerry for client-side encryption, because I wanted more control over a reliable service. To learn more about cloud storage, please visit this post.
In the latest update, I have transitioned from Amazon services to Google Cloud, and I am very satisfied with the change. You can manage Google Cloud backups with the MSP360 Backup tool (formerly CloudBerry Backup).