Monday, 30 July 2007

Backup your system using TAR

My system is a Debian Sid one (This article was first tested in an Ubuntu Feisty system)
Anyway this article is also useful in most other linux distros.

Tar tool allows us to make backups or even restore them, while our system is running.


It is a good idea becoming root to be able to access every file on the system, or at least you should use sudo command:

$sudo tar cvzpf /home/Backup.tgz --same-owner --exclude=/home/Backup.tgz --exclude=/home/error.log --exclude=/proc/* --exclude=/media/* --exclude=/dev/* --exclude=/mnt/* --exclude=/sys/* --exclude=/tmp/* / 2>/home/error.log


Tar options and arguments


c creates a tar backup
v activates verbose mode
z will compress data using gzip format
p preserves file permissions
f sends output to a file instead of standard output.

After f option it is necessary to write the file name where backup will be stored.
i.e: tar cvzpf /home/Backup.tgz is correct, but
tar cvzfp /home/Backup.tgz is wrong.

/home/Backup.tgz is our backup file name.

--same-owner preserve file ownership, it is not necessary because, being root, is a default option.



Excluding files from Backup


--exclude option gives us the possibility of excluding files and directories from backup.

--exclude=/home/Backup.tgz --exclude=/home/error.log Prevent Backup file and error log file to be stored.


--exclude=/proc/* --exclude=/dev/* --exclude=/sys/*

These are virtual directories so will not be stored.

NOTE: Difference between --exclude=/proc/* and --exclude=/proc is that first option would store a void proc directory, and second one would store nothing.

--exclude=/tmp/*

/tmp is a temporary directory, and is deleted on every boot, so will not be backed up either.


--exclude=/mnt/* --exclude=/media/*

If you want to save your main system only, and not what you have mounted on it, you should exclude these directories.


WHICH DIRECTORIES TO EXCLUDE?


To get some points about which directories do you need you could:

* Run mount or df commands, to know which filesystems and partitions are mounted in your system. e.g:
$ mount
$ df -h

* Read the Filesystem Hierarchy Standard (FHS) which tells you the meaning about most system and user directories.

* Read Linux System Directories and their Filesystems


NOTE: The easiest way to backup a system is booting from a live-cd or live-usb, then mount the partition you want to back up. This way you do not need to worry about which files to exclude, etc.



Logging errors


2>/home/error.log redirects standard error output to error.log file so we will be able to check if there have been important errors when creating backup file.

Normally some warnings appear about not storing some sockets and not including / at beginning of files. You can simply ignore them.



RESTORING YOUR SYSTEM


exec:
$sudo tar xvzf /home/Backup.tgz -C /

IF YOU RESTORE THE BACKUP IN A DIFFERENT PARTION OR MACHINE:
* Remember to change /etc/fstab file in your restored copy to correct new values.
* Also be sure, that the restored copy has all needed drivers for the new hardware.
* And in the end set correctly the bootloader configuration.



TESTING BACKUP FILE


To test if your backup file has no errors try this command:
$tar tvzf /home/Backup.tgz



BACKING UP VIA NETWORK


We are going to send our backup copy over the network. We will explain two methods:

* using netcat tool.
Bytes are sent without encryption.
No need of a previous server, we set it on the fly.

* using ssh tool.
It needs a ssh server already installed.
Provides encryption.
Simpler command setup.
It manages the connection.


We have two computers:

- Local computer with IP_local address. Contents of this computer will be backed up.
- Remote computer, which ip address is IP_remote. Backup file will be stored in this computer.



USING NETCAT TOOL

First at remote computer launch netcat in listening mode:
$nc -l -p 6000 >/home/Backup.tgz

That command starts listening at port 6000 and sends what receives to file /home/Backup.tgz.

Second at local computer:
$sudo tar cvzp --same-owner --exclude=/home/error.log --exclude=/proc/* --exclude=/media/* --exclude=/dev/* --exclude=/mnt/* --exclude=/sys/* --exclude=/tmp/* / 2>/home/error.log | nc -w 3 IP_remote 6000

NOTE: We have deleted f and --exclude=/home/Backup.tgz options because tar data is sent to standard output instead of to a file.

Backup starts, is sent via network and stored at remote computer.

NOTE: You need to start first netcat in listening mode or it will give you a connection error.



USING SSH TOOL

Depending on where is the ssh server installed we have two possibilities:

ssh server in remote computer: (We exec this command in the local computer)
$sudo tar cvzp --same-owner --exclude=/home/error.log --exclude=/proc/* --exclude=/media/* --exclude=/dev/* --exclude=/mnt/* --exclude=/sys/* --exclude=/tmp/* / 2>/home/error.log | ssh IP_remote "cat - > /home/Backup.tgz"

ssh server in local computer: (We exec this command in the remote computer)
$ssh IP_local "sudo tar cvzp --same-owner --exclude=/home/error.log --exclude=/proc/* --exclude=/media/* --exclude=/dev/* --exclude=/mnt/* --exclude=/sys/* --exclude=/tmp/* / 2>/home/error.log" > /home/Backup.tgz


More info at ubuntu forums:
BackupYourSystem using Tar
BackupYourSystem


YOU MAY ALSO BE INTERESTED IN:


GNU/Linux SYSTEM DIRECTORIES and their FILESYSTEMS

7 comentarios:

jon said...

Netcat is a really cool tool, but you could also just initiate the backup from the remote machine via ssh:

ssh backuptarget "tar cvpzf - --exclude=/proc --exclude=/sys --exclude=/var/run /" > backuptarget.tar.gz

Vicente Hernando said...

Hi Jon,

thank you a lot for the point! I will add it to the next article revision.

mojtaba roohi said...

hello
first thanks for good article then i want to ask question which fhs must backup

Vicente Hernando said...

Hello Mojtaba roohi,

FHS stands for Filesystem Hierarchy Standard. It explains the directories filesystem structure and the meaning of most directories in a unix based system.

Quran said...

Hi,

I found this article useful for setting my backups, I do not want to backup all my installation, I prefer to reinstall the system and then to restore my application, it is more fast and I guess more clean.

Thank you,
Quran

Anonymous said...

Perfect info, good to understand and help full tips. Now I have a good feeling before I install a new mod.
Thanks,
Ronald

Caiden Craig said...

Hello matee great blog post