As the design of our program is now stable, we can write the code which is an implementation of our solution.
Example 10.1. Backup Script - The First Version
#!/usr/bin/python # Filename : backup_version1.py import os import time # 1. The files and directories to be backed up are given in a list. source = ['/home/g2swaroop/all', '/home/g2swaroop/bin'] # If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] # 2. The backup must be stored in a main backup directory. target_dir = '/mnt/d/backup/' # 3. The files are backed up into a zip file. # 4. The name of the zip archive is today's date and time. target = target_dir + time.strftime('%Y%m%d%H%M%S') + '.zip' # 5. We use the zip command (in Unix/Linux) to put the files in a zip # archive zip_command = "zip -qr '%s' %s" % (target, ' '.join(source)) # Run the backup if os.system(zip_command) == 0: print 'Successful backup to', target else: print 'Backup FAILED'
$ python backup_version1.py Successful backup to /mnt/d/backup/20031124174143.zip
Now, we are in the testing phase where we test that our program works properly. If it doesn't behave as expected, then we have to debug our program i.e. remove the bugs (errors) from the program.
You will notice how we have converted our design into code in a step-by-step manner.
We first import the os and time modules to use some of the functionality of these modules. Then, we specify the files and directories to backup in the source list. The target directory is where we store all our backup files and this is specified by the target_dir variable. The name of the zip archive backup that we are going to create is the current date and time as returned by the time.strftime() function with the .zip extension and this archive is stored in the target_dir directory.
The time.strftime() function takes a specification like the one we have used in the above program. The %Y specification will be replaced by the year without the century. The %m specification will be replaced by the month as a decimal number between 01 and 12 for the current date, and so on. The complete list of such specifications can be found in the [Python Reference Manual] that comes with your Python distribution.
Then we create the name of the target zip file using the addition operator which concatenates the strings i.e. returns a string which combines those two strings. Then, we create a string zip_command which contains the command that we are going to execute. You can execute this command directly from the shell (Linux terminal or DOS prompt) to check if it works properly.
The zip command that we are using is like this - we use the option -q to indicate that the zip command should work quietly. The option -r indicates that the zip command should work recursively for directories i.e. it should include subdirectories and files within the subdirectories as well. The two options are combined to get -qr. The options are followed by the name of the zip archive to create, followed by the list of files and directories to backup. We convert the source list into a string using the join method of strings which we have already seen how to use.
Then, we finally run the command using the os.system function which runs the command as if it was run from the system i.e. the shell. It then returns 0 if the command was successfully. It will return an error number otherwise.
Depending on the outcome of the command we print an appropriate message and that's it, we have created a backup of our important files!
You can set the source list and target directory to any file and directory names in Windows, but you have to be a little careful. The problem is that Windows uses the backslash as the directory separator character but Python uses backslashes to represent escape sequences! So, you have to represent a backslash itself as an escape sequence or you have to use raw strings. For example, use 'C:\\Documents' or use r'C:\Documents', but do not use 'C:\Documents' - you are using an unknown escape sequence \D in this case!
Now that we have a working backup script, we can use it whenever we want to take the backup of files. Linux/Unix users are advised to use the executable method we discussed earlier so that they can run the backup script anytime anywhere. This is called the operation phase or the deployment phase of the software.
The above program works properly, but (usually) first programs may not work exactly as you expect. For example, there might be problems if you have not designed the program properly or if you have not written the code according to the design or you might have made a mistake in typing. Appropriately, you will have to go back to the design phase or you will have to debug your program.
The first version at our script is good, but we can make some refinements to it so that it can work better. This is called the maintenance phase of the software.
One of the refinements I felt was useful is a better file-naming mechanism - using the time as the name of the file within a directory with the current date as time within the main backup directory. One advantage is that your backups are stored in a hierarchical manner and therefore much easier to manage. Another advantage is that the length of the filenames are much shorter this way. Another advantage is that separate directories will help you to check that you have taken a backup for each day since the directory will be created only if you have taken a backup that day.
Example 10.2. Backup Script - The Second Version
#!/usr/bin/python # Filename : backup_version2.py import os import time # The files and directories to backup source = ['/home/g2swaroop/all', '/home/g2swaroop/bin'] # If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] # The directory where to store the backup target_dir = '/mnt/d/backup/' # The date - the subdirectory in the main backup directory today = target_dir + time.strftime('%Y%m%d') # The time - the name of the zip archive now = time.strftime('%H%M%S') # Create the subdirectory if it doesn't exist if not os.path.exists(today): os.mkdir(today) # make directory print 'Successfully created directory', today # The name of the zip file target = today + os.sep + now + '.zip' # The zip command to run zip_command = 'zip -qr %s %s' % (target, ' '.join(source)) # Run the backup if os.system(zip_command) == 0: print 'Successful backup to', target else: print 'Backup FAILED'
$ python backup_version2.py Successfully created directory /mnt/d/backup/20031124 Successful backup to /mnt/d/backup/20031124/174239.zip $ python backup_version2.py Successful backup to /mnt/d/backup/20031124/174241.zip
Most of the program remains the same. The addition is that we check if the directory with the current date as name exists inside the main backup directory using the os.exists function. If not, we create it using the os.mkdir function (which is short for make directory). Notice the use of the os.sep variable - this gives the directory separator according to your operating system i.e. it is '/' in Linux/Unix, it is '\\' in Windows and ':' in Mac OS. Using os.sep instead of these characters makes our programs portable.
The second version works fine, but when I do many backups, I am finding it hard to differentiate what the backups were for. For example, I might have made some major changes to a document, then I want to associate what those changes are with the name of the backup archive. This can be achieved by attaching a user-supplied comment to the name of the zip archive.
Example 10.3. Backup Script - The Third Version (does not work!)
#!/usr/bin/python # Filename : backup_version3.py import os, time # The files and directories to backup source = ['/home/g2swaroop/all', '/home/g2swaroop/bin'] # If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] # The directory where to store the backup target_dir = '/mnt/d/backup/' # The date - the subdirectory in the main backup directory today = target_dir + time.strftime('%Y%m%d') # The time - the name of the zip archive now = time.strftime('%H%M%S') # Take a comment from the user comment = raw_input('Enter a comment --> ') if len(comment) == 0: # Check if a comment was entered # The name of the zip file target = today + os.sep + now + '.zip' else: target = today + os.sep + now + '_' + comment.replace(' ', '_') + '.zip' # Create the subdirectory if it doesn't exist if not os.path.exists(today): os.mkdir(today) print 'Successfully created directory', today # The zip command to run zip_command = 'zip -qr %s %s' % (target, ' '.join(source)) # Run the backup if os.system(zip_command) == 0: print 'Successful backup to', target else: print 'FAILED to take the backup'
$ python backup_version3.py File "backup_version3.py", line 23 target = today + os.sep + now + '_' + ^ SyntaxError: invalid syntax
This program does not work! Python says there is a syntax error which means that the script does not satisfy the structure that Python it expects. When we observe the error given by Python, we see that it gives us the place where it detected the error as well. So we start debugging our program from that line.
On careful observation, we see that the single logical line has been split into two physical lines and we have not specified that these two physical lines belong together. Basically, Python has found the addition operator (+) without any operand in that logical line. We can specify that the logical line continues in the next physical line by the use of a backslash at the end of the physical line as we have already seen. So we make this correction to our program.
Example 10.4. Backup Script - The Fourth Version
#!/usr/bin/python # Filename : backup_version4.py import os, time # The files and directories to backup source = ['/home/g2swaroop/all', '/home/g2swaroop/bin'] # If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] # The directory where to store the backup target_dir = '/mnt/d/backup/' # The date - the subdirectory in the main backup directory today = target_dir + time.strftime('%Y%m%d') # The time - the name of the zip archive now = time.strftime('%H%M%S') # Take a comment from the user comment = raw_input('Enter a comment --> ') if len(comment) == 0: # Check if a comment was entered # The name of the zip file target = today + os.sep + now + '.zip' else: target = today + os.sep + now + '_' + \ # Notice backslash comment.replace(' ', '_') + '.zip' # Create the subdirectory if it doesn't exist if not os.path.exists(today): os.mkdir(today) print 'Successfully created directory', today # The zip command to run zip_command = 'zip -qr %s %s' % (target, ' '.join(source)) # Run the backup if os.system(zip_command) == 0: print 'Successful backup to', target else: print 'FAILED to take the backup'
$ python backup_version4.py Enter a comment --> fixed bug Successful backup to /mnt/d/backup/20031124/181157_fixed_bug.zip $ python backup_version4.py Enter a comment --> Successful backup to /mnt/d/backup/20031124/181202.zip
This program now works. Let us go through the actual enhancements that we had made in version 3. We take the user's comment using the raw_input function and then check if the user actually entered something or not. If the user has just pressed enter for some reason (maybe it was a routine backup and no special changes were made), then we proceed as before.
However, if a comment was supplied, then this is attached to the name of the zip archive just before the .zip extension. Notice that we replace spaces in the comment with underscores because managing such filenames are easier.
The fourth version must be a satisfactorily working script for most users, but there is always room for improvement. For example, you can include a verbosity level for the program where you can specify -v option to make your program more talkative, or you can backup additional files and directories specified on the command line using the sys.argv list.
One refinement I prefer is the use of the tar command instead of the zip command in Linux/Unix. One advantage is that when you use tar along with gzip, the backup is much faster and the archive size created is also much smaller. If I need to use this archive in Windows, then WinZip handles such .tar.gz files as well.
The command to use for utilising the tar is
tar = 'tar -cvzf %s %s -X /home/g2swaroop/bin/excludes.txt' % (dst, ' '.join(srcdir))
where the options are explained below.
-c indicates creation of an archive.
-v indicates verbose i.e. the command should be more talkative.
-z indicates that the gzip filter should be used.
-f indicates force in creation of archive i.e. over-writing.
-X indicates a file which contains a list of filenames which must be included from backup. For example, you can specify *~ in this file to not include any filenames ending with ~ in the backup.
An even better way of creating a backup script is to use the zipfile module included in the Python Standard Library. This avoids using os.system() which is generally not advisable to use.
For pedagogical purposes, I decided to use os.system() so that the example is simple enough to be understood by everybody but real enough to be useful.