Automated backup to a Dropbox folder
In parts 1 and 2 I documented how I setup a Raspberry Pi to provide Internet-based access to a Subversion server. The task here is to write a Python script that creates a hot copy of a repository, zips it, and sends to an appropriate place for archiving.
Note: after writing my own scripts I found some example scripts including hot-backup.py. This has more goodies than mine, including cleaning up excess (old) backups.
Python scripting notes: to allow a Python script to be ran directly from the command line set the first line to:
and then make it executable with:
jon@raspberrypi ~ $ sudo chmod u+x your_filename.py
#! text on the first line is called the shebang and tells the shell what type of interpreter to use for the rest of the script. We’ll need to make system calls from the Python script – this site has lots of cool examples.
Dropbox via Python on the RPi
Note: using the Dropbox API requires a Dropbox account.
An application key is required to use the Python API. Go to the Dropbox app console, agree with Terms and Conditions, and select the following options:
jon@raspberrypi ~ $ wget https://www.dropbox.com/static/developers/dropbox-python-sdk-1.6.zip --2013-08-28 14:09:38-- https://www.dropbox.com/static/developers/dropbox-python-sdk-1.6.zip Resolving www.dropbox.com (www.dropbox.com)... 18.104.22.168 Connecting to www.dropbox.com (www.dropbox.com)|22.214.171.124|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 812016 (793K) [application/zip] Saving to: `dropbox-python-sdk-1.6.zip' 100%[=======================================================================================>] 812,016 118K/s in 8.3s 2013-08-28 14:09:53 (96.1 KB/s) - `dropbox-python-sdk-1.6.zip' saved [812016/812016]
Unzip the archive and cleanup:
jon@raspberrypi ~ $ unzip dropbox-python-sdk-1.6.zip jon@raspberrypi ~ $ rm dropbox-python-sdk-1.6.zip
Install the Python SetupTools package, then the Dropbox Python SDK :
jon@raspberrypi ~ $ sudo apt-get install python-setuptools jon@raspberrypi ~ $ cd dropbox-python-sdk-1.6/ jon@raspberrypi ~/dropbox-python-sdk-1.6 $ sudo python setup.py install
The Dropbox Core API page is useful in developing upload scripts. For this backup we need two scripts:
- A temporary script that requires operator interaction to generate a Dropbox access code. This access code will be used by the main script for authentication and is the key to writing non-interactive Dropbox scripts.
- A backup script that provides non-interactive repository archiving (via a hotcopy), compression, Dropbox upload, and caretaking.
First the access token: create a new script,
db_get_access_token.py (and insert the actual app_key and app_secret strings). (For my scripts I created a new folder,
svn_pi, underneath the
#!/usr/bin/env python import dropbox
# Get your app key and secret from the Dropbox developer website app_key = '*** insert ***' app_secret = '*** insert ***'
flow = dropbox.client.DropboxOAuth2FlowNoRedirect(app_key, app_secret) authorize_url = flow.start()
print '1. Go to: ' + authorize_url print '2. Click "Allow" (you might have to log in first)' print '3. Copy the authorization code.' code = raw_input("Enter the authorization code here: ").strip()
access_token, user_id = flow.finish(code) client = dropbox.client.DropboxClient(access_token)
print 'linked account: ', client.account_info() print 'Access token [' print access_token print ']'
Run the script, copy the URL from the terminal window into a browser, then allow the new app to have access:
Copy the code and paste into the terminal session prompt: ‘
linked account‘. If everything works as expected the ‘
access token‘ information will be displayed. Otherwise something similar to
dropbox.rest.ErrorResponse:  u'invalid_grant' will be displayed. Copy the access token string – this is needed to allow dropbox file transfers without operator interaction.
Then create the main backup script,
svn_hotcopy_db.py. This will process all the repositories, for each one creating a hot copy, compressing it to a zip file, coping the zip to Dropbox, then finally cleaning up. Replace any
*** string sections in the script with the appropriate information before running!
#!/usr/bin/env python import datetime import subprocess import os import dropbox import shutil homePath = '/home/***/' def makeDateTimeStamp(): now = datetime.datetime.today() nowAsString = now.strftime('%Y_%m_%d_%H_%M_%S') return nowAsString def backupRepository(repoName, db_client): # Step 1: make hotcopy pathToSourceRepository = homePath + 'svnrepos/' + repoName + '/' pathToHotCopyRepository = homePath + 'svn_py/hotcopy/' cmdMakeHotCopy = 'svnadmin hotcopy ' + pathToSourceRepository + ' ' + pathToHotCopyRepository print cmdMakeHotCopy subprocess.call(cmdMakeHotCopy, shell=True) # Step 2: zip the hotcopy zipFileNameWithoutPath = repoName + '_' + makeDateTimeStamp() + '.zip' zipFileName = homePath + 'svn_py/' + zipFileNameWithoutPath cmdMakeZip = 'zip -r ' + zipFileName + ' ' + pathToHotCopyRepository print cmdMakeZip subprocess.call(cmdMakeZip, shell=True) # Step 3: send zip file to Dropbox print 'Sending zip file to Dropbox' with open(zipFileName, 'r') as zipFile: db_client.put_file(zipFileNameWithoutPath, zipFile) # Step 4: cleanup print 'Cleaning up' os.remove(zipFileName) shutil.rmtree(pathToHotCopyRepository) # Configure dropbox access db_access_token = '***' db_client = dropbox.client.DropboxClient(db_access_token) # Setup list of repositories to dump/zip/backup repositories = ['***', '***', '***'] # Run ! for repository in repositories: backupRepository(repository, db_client)
The backup can be scheduled to run automatically. First edit the schedule for the current user:
jon@raspberrypi ~ $ sudo crontab -e
Edit the file and include the following line:
0 2 * * * /home/jon/svn_py/svn_hotcopy_db.py &
0 2 * * *: Run at 02:00 every day.
/home/jon/svn_py/svn_hotcopy_db.py: script to run
&: run in background