Erik's Engineering

something alliterative

Using Backup With Rails

Have you noticed that Rubygems.org has become a really nice cross-platform application distribution system?  We think of Rails as a dev framework, but it ships with a command line app for bootstrapping a project.  Capistrano is all about the command line app.  Lunchy is new and cool.

Backup is the same way.  It's a tool for backing up databases and directories, that happens to be written in ruby and distributed via Rubygems.org.  It recently hit 3.0 and I figured it was a good choice for backing up my personal stuff.

First off, I was delighted to find the excellent documentation on how to set it up.  The Getting Started doc was great for my photoblog (a PHP app), but I have a bunch of personal Rails apps and I wanted to take things a step further for them.

DRY up the config

The stock instructions have you duplicating your database credentials.  You use the handy app to generate a config file with the options you want. Plug in some credentials and backups just work.  You can even point it at S3 and save having to come up with a second server to send the backups to. This is really awesome, but the notion of having credentials in both my database.yml and another backup config file bothered me.

Further, because all backup jobs (or "triggers" in backup-speak) share the same DB credentials, you either have to have the same db username and password for backups on all your databases or you need a separate backup config file for each app.

So I'm doing something different.  I've got a config/backup.rb in each Rails app.  It looks something like this:

#
# Backup
# Generated Template
#
# For more information:
#
# View the Git repository at https://github.com/meskyanichi/backup
# View the Wiki/Documentation at https://github.com/meskyanichi/backup/wiki
# View the issue log at https://github.com/meskyanichi/backup/issues
#
# When you're finished configuring this configuration file,
# you can run it from the command line by issuing the following command:
#
# $ backup -t my_backup [-c <path_to_configuration_file>]

database_yml = File.expand_path('../config/database.yml',  __FILE__)
RAILS_ENV    = ENV['RAILS_ENV'] || 'development'

require 'yaml'
config = YAML.load_file(database_yml)


Backup::Model.new(:db_backup, 'Database Backup to S3') do

  database PostgreSQL do |db|
    db.name               = config[RAILS_ENV]["database"]
    db.username           = config[RAILS_ENV]["username"]
    db.password           = config[RAILS_ENV]["password"]
    db.host               = config[RAILS_ENV]["host"]
    db.port               = config[RAILS_ENV]["port"]
    db.skip_tables        = []
  end

  store_with S3 do |s3|
    s3.access_key_id      = '<SECRET>'
    s3.secret_access_key  = '<SECRET>'
    s3.region             = 'us-east-1'
    s3.bucket             = '<BUCKET NAME>'
    s3.path               = config[RAILS_ENV]["database"]
    s3.keep               = 10
  end

  compress_with Gzip do |compression|
    compression.best = true
    compression.fast = false
  end

end

Because there are so many options when building a backup config file (database, storage and sync engines, not to mention encryption, compression and notification) you'll want to make your own using the generator.

Since the backup config file is just a ruby script with a bit of a DSL, you can hook your database.yml and avoid having to repeat your credentials. I'm tempted to add an S3.yml and do the same with that. Notice that I can even automate what path to put the backup in - either naming it after the environment or (in this case) the database name.

Don't Make Me Remember The Command

When I think of backing up my rails app, I don't think of a command line with a bunch of options. I think of something like 'rake db:backup'.

I've created the backup-task gem to take care of that under Rails 3. It's really simple - it just adds the following task:

namespace :db do
  desc "Back up the database"
  task :backup do
    sh "backup perform --trigger db_backup --config_file config/backup.rb --data-path db --log-path log --tmp-path tmp"
  end
end

This performs a backup of the 'db_backup' trigger, using the config/backup.rb file. It manually sets the data-path, log-path and tmp-path settings so that they'll all end up being relative to this Rails app. By default, they end up in standard locations that are either shared by all users on the system or all backups done by this user. Doing it this way avoids problems caused by having all my backups use the same trigger name.

Schedule It

The final step (after manually running it once or twice to make sure my config trickery works) is to set this up to run without having to think about it. A line like the following in your crontab will take care of that.

1 1 * * * bash -l -c "rvm use ruby-1.9.2-p0 && cd ~/sites/waiter && RAILS_ENV=production rake db:backup"

That was not especially obvious to me. I mean, the cron scheduling is easy, but there was some experimenting and googling to get the command line working with rvm. Explicitly calling bash and passing it the -l -c options did the trick.

TODO

This all rocks. It was really easy to set up, but there's still something missing. As things stand, it's entirely possible that my backups will crap out and I won't know. Maybe the notification options would handle that, but I'd kind of prefer something that watched my S3 bucket and complained at me if it didn't see a reasonably sized backup dated to some time in the last 24 hours. That'll have to be a future project.

Published on 26/03/2011 at 21h15 .

  • By Jason Weathered 27/03/2011 at 21h58

    For scheduling the backup with cron, look into using whenever with Capistrano integration instead of editing the crontab directly.

    An equivalent schedule.rb would look something like:

    every 1.day, :at => '01:01' do
      rake 'db:backup'
    end
    

  • By Steve Schwartz 27/03/2011 at 22h42

    This is exactly what I had done with setting up backup with Rails as well. One additiona is that I also have a yml file for my s3 buckets, using different buckets for different environments, and use the same method for using the proper bucket credentials.

    Also, add this to the end of your backup task in crontab to log the results. When things go wrong, you’ll see the error message from the backup task in the log:

    >> /path/to/backup_cron.log 2>&1
    

  • By Kevin admin 29/03/2011 at 09h24

    You can also use a program to make data backup like Handy Backup (http://www.handybackup.net)


Comment Using Backup With Rails

Trackbacks are disabled

Powered by Typo – Thème Frédéric de Villamil | Photo Glenn