Synching Your Amazon S3 Asset Host using Capistrano
published 2007-11-06
Note: This article is out of date. The latest version of this article is on the new permanent page in the projects section.
So you've got multiple asset hosts running in your Rails application, and you're using Amazon's S3 to host your assets. Now you want to make sure that your assets are kept up to date. This plugin is a Capistrano recipe that keeps the asset hosts synchronized with the public directory in your subversion repository.
Usage
After you get everything setup and do your first deploy, just run
cap deploy as normal and all changed files in RAILS_ROOT/public will
be uploaded to all of your asset host buckets before the final
deploy:symlink task.
The following tasks are also available:
- cap s3_asset_host:get_s3_revision
- cap s3_asset_host:find_changed
- cap s3_asset_host:list_changed
- cap s3_asset_host:find_all
- cap s3_asset_host:upload_changed
- cap s3_asset_host:upload_all
- cap s3_asset_host:upload
- cap s3_asset_host:reset_and_upload
- cap s3_asset_host:setup
- cap s3_asset_host:create_buckets
- cap s3_asset_host:delete_all
- cap s3_asset_host:connect
You can get documentation on these tasks by running cap -T
Requirements
This plug-in is a Capistrano extension. It requires Capistrano 2.0.0 or greater.
You will also require the aws-s3 gem
So far, this plug-in:
- assumes that you are using the 'checkout' method of deployment.
- only works with svn.
If you are using another version control system, I think all you'll have to change is the two methods in lib/scm.rb. If you do get something other than svn working, please let me know.
If you want to use more than one asset host, then you have to either
install the multiple asset hosts plugin
or upgrade to Rails 2.0 (see setting up multiple asset hosts in
Rails)
Setup
To set-up, you need to do the following
- Install the plug-in
- Install the AWS-S3 gem.
- Set up your Rails application to use asset hosts.
- Set up your asset hosts.
- Configure Capistrano.
Installing the plug-in
From RAILS_ROOT, run:
script/plugin install svn://svn.spattendesign.com/svn/plugins/synch_s3_asset_host
Installing the AWS-S3 gem
You need to do this on both your local computer and the computer that is defined as the asset_host_syncher (see Capistrano Configuration, below).
$> sudo gem install aws-s3
Setting up your Rails app to use asset hosts
Single asset host
For a single asset host, simply add the following line to
RAILS_ROOT/config/environments/production.rb:
config.action_controller.asset_host = "http://assets.example.com"
Multiple asset hosts
Follow the instructions in setting up multiple asset hosts in Rails
Setting up your asset hosts
Set up a CNAME entry for each asset host pointing to s3.amazonaws.com.
How you do this depends on your domain host. Here's what it looks like
on easydns

You may need to wait up to 24 hours for the DNS entries for these new hosts to propagate.
Configuring Capistrano
Capistrano installation
This plugin requires Capistrano 2.0.0 or greater.
To upgrade to the latest version (currently 2.1.0):
$> gem install capistrano
Once the plug-in is installed, make sure that the recipes are seen by Capistrano
$> cap -T | grep s3_asset_host
should return a bunch of tasks. If you don't see anything listed, then
you need to update your Capfile by doing the following (this is from
Jamis
Buck):
In Capistrano 2.1.0 or above:
$> cd RAILS_ROOT $> rm Capfile $> capify .
If you do not want to delete your Capify file, or if you are using
Capistrano 2.0.0, add the following line to your Capify
file:<br/>{=html}
Dir['vendor/plugins/*/recipes/*.rb'].each { |plugin| load(plugin) }
Capistrano configuration
Create a new file in RAILS_ROOT/config called
synch_s3_asset_host.rb. Add the following lines to it, and edit to
suit:
# ============================================================================= # S3 ASSET HOST OPTIONS # ============================================================================= set :asset_host_name, "assets%d.example.com" set :aws_access_key, "your Amazon AWS access key" # You can also set this in your environment as AMAZON_ACCESS_KEY_ID set :amazon_secret_access_key, "your Amazon AWS secret" # You can also set this in your environment as AMAZON_SECRET_ACCESS_KEY # set :dry_run, false# Set to true if you want to test the asset_host uploading without doing anything on Amazon S3 before "deploy:symlink", "s3_asset_host:upload_changed"
You have to do one more thing: in RAILS_ROOT/config/deploy.rb. Specify
one of your web hosts as an "asset_host_syncher", like this:
role :web, webserver1, :asset_host_syncher => true
The first deploy
Commit all changes to your rails application and do the initial bucket setup:
$> cap s3_asset_host:setup $> svn commit -m "Adding synch_s3_asset_host plugin" $> cap deploy
This will do the following:
- Create your Amazon S3 AWS buckets
- upload everything in RAILS_ROOT/public (in your svn repository) to each bucket
- Set the revision in each bucket to the latest revision in your repository.
This could take a while if you have lots of images or other big files.
You're done!
That should do it. Now, every time you run cap deploy, your asset
hosts should be updated with any changes to files in
RAILS_ROOT/public.
Let me know if you have any problems, suggestions or comments.