blog-banner

My Experience With s3cmd Utility

  • Aws S3
  • BACKUP
  • CLOUD HOSTING

s3cmd Config

 

I came across this interesting utility to sync backup files to Amazon S3. With this utility in less than ten minutes, I learned how easy it is to upload a local file to a remote folder in an S3 bucket.

I already had a script in place to back up the site's files and database dump. To make this script complete I needed a script/ utility to sync backup files. And s3cmd came there as a savior. I would like to share in this blog post how easy it was to install, configure, and get started to make the potential use of the same.

Our servers are running Ubuntu 12.04.1 LTS. Luckily this s3cmd package is available in Ubuntu's official repository. In no time, I got this installed in our machine with apt-get command.

The following command did the trick of installing,

 $ sudo apt-get install s3cmd
This command would prompt the installation of a dependency package python-support. Once the installation is done it can be validated with the below command,
 $ s3cmd
The output returned might be as below,
1 ERROR: /home/user/.s3cfg: No such file or directory
2 ERROR: Configuration file not available.
3 ERROR: Consider using --configure parameter to create one.
Essentially it looks for the presence of the config file in the home folder .s3cfg. In our case, this is not in place, since this is the first time the utility is being used.
 
As suggested we need to create one by issuing the same command again with --configure argument.
1 $ s3cmd --configure
This will ask for a series of questions. Essentially Access Key, Secret Key, and other optional settings like a proxy, choice of protocol (http vs https), etc.
 
When all the needful inputs are given, the utility smartly prompts to make a test before saving it to the configuration file.
 1 New settings:
 2   Access Key: [your access key]
 3   Secret Key: [your securet key]
 4   Encryption password: somepassword
 5   Path to GPG program: /usr/bin/gpg
 6   Use HTTPS protocol: False
 7   HTTP Proxy server name: 
 8   HTTP Proxy server port: 0
 9 
10 Test access with supplied credentials? [Y/n] Y
11 Please wait...
12 Success. Your access key and secret key worked fine :-)
13 
14 Now verifying that encryption works...
15 Success. Encryption and decryption worked fine :-)
16 
17 Save settings? [y/N] y
I found this tiny feature very handy. I noticed that the Security key I had entered in the first attempt was wrong :-(
 
Finally, to achieve the intended goal, that is to sync the needed file to S3, I used the following command,
 $ s3cmd --delete-removed --reduced-redundancy sync my_backup.tar.gz s3://backup.example.com/project_backup/
The interactive output of the command is below,
1 my_backup.tar.gz -> s3://backup.example.com/project_backup//kst_backup/kst_.tar.gz  [1 of 1]
2  63935864 of 63935864   100% in  186s   335.43 kB/s  done
3 Done. Uploaded 63935864 bytes in 186.1 seconds, 335.42 kB/s