We have a folder /backups on your server that you want to syncronize to an AWS S3 Storage bucket. First thing is first, we'll want to install the aws command line tools. I'll use python virtual environments specific to functions or django webapps, etc. So first, create a virtualenv and source it (if it's successfully created using &&).
virtualenv /opt/python/envs/awsenv && source /opt/python/envs/awsenv/bin/activate
We will use pip to install the amazon cli tools. These are useful for more than s3 storage, but this is an example to just synchronize the folder on your server to AWS for backup purposes.
pip install awscli
Before creating this file, we need to ensure we've generated an Access Key ID/Secret Access Key pair. Logged into the AWS Web GUI, click your login name in the top right and select "My Security Credentials". From here, you can generate these, and copy and paste them into a config file located in ~/.aws/config. We also don't want this file to be world readable as anyone with access to it can access your AWS account which is an obvious security issue.
mkdir ~/.aws && vi ~/.aws/config
The contents of this should at minimal look something like this. AWS has pretty good documentation on other things that can go in here, so I won't get into all that right now.
Now, set the permissions on this folder/file, I use 700/600 and also mute it.
chmod 700 ~/.aws && chmod 600 ~/.aws/config && chattr +i ~/.aws && chattr +i ~/.aws/config
Lets now list the current contents of the bucket which you could have created from the webgui.
aws s3 ls s3://mybucketname
And finally, we can sync our data to the buket. In this case, we will exclude the .git/* folder as it can at least double the amount of storage used if you're using git to manage changes and the integrity of binary files. (I've had my data tampered with/stolen by "trusted" parties in the past)...
aws s3 sync /path/to/sync/ s3://mybucketname --exclude ".git/*"Share on Twitter Share on Facebook