Shantanu's Blog

S3cmd is a tool for managing objects in Amazon S3 storage. It allows for making and removing "buckets" and uploading, downloading and removing "objects" from these buckets.

Install s3cmd package on Ubuntu or CentOS using the following commands respectively.

# sudo apt-get install s3cmd
# yum install s3cmd

Or download the file and isntall

wget http://tinyurl.com/6lmbsa8
unzip s3cmd-1.1.0-beta3.zip
cd s3cmd-1.1.0-beta3
python setup.py install

# s3cmd --configure
Keep the access and secret key handy. Config file defaults to /root/.s3cfg

Important Commands:
List objects or buckets
s3cmd ls [s3://BUCKET[/PREFIX]]
Get file from bucket
s3cmd get s3://BUCKET/OBJECT LOCAL_FILE
Put file into bucket
s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]
Delete file from bucket
s3cmd del s3://BUCKET/OBJECT
Synchronize a directory tree to S3
s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR
Copy object
s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
Move object
s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]

options:
-f, --force Force overwrite and other dangerous operations.
--continue Continue getting a partially downloaded file
-r, --recursive Recursive upload, download or removal.
-P, --acl-public Store objects with ACL allowing read for anyone.
-H, --human-readable-sizes Print sizes in human readable form (eg 1kB instead of 1234).

Other useful Commands:
Make bucket
s3cmd mb s3://BUCKET
Remove bucket
s3cmd rb s3://BUCKET
List all object in all buckets
s3cmd la
Get various information about Buckets or Files
s3cmd info s3://BUCKET[/OBJECT]
Disk usage by buckets
s3cmd du [s3://BUCKET[/PREFIX]]

See program homepage for more information at
http://s3tools.org


It is possible to share buckets between existing AWS users.

Using the console: In the Grantee field add the users email address or canonical userid
Using s3cmd: s3cmd setacl --grant-acl=read: s3://BUCKETNAME[/OBJECT]

 // recursively copy entire folder to s3
time /root/s3cmd-1.1.0-beta3/s3cmd put --recursive /path/folder s3://redis_archive/