AWS S3 CLI Commands

LIST:

We use ls commands to show the lists of buckets and content of buckets.

# List of buckets

aws s3 ls

# Contents of bucket

aws s3 ls s3://s3-sea-nkcode-bucket

# ls command will recursively list objects in a bucket and files inside the subfolders

aws s3 ls s3://s3-sea-nkcode-bucket/ --recursive

2021–1–14 10:22:30 5000 DBLog.txt
2021–1–14 10:39:39 2259 nkcode.txt

# ls command can also be human-readable and summarized

aws s3 ls s3://s3-sea-nkcode-bucket/ --recursive --human-readable --summarize

2021–1–14 10:22:30 5000 DBLog.txt
2021–1–14 10:39:39 2259 nkcode.txt

COPY:

The cp command is used to copy files from either local to a bucket or one bucket to another bucket.

# Copying a file from S3 to S3

aws s3 cp s3://s3-sea-nkcode-bucket/DBLog.txt s3://s3-sea-nkcode-bk-bucket

# If you want to give a new name in the destination bucket

aws s3 cp s3://s3-sea-nkcode-bucket/vmware.txt s3://s3-sea-nkcode-bk-bucket/new_vmware.txt

# Copying an S3 object to local

aws s3 cp s3://gaurav-test-today/DBLog.txt MyFile.txt

# Copying a local file to S3

aws s3 cp nkcodeaws.txt s3://s3-sea-nkcode-bucket

# Copying a local file to S3 with an expiration date

aws s3 cp test.txt s3://s3-sea-nkcode-bk-bucket/nkcodeawscode.txt --expires 2021–10–01T20:30:00Z

# Recursively copying S3 objects (all contents of bucket) to a local directory

aws s3 cp s3://s3-sea-nkcode-bucket . -- recursive

# Recursively copying S3 objects to another bucket

aws s3 cp s3://s3-sea-nkcode-bucket/ s3://s3-sea-nkcode-bk-bucket/ --recursive

CREATE:

Create a bucket.

# Create a public bucket

aws s3 mb s3://s3-sea-nkcode-bucket/

# create a bucket in a region

aws s3 mb s3://s3-sea-nkcode-bucket --region ap-southeast-1

MOVE:

mv is used to move files. Let’s see the difference between cp and mv.

# Difference between mv and cp

aws cp will copy a local file or S3 object to another location locally or in S3.

aws mv will move a local file or S3 object to another location locally or in S3. i.e. it will delete it from the source and put it on the target path.

# Moves a single file to a specified bucket

aws s3 mv s3://s3-sea-nkcode-bucket/nkcode1.txt s3://s3-sea-nkcode-bk-bucket

PRE-SIGN:

Generate a pre-signed URL for an Amazon S3 object. This allows anyone who receives the pre-signed URL to retrieve the S3 object with an HTTP GET request.

# To create a pre-signed URL with the default one hour lifetime that links to an object in an S3 bucket

The following presign the command generates a pre-signed URL for a specified bucket and that key is valid for one hour.

aws s3 presign s3://s3-sea-nkcode-bucket/nkcode1.txt

# To create a pre-signed URL with a custom lifetime that links to an object in an S3 bucket

aws s3 presign s3://s3-sea-nkcode-aws-bucket/nkcode1.txt --expires-in 604800

RB:

Deletes an empty S3 bucket.

# Remove an empty bucket

aws s3 rb s3://s3-sea-nkcode-aws-bucket

# Remove a non-empty bucket

aws s3 rb s3://s3-sea-nkcode-aws-bucket --force

DELETE:

Delete the contents of a bucket.

# Delete a file

aws s3 rm s3://s3-sea-nkcode-aws-bucket/vmware.txt

# Recursively deletes all objects

aws s3 rm s3://s3-sea-nkcode-aws-bucket--recursive

# Delete everything except but with excluded files

aws s3 rm s3://s3-sea-nkcode-bucket --recursive --exclude “*.jpg”

SYNC:

Syncs directories and S3 prefixes. Recursively copies new and updated files from the source directory to the destination.

Difference between AWS cp vs AWS sync

Using aws s3 cp will require the --recursive the parameter to copy multiple files.

The aws s3 sync the command will, by default, copy a whole directory. It will only copy new/modified files.

The sync command syncs objects under a specified prefix and bucket to files in a local directory by uploading the local files to S3. A local file will be uploaded if the size of the local file is different than the size of the S3 object, the last modified time of the local file is newer than the last modified time of the S3 object, or the local file does not exist under the specified bucket and prefix.

# Sync from local to a bucket

aws s3 sync . s3://s3-sea-nkcode-bucket

Website

Set the website configuration for a bucket.

# Configures a bucket named “my-bucket” as a static website

aws s3 website s3://s3-sea-nkcode-web-bucket/ --index-document index.html --error-document error.html