AWS S3 Cheat Sheet

An AWS S3 (Simple Storage Service) cheat sheet can be handy for managing objects and buckets within the Amazon S3 storage service. Here’s a basic cheat sheet for AWS S3:

Basic Commands

Create a Bucket:

aws s3 mb s3://your-bucket-name

Remove a Bucket:

aws s3 rb s3://your-bucket-name

List Buckets:

aws s3 ls

List Objects in a Bucket:

aws s3 ls s3://your-bucket-name

Uploading and Downloading

Upload a File:

aws s3 cp your-file.txt s3://your-bucket-name/

Download a File:

aws s3 cp s3://your-bucket-name/your-file.txt .

Sync Local Directory to Bucket:

aws s3 sync your-local-directory s3://your-bucket-name/

Permissions

Make an Object Public:

aws s3 cp your-file.txt s3://your-bucket-name/ --acl public-read

Grant Bucket Read Access to Everyone:

aws s3api put-bucket-acl --bucket your-bucket-name --acl public-read

Metadata

Set Metadata for an Object:

aws s3 cp your-file.txt s3://your-bucket-name/ --metadata key=value

Lifecycle Policies

Create a Lifecycle Configuration:

  • Define a JSON file with the lifecycle policy and apply it to the bucket.

Versioning

Enable Versioning:

aws s3api put-bucket-versioning --bucket your-bucket-name --versioning-configuration Status=Enabled

Multipart Uploads

Initiate Multipart Upload:

aws s3api create-multipart-upload --bucket your-bucket-name --key your-large-file.txt

Upload Parts:

  • Use aws s3api upload-part to upload parts of a large file.

Complete Multipart Upload:

aws s3api complete-multipart-upload --bucket your-bucket-name --key your-large-file.txt --upload-id your-upload-id --multipart-upload '{"Parts": [{"ETag": "etag1", "PartNumber": 1}, {"ETag": "etag2", "PartNumber": 2}, ...]}'

Presigned URLs

Generate a Presigned URL for Upload:

aws s3 presign s3://your-bucket-name/your-file.txt --expires-in 3600

Generate a Presigned URL for Download:

aws s3 presign s3://your-bucket-name/your-file.txt --expires-in 3600

S3 Transfer Acceleration

Enable Transfer Acceleration for a Bucket:

aws s3api put-bucket-accelerate-configuration --bucket your-bucket-name --accelerate-configuration Status=Enabled

S3 Select

Run S3 Select Query:

aws s3api select-object-content --bucket your-bucket-name --key your-file.txt --expression "SELECT * FROM S3Object" --expression-type 'SQL' --input-serialization '{"CompressionType": "NONE", "CSV": {}, "JSON": {}}' --output-serialization '{"CSV": {}}' output.txt

This cheat sheet covers a variety of AWS S3 commands for common tasks. For more details and options, refer to the official AWS CLI S3 documentation.