This post will walk you through the steps of copying an S3 bucket from one Amazon Web Services account to another. Throughout this post, the destination account refers to the account that will be receiving the bucket's contents, and the source account refers to the account that will be transmitting the bucket's contents.
First, you need to get the Account ID of the destination account. To do this, login to the destination account, click on the "Support" tab in the upper-right corner, and select "Support Center". The Account ID will be found in the upper-right corner of the page.
Next, you need to give the destination account access to the source account's bucket. To do this, login to the source account, select the S3 bucket that you want to copy over, view its properties, open the permissions tab, locate the "Add/Edit bucket policy" button, click on it, and paste the following policy into the pop-up:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DelegateS3Access",
"Effect": "Allow",
"Principal": {
"AWS": "<destination_id>"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<old_bucket_name>/*",
"arn:aws:s3:::<old_bucket_name>"
]
}
]
}
Replace destination_id with the Account ID acquired in the previous step and replace old_bucket_name with the bucket name that is being copied over. This policy grants bucket access to the destination account. Once this is done, save the policy.
Next, you need to create a policy for the destination account that grants access to both the receiving and transferring buckets. In the destination account, go to "Security Credentials", select "Policies", and create a new "Create Your Own" policy. Paste the following into this new policy:
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<old_bucket_name>",
"arn:aws:s3:::<old_bucket_name>/*",
"arn:aws:s3:::<new_bucket_name>",
"arn:aws:s3:::<new_bucket_name>/*"
]
}
}
Replace old_bucket_name and new_bucket_name in the above policy with their respective bucket names. Then, assign this policy to one of your AWS users.
After setting up these policies, you can use the AWS command line tool to copy the old bucket into the new bucket. The AWS command outlined below will do this, but before using this command, I want to highlight these parameters:
--dryrun - this parameter will show you what the command will do without actually performed the steps. It is recommended to run the below command with --dryrun specified first to ensure it does what you expect it to do.--source-region and --region - these parameters are needed if the source and destination buckets are in different regions# shows output of what will happen using --dryrun
# source-region and region are used since buckets are in different AWS regions
aws s3 sync s3://<old_bucket_name> s3://<new_bucket_name> --dryrun --profile <user_name> --source-region us-west-2 --region us-east-1
# remove --dryrun to actually run the command
aws s3 sync s3://<old_bucket_name> s3://<new_bucket_name> --profile <user_name> --source-region us-west-2 --region us-east-1
Replace user_name with a user profile that you have defined in your AWS credentials file (~/.aws/credentials) and properly set the regions if needed.
If you run the above command and notice that a few files failed to copy, run the command below to try copying over just one specific file.
aws s3 cp s3://<old_bucket_name>/<file_path> s3://<new_bucket_name>/<file_path> --profile <user_name> --source-region us-west-2 --region us-east-1
Once these commands are run, the old bucket will be successfully copied over to the new bucket!
After transferring the bucket, you should remove all the policies that you set up in the above steps, since they are no longer needed.