Export EC2 Instance to VMware Virtual Machine Disk (VMDK) image

All the scripts in this article are bash scripts

Setup for macOS

Make sure you do this setup first:

Requirements

Follow the steps in the aws-ec2-terraform repository to deploy a new EC2 instance.

Set variables

instance_name="nginx-server-tf"

image_name="nginx-server-tf-image"
image_description="AMI created from nginx-server-tf EC2 instance"

s3_bucket="nginx-server-tf-images"
s3_prefix="exports/"
region="us-west-2"

role_name="vmimport"

Fix GRUB config

In order for the VMDK export to work, you need to set the GRUB_ENABLE_BLSC parameter to false in the /etc/default/grub file and rebuild the grub configuration file on the instance:

Connect to the instance via ssh:

key="aws-ec2-key"

instance_public_ip=$(aws ec2 describe-instances \
                --filters Name=tag:Name,Values=$instance_name \
            | jq -r '.Reservations[-1].Instances[-1].PublicIpAddress')

ssh -i ~/.ssh/$key ec2-user@$instance_public_ip

Run the following commands:

# set default kernel entry
sudo grub2-set-default 0
sudo grubby --set-default-index 0

# rebuild the grub config file to apply changes
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Create Amazon Machine Image AMI

The process is as follows:

Stop the instance
Create AMI image from the stopped instance
Start the instance

# Get instance ID
instance_id=$(aws ec2 describe-instances \
            --filters Name=tag:Name,Values=$instance_name \
            | jq -r '.Reservations[-1].Instances[-1].InstanceId')

echo "Stopping instance ..."
aws ec2 stop-instances --instance-ids $instance_id

echo "Waiting for instance to stop..."
aws ec2 wait instance-stopped --instance-ids $instance_id

echo "Creating AMI ..."
ami_id=$(aws ec2 create-image --instance-id $instance_id \
            --name "$image_name" \
            --description "$image_description" \
            --query 'ImageId' --output text)

echo "Waiting for AMI to be available ..."
aws ec2 wait image-available --image-ids $ami_id

echo "Starting instance ..."
aws ec2 start-instances --instance-ids $instance_id

echo "Waiting for instance to start ..."
aws ec2 wait instance-running --instance-ids $instance_id

echo "Created AMI ID: $ami_id"

Export AMI to VMDK

Create output S3 bucket

Create bucket:

echo "Creating S3 bucket ..."
aws s3api create-bucket --bucket $s3_bucket --region $region --create-bucket-configuration LocationConstraint=$region

Verify:

aws s3 ls

Create `vmimport` role

Create temporary directory:

mkdir -p ~/tmp/export-ec2-vmdk
cd ~/tmp/export-ec2-vmdk

Create role:

# Create trust policy
cat <<EOF > vmimport-trust-policy.json
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Principal": { "Service": "vmie.amazonaws.com" },
         "Action": "sts:AssumeRole",
         "Condition": {
            "StringEquals":{
               "sts:Externalid": "vmimport"
            }
         }
      }
   ]
}
EOF

# Create role
aws iam create-role --role-name $role_name --assume-role-policy-document file://vmimport-trust-policy.json

# Create role policy
cat <<EOF > vmimport-role-policy.json
{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:GetBucketLocation",
            "s3:GetObject",
            "s3:ListBucket", 
            "s3:PutObject",
            "s3:GetBucketAcl"
         ],
         "Resource":[
            "arn:aws:s3:::nginx-server-tf-images",
            "arn:aws:s3:::nginx-server-tf-images/*"
         ]
      },
      {
         "Effect":"Allow",
         "Action":[
            "ec2:ModifySnapshotAttribute",
            "ec2:CopySnapshot",
            "ec2:RegisterImage",
            "ec2:Describe*"
         ],
         "Resource":"*"
      }
   ]
}
EOF

# Create policy
policy_arn=$(aws iam create-policy --policy-name vmimport-policy --policy-document file://vmimport-role-policy.json --query 'Policy.Arn' --output text)

# Attach policy to role
aws iam attach-role-policy --role-name $role_name --policy-arn $policy_arn

Export the AMI to S3

# Export image to S3
echo "Exporting AMI to S3 ..."
export_task_id=$(aws ec2 export-image --image-id $ami_id --disk-image-format VMDK --s3-export-location S3Bucket=$s3_bucket,S3Prefix=$s3_prefix --role-name $role_name --query 'ExportImageTaskId' --output text)

Wait for the export task to complete

echo "Waiting for export task to complete ..."
while true; do
    status=$(aws ec2 describe-export-image-tasks \
        --export-image-task-ids $export_task_id \
        --query 'ExportImageTasks[0].Status' \
        --output text)

    echo "Current status: $status"
    if [ "$status" == "completed" ]; then
        echo "Export task completed successfully"
        break
    elif [ "$status" == "failed" ]; then
        echo "Export task failed"
        exit 1
    elif [ "$status" == "deleting" ] || [ "$status" == "deleted" ]; then
        echo "Export task was deleted"
        exit 1
    fi
    sleep 10  # Wait for 10 seconds before checking again
done

Cleanup

Follow the Cleanup steps in the aws-ec2-terraform repo to delete the EC2 instance.

Deregister AMI

# Get image ID
image_id=$(aws ec2 describe-images --owners self --filters "Name=name,Values=$image_name" --query 'Images[0].ImageId' --output text)
echo "Found image ID: $image_id"

# Get snapshots associated with the AMI
echo "Getting snapshots associated with the AMI ..."
snapshots=$(aws ec2 describe-images --image-ids $image_id --query 'Images[0].BlockDeviceMappings[*].Ebs.SnapshotId' --output text)

# Deregister AMI
echo "Deregistering AMI ..."
aws ec2 deregister-image --image-id $image_id

# Delete snapshots associated with the AMI
echo "Deleting snapshots associated with the AMI ..."
for snapshot in $snapshots; do
    aws ec2 delete-snapshot --snapshot-id $snapshot
    echo "Deleted snapshot $snapshot"
done

Delete `vmimport` role

# Delete vmimport role
echo "Deleting vmimport role ..."

# First, detach all policies
attached_policies=$(aws iam list-attached-role-policies --role-name $role_name --query 'AttachedPolicies[].PolicyArn' --output text)

for policy in $attached_policies; do
    aws iam detach-role-policy --role-name $role_name --policy-arn $policy
    echo "Detached policy $policy from role $role_name"
done

# Next, delete the policies
for policy in $attached_policies; do
    aws iam delete-policy --policy-arn $policy
    echo "Deleted policy $policy"
done

# Then, delete the role
aws iam delete-role --role-name $role_name
echo "Deleted role $role_name"

Delete S3 bucket

Optionally, if you want to delete the S3 bucket and the exported image, run these commands:

# Remove all objects from the S3 bucket
echo "Removing all objects from S3 bucket ..."
aws s3 rm s3://$s3_bucket --recursive

# Delete the S3 bucket
echo "Deleting S3 bucket ..."
aws s3 rb s3://$s3_bucket