Deploy Docker Swarm on AWS EC2 via cloud-formation templates - Step 8 - Cleanup

In this step we cleanup the resources that we creater for the Docker Swarm cluster on EC2.

This post is part of a thread that includes these steps:

1. Network Setup
2. Storage
3. Roles
4. Manager Instance
5. Worker Launch Template
6. Worker Instances
7. Docker Swarm
8. Cleanup (this post)

Cleanup

Docker Swarm

Login to manager machine

./ssh/ssh-manager.sh

Switch user to the worker user:

sudo su - worker

Remove all nodes from Swarm

pssh --hosts=$HOME/nodes/hosts --inline 'docker swarm leave'

Destroy Swarm

On manager machine:

docker swarm leave --force 

Clean up dangling networks

After destroying the Swarm make sure that you do not have dangling networks ingress and docker_gwbridge:

docker network ls

If you see ingress or docker_gwbridge, force remove them:

docker network disconnect --force docker_gwbridge
docker network rm docker_gwbridge

docker network disconnect --force ingress
docker network rm ingress

Also you have to do the same for all nodes:

pssh --hosts=$HOME/nodes/hosts --inline 'docker network rm docker_gwbridge'
pssh --hosts=$HOME/nodes/hosts --inline 'docker network rm ingress'

Remove swift_default network:

docker network rm swift_default

Docker Cleanup

These commands will remove all stopped containers and all images that do not have a running container on all nodes:

# stop and femove containers
pssh --timeout 300 --hosts=$HOME/nodes/hosts --inline \
  'docker container prune --force'

# delete unused images
pssh --timeout 300 --hosts=$HOME/nodes/hosts --inline \
  'docker image prune --all --force'

# check root file system
pssh --timeout 300 --hosts=$HOME/nodes/hosts --inline \
  'df --human-readable /ebs/docker'  

Route 53 Cleanup Script

Start in the project directory:

cd ~/swift-aws-ec2-swarm

Create a folder route53 and a route53-delete-record.sh file in it.

mkdir -p route53
touch route53/route53-delete-record.sh
nano route53/route53-delete-record.sh

Copy and paste this code into route53-delete-record.sh:

#!/usr/bin/env bash

# =============================================================================================================
# Usage:
#   ./route53-delete-record.sh [HostedZoneName] [Hostname] [Type]
#
# Example:
#   ./route53-delete-record.sh example.org dummy.example.org
#   ./route53-delete-record.sh example.org dummy.example.org TXT
#   ./route53-delete-record.sh example.org dummy.example.org txt
#   ./route53-delete-record.sh example.org dummy.example.org CNAME
# =============================================================================================================

# output coloring
RED=$(tput setaf 1)
GREEN=$(tput setaf 2)
YELLOW=$(tput setaf 3)
CLEAR=$(tput sgr0)

# put your value here
# note that jq can work with env var
HOSTED_ZONE=${1:-example.org}
DNS_NAME=${2:-test.example.org}
DNS_TYPE=${3:-A}

[[ -z "$HOSTED_ZONE" ]] && HOSTED_ZONE=example.org

# add . to the end
DNS_NAME="$DNS_NAME."

# capitalize
DNS_TYPE=${DNS_TYPE^^}

echo Deleting record: \'$DNS_TYPE\' \'$DNS_NAME\' from hosted zone \'$HOSTED_ZONE\' ... 

# find Zone ID
ZONE_ID=$(aws route53 list-hosted-zones-by-name --dns-name $HOSTED_ZONE --output json \
  | jq .HostedZones[].Id --raw-output \
  | awk -F / '{print $3}')

if [[ -z "$ZONE_ID" ]]; then
  echo ${RED}$HOSTED_ZONE hosted zone not found!$CLEAR
  exit 1
fi

echo Zone ID: $YELLOW$ZONE_ID$CLEAR
echo

# find resource record set
RECORD_SETS=$(aws route53 list-resource-record-sets --hosted-zone-id=$ZONE_ID --output json \
  | jq '.ResourceRecordSets[] | select ((.Name == '\"$DNS_NAME\"') and (.Type=='\"$DNS_TYPE\"'))')

if [[ -z "$RECORD_SETS" ]]; then
  echo ${RED}$DNS_NAME $DNS_TYPE record not found!$CLEAR
  exit 1
fi

echo Resource Record Sets:
jq <<< "$RECORD_SETS"
echo

# prepare the change batch value
CHANGE_BATCH=$(cat << EOF
{
    "Comment": "delete this record",
    "Changes": [
        {
            "Action": "DELETE",
            "ResourceRecordSet":
              $RECORD_SETS

        }
    ]
}
EOF
)

echo Change batch:
jq <<< "$CHANGE_BATCH"
echo

# perform the deletion
aws route53 change-resource-record-sets --hosted-zone-id=$ZONE_ID --change-batch "$CHANGE_BATCH"

Make the script executable:

chmod +x route53/route53-delete-record.sh

Delete AWS resources

Instances

Worker

./ec2-worker/rm-ec2-worker.sh
./ec2-worker-lt/rm-ec2-worker-lt.sh

Manager

./ec2-manager/rm-ec2-manager.sh

IAM Roles / Instance Profiles

Worker

./iam/rm-iam-worker.sh

Manager

./iam/rm-iam-manager.sh

Elastic Block Storage (EBS)

./ebs/rm-ebs.sh

Route 53

./route53/route53-delete-record.sh swift.internal manager.swift.internal

./route53/route53-delete-record.sh swift.internal worker-1.swift.internal
./route53/route53-delete-record.sh swift.internal worker-2.swift.internal

VPC

./vpc/rm-vpc.sh

Congratulations!

We are done with Step 8. Cleanup. This is the final step of this series.

Here are all the steps again for reference:

1. Network Setup
2. Storage
3. Roles
4. Manager Instance
5. Worker Launch Template
6. Worker Instances
7. Docker Swarm
8. Cleanup (this post)