Deploy Docker Swarm on AWS EC2 via cloud-formation templates - Step 6 - Worker Instances

In this step we will launch the Worker EC2 instances.

This post is part of a thread that includes these steps:

Worker Instances (AWS EC2)

Start in the project directory:

cd ~/swift-aws-ec2-swarm

cloud-formation Template

Create a folder ec2-worker and a ec2-worker.yml file in it.

mkdir -p ec2-worker
touch ec2-worker/ec2-worker.yml
nano ec2-worker/ec2-worker.yml

Copy and paste this code into ec2-worker.yml:

Description: Launch Docker Swarm Worker instances

Parameters:
  KeyPair:
    Type: AWS::EC2::KeyPair::KeyName
    Description: Key pair that will be used to launch instances

  SubnetId:
    Type: AWS::EC2::Subnet::Id
    Description: Subnet in the VPC where the instance will be launched

  SecurityGroupId:
    Type: AWS::EC2::SecurityGroup::Id
    Description: Security group for the instance

  InstanceProfile:
    Type: String
    Description: Instance profile to use for the instance

  InstanceType:
    Type: String
    Default: c5.large
    Description: Instance type to use for the instance

  LaunchTemplateId:
    Type: String
    Description: The ID of the launch template.

Resources:
  Worker1Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
      KeyName: !Ref KeyPair

      SubnetId: !Ref SubnetId
      SecurityGroupIds:
        - !Ref SecurityGroupId      

      IamInstanceProfile: !Ref InstanceProfile

      InstanceType: !Ref InstanceType

      LaunchTemplate:
        LaunchTemplateId: !Ref LaunchTemplateId
        Version: "1"

      # categories:
        - Key: "Name"
          Value: "worker-1"

  Worker2Instance:
    Type: 'AWS::EC2::Instance'
    Properties:
    Properties:
      KeyName: !Ref KeyPair

      SubnetId: !Ref SubnetId
      SecurityGroupIds:
        - !Ref SecurityGroupId      

      IamInstanceProfile: !Ref InstanceProfile

      InstanceType: !Ref InstanceType

      LaunchTemplate:
        LaunchTemplateId: !Ref LaunchTemplateId
        Version: "1"

      # categories:
        - Key: "Name"
          Value: "worker-2"

Outputs:
  Worker1InstanceId:
    Description: ID of the launched worker-1 instance
    Value: !Ref Worker1Instance
  Worker2InstanceId:
    Description: ID of the launched worker-2 instance
    Value: !Ref Worker2Instance

Scripts

Add a script deploy-ec2-worker.sh and paste this code in it:

#!/usr/bin/env bash

# switch to parent directory
script_path=`dirname ${BASH_SOURCE[0]}`
pushd $script_path/..

source config/names.sh

echo
echo "Deploying $stack_ec2_worker stack via cloud-formation:"
echo 'https://us-west-2.console.aws.amazon.com/cloudformation/home'
echo

subnet_id=$(aws ec2 describe-subnets --filters Name=tag:Name,Values=$subnet_priv_1 | jq -r '.Subnets[0].SubnetId')
security_group_id=$(aws ec2  describe-security-groups --filters Name=tag:Name,Values=$security_group_priv_1 | jq -r '.SecurityGroups[0].GroupId')

instance_profile=$(aws cloudformation describe-stacks --stack-name $stack_iam_worker | jq -r '.Stacks[0].Outputs[] | select(.OutputKey == "InstanceProfile") | .OutputValue')

launch_template_id=$(aws cloudformation describe-stacks --stack-name $stack_ec2_worker_lt | jq -r '.Stacks[0].Outputs[] | select(.OutputKey == "LaunchTemplateId") | .OutputValue')

set -x

aws cloudformation deploy \
    --template-file ec2-worker/ec2-worker.yml \
    --stack-name $stack_ec2_worker \
    --parameter-overrides \
        KeyPair=$ec2_key_pair \
        SubnetId=$subnet_id \
        SecurityGroupId=$security_group_id \
        InstanceProfile=$instance_profile \
        LaunchTemplateId=$launch_template_id

popd

Let's also add a clean up script rm-ec2-worker.sh:

#!/usr/bin/env bash

# switch to parent directory
script_path=`dirname ${BASH_SOURCE[0]}`
pushd $script_path/..

source config/names.sh
echo
echo "Destroying $stack_ec2_worker stack via cloud-formation:"
echo 'https://us-west-2.console.aws.amazon.com/cloudformation/home'
echo

set -x

aws cloudformation delete-stack \
    --stack-name $stack_ec2_worker

aws cloudformation wait stack-delete-complete \
    --stack-name $stack_ec2_worker

popd

Make the scripts executable:

chmod +x ec2-worker/deploy-ec2-worker.sh 
chmod +x ec2-worker/rm-ec2-worker.sh

Deploy

Finally let's run the "deploy" script to create the Worker launch template:

./ec2-worker/deploy-ec2-worker.sh

You should see output similar to this:

Deploying swift-swarm-ec2-worker stack via cloud-formation:
https://us-west-2.console.aws.amazon.com/cloudformation/home

+ aws cloudformation deploy --profile swift --template-file ec2-worker/ec2-worker.yml --stack-name swift-swarm-ec2-worker --parameter-overrides KeyPair=aws-ec2-key SubnetId=subnet-0b019a9e1c3f56adb SecurityGroupId=sg-07c3b321f35b08b18 InstanceProfile=swift-swarm-iam-worker-InstanceProfile-qF4QqiJApx6p LaunchTemplateId=lt-06f34a0e6c33a503a

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - swift-swarm-ec2-worker

At this point your project structure should look like this:

.
├── config
│   └── names.sh
├── ebs
│   ├── deploy-ebs.sh
│   ├── ebs.yml
│   └── rm-ebs.sh
├── ec2-manager
│   ├── deploy-ec2-manager.sh
│   ├── ec2-manager.yml
│   └── rm-ec2-manager.sh
├── ec2-worker
│   ├── deploy-ec2-worker.sh
│   ├── ec2-worker.yml
│   └── rm-ec2-worker.sh
├── ec2-worker-lt
│   ├── deploy-ec2-worker-lt.sh
│   ├── ec2-worker-lt.yml
│   └── rm-ec2-worker-lt.sh
├── iam
│   ├── deploy-iam-manager.sh
│   ├── deploy-iam-worker.sh
│   ├── iam-manager.yml
│   ├── iam-worker.yml
│   ├── rm-iam-manager.sh
│   └── rm-iam-worker.sh
├── ssh
│   └── ssh-manager.sh
└── vpc
    ├── deploy-vpc.sh
    ├── rm-vpc.sh
    └── vpc.yml

Congratulations!

We are done with Step 6. Worker Instances.

Next step is: Step 7. Docker Swarm