AWS Elastic Container Service
To begin this orchestration, i would like to have a brief overview on the containers and they internal architecture .
Container :
A container is an isolation in which your application or service uses and runs, the advantage of a container vs a virtual machine is that the kernel is integrated so you will not have any incompatibilities after deployment. there is no Guest OS and Hypervisor all are replaced by a docker daemon and some libraries or packages called Docker Images but you need infrastructure and Host OS.
Docker:
Docker is a container which is supported by almost all the service providers , it supports Linux and windows , this is GREAT for most of usages .
Linus vs Windows :
That totally depends on you and your needs and possibilities that you have to select one of them, but there are some differences as the size of Linux images are very small in comparison to Windows images.
both use a docker daemon to facilitate the interaction between OS and Container
Docker File :
A docker file is a text file consisting of different layers , a layer is a simple instruction, for example if you run the docker file for an application you need to say the container launch what it needs to prepare, deploy and run the container:
- Install the Apache server
- Get my application code
- Copy it to the www directory
- Grant permissions
- & …
Theses are the layers of a Docker image, the docker file needs be placed in the root of your application directory. An example of Asp.Net application docker file for windows container is as bellow:
FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build
WORKDIR /app
# copy csproj and restore as distinct layers
COPY *.sln .
COPY aspnetapp/*.csproj ./aspnetapp/
RUN dotnet restore
# copy everything else and build app
COPY aspnetapp/. ./aspnetapp/
WORKDIR /app/aspnetapp
RUN dotnet publish -c Release -o out
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2 AS runtime
WORKDIR /app
COPY --from=build /app/aspnetapp/out ./
ENTRYPOINT ["dotnet", "aspnetapp.dll"]
This DockerFile has two parts Sdk to build and publish and the Runtime to run the application, as the example shows we have different layers as restoring packages , publishing , changing work directories, coping files and folders.
Locally:
You can easily run your containers locally for testing purpose, simply install docker desktop and go run it, add a docker file , Build your image and run the container.
> docker build -t mydockerimageapp
Or
> docker image build -t myappdockerimageThese are both same, just the docker build is a pseudo for docker image build.
Next we need run it :
> docker run -it -rm myappdockerimageNow the container is running , have fun
Amazon ECS (Elastic Container Service):
ECS is a orchestration service used by amazon to manage your complex containerized solutions, it’s an easy setup process and highly integrated by the other services as ECR , Route53, ELB.
ECS Launch Types:
When using ECS you can have two launch types Ec2 and Fargate, As the name says the EC2 is based on an instance with a dedicated resource configuration, and Fargate is based on serverless architecture and allows you to use resources without any headache.
Considerations when selecting your Launch type:
- If you need large capacity and you consider Cost optimization and control so the EC2 will be best match for your solution , you can use on-demand, Spot or reserved instances to optimize better your costs.
- If your solution is based on a decoupled scenario or you use kind of batching Process so Fargate stands better.
- If your Solution is not heavily loaded and a burst on system is occasionally reached so Fargate matches better.
- On any Tiny workload as Dev, Test or integration Fargate is better as EC2 cause you lost of resources on these kind of environments.
ECS Clusters:
A cluster is a centralized part of ECS to Maintain , Schedule and Scale your containers, when using ECS the first part to be created is a cluster to orchestrate your containers.
Task Definition:
A Task Definition is a Json file which acts as a blueprint for Tasks describing which containers to run. to understand TaskDefnition let’s have a look at the snippet bellow:
{
"containerDefinitions": [
{
"command": [],
"cpu": 128,
"entryPoint": null,
"environment": [
{
"name": "Env_var_sample_name",
"value": "envvalue"
}
],
"image": "image_path",
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "ecs",
"awslogs-region": "eu-east-3",
"awslogs-stream-prefix": "testExample"
}
},
"memory": 300,
"memoryReservation": 128,
"name": "myapp"
}
],
"family" : "apptest",
"networkMode" : "host, bridge, none ,awsvpc ,Nat"}
the network mode for windows images are restricted to just NAT and must not specify it as it s set by default.
Task :
A Task is a logical parent for one or more container, the containers in the task can communicate with each other using localhost but any failure to one container will impact the other containers , for example if the container goes to a de-reregistration step so all the containers in the task go down.
Keep multiple containers in one task if really it’s a need by your architectural design , whenever possible.
Create the cluster:
To create your Ecs containerized platform you need to follow creating the following components
- VPC
- Subnets
- Security Grpoup
- ECS cluster
- ECS Cluster Instance
- Task Definition
- Service
- Namespace (optional)
VPC:
Use the Aws Cli to create a VPC with a cidr 10.0.0.0/16
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16Subnets:
The Subnets let you to have the sub-networks in your VPC in which you can achieve high availability by dividing your network in different facilities called AZ or DMZ
Create 3 subnets to achieve HA in 3 AZs eu-west-1a, eu-west-1b, eu-west-1c
aws ec2 create-subnet
--vpc-id vpc-078ac380b7c1d7df6
--cidr-block 10.0.1.0/24
--availability-zone eu-west-1aSecurity Group:
$ aws ec2 create-security-group
--group-name sgp-wksp-ecs
--vpc-id vpc-078ac380b7c1d7df6
--description "ecs cluster instance sg"ECS Cluster :
$ aws ecs create-cluster --cluster-name my-wksp-ecs-clusterCreating a cluster is really easy in ECS , also ECS is integrated with ECR a Container registry where you push your docker images, it’s an ideal idea for Your Golden docker images where you configure your app in a docker to simplify your deployment.
Next you need to run ECS instance to be able to run tasks , lunch the instance from Community AMI ‘amzn-ami-2018.03.x-amazon-ecs-optimized’ in the region i’m using the ami-id is ami-065adb4a6e048bdce
Cluster instance :
Use the Community AMI to launch your instance with a role with registration rights , consider using a role with AmazonEC2ContainerServiceforEC2Role policy and attach it to an instance profile , here i used a predefined role but you can create your own role.
$ aws iam create-instance-profile --instance-profile-name ecs-profile
$ aws iam add-role-to-instance-profile --instance-profile-name ecs-profile --role-name ecsInstanceRoleThe launch an instance
$ aws ec2 run-instances
--image-id ami-0ddb58069b428158d
--count 1
--iam-instance-profile Name="ecs-profile"
--instance-type t2.micro
--security-group-ids sg-003096d9f4923df59
--subnet-id subnet-00caaeb2d8565b61f
--user-data file://userdata.txtThe user data text file contains
#!/bin/bash
echo ECS_CLUSTER=my-wksp-ecs-cluster >> /etc/ecs/ecs.configIf you want to lunch any kind of other instances as Ubuntu you shall install the ECS agent on it by bootstraping step.
SG Ingress Rules:
aws ec2 authorize-security-group-ingress --group-id sg-00d6151aaa9107786 --protocol tcp --port 22 --cidr 0.0.0.0/0This lets you to connect to your instance via ssh
Create the Task Definition:
A Task is a service and the task definition defines the containers that run side by side. set the network mode to awsvpc and run following command
aws ecs register-task-definition --cli-input-json file://tasks/mytaskdefinition.jsonThe SG:
Let’s create the Security group to secure our platform, a security group lets you to define inbound and outbound traffic at instance level. considering that your cluster master is on an EC2 instance it’s a best practice to handle all traffic carefully.
in our scenario we need two SGs one for container service and one for ALB
$ aws ec2 create-security-group
--group-name sgp-wksp-elb
--description "elb sg"
--vpc-id vpc-078ac380b7c1d7df6Authorize ingress trafic from public
$ aws ec2 authorize-security-group-ingress --group-name sgp-wksp-elb --protocol tcp --port 80 --cidr 0.0.0.0/0And authorize ecs security group ingress traffic from alb
$ aws ec2 authorize-security-group-ingress --group-name sgp-wksp-ecs --protocol tcp --port 1-65535 --source-group sgp-wksp-elbCreate an I.G.W
$ aws ec2 create-internet-gatewayAttach it to the VPC
$ aws ec2 attach-internet-gateway --internet-gateway-id igw-09deafe329b17a969 --vpc-id vpc-078ac380b7c1d7df6Modify Route Table :
$ aws ec2 describe-route-tables --filters Name=vpc-id,Values=vpc-078ac380b7c1d7df6<you will recieve the following output>
{
"RouteTables": [
{
"Associations": [
{
"Main": true,
"RouteTableAssociationId": "rtbassoc-045e44d6e66a759b4",
"RouteTableId": "rtb-09c1c3459ea438cbc"
}
],
"PropagatingVgws": [],
"RouteTableId": "rtb-09c1c3459ea438cbc",
"Routes": [
{
"DestinationCidrBlock": "10.0.0.0/16",
"GatewayId": "local",
"Origin": "CreateRouteTable",
"State": "active"
},
{
"DestinationCidrBlock": "0.0.0.0/0",
"GatewayId": "igw-09deafe329b17a969",
"Origin": "CreateRoute",
"State": "active"
}
],
"Tags": [],
"VpcId": "vpc-043691d6593f7d674",
"OwnerId": "626540152810"
}
]
}
pick up the rtb-id and create a route as bellow, by this you transfer all traffic to your public entry point in VPC by Internet Gateway
$ aws ec2 create-route --route-table-id rtb-09c1c3459ea438cbc --destination-cidr-block 0.0.0.0/0 --gateway-id igw-09deafe329b17a969Create a ELB:
aws elb create-load-balancer
--load-balancer-name my-load-balancer
--listeners "Protocol=HTTP,LoadBalancerPort=80,InstanceProtocol=HTTP,InstancePort=80"
--subnets "subnet-01a9bc5a" "subnet-af479ee7" "subnet-c930fbaf"
--security-groups sg-06a4a13d5f5da9c10Create a Target Group:
$ aws elbv2 create-target-group --name my-wksp-targets --protocol HTTP --port 80 --target-type ip --vpc-id vpc-043691d6593f7d674Now we need to have a Autoscaling launch configuration
$ aws autoscaling create-launch-configuration
--launch-configuration-name my-launch-config
--security-groups sg-06a4a13d5f5da9c10
--instance-type t2.micro
--no-ebs-optimized
--no-associate-public-ip-address
--image-id ami-0ce71448843cb18a1Create Autoscaling Group:
aws autoscaling create-auto-scaling-group
--auto-scaling-group-name my-asg
--launch-configuration-name my-launch-config
--load-balancer-names my-load-balancer
--health-check-type ELB
--health-check-grace-period 120
--min-size 1
--max-size 3
--vpc-zone-identifier "subnet-01a9bc5a,subnet-af479ee7,subnet-c930fbaf"Now attach the target group to ELB
$ aws autoscaling attach-load-balancer-target-groups
--auto-scaling-group-name my-asg
--target-group-arn arn:aws:elasticloadbalancing:eu-west-1:626540152810:targetgroup/my-
targets/f5f7dba969d0391dRun It:
Now we are ready to run our task bu the following command
$ aws ecs run-task --cluster my-cluster-example --task-definition mytask:1 --count 1We could have an ECS cluster secured by a SG and a task running on it.
