Day 48 - Amazon Elastic Container Service(ECS)

What is Amazon ECS?

Amazon Elastic Container Service (ECS) is a cloud computing service in Amazon Web Services (AWS) that manages containers and lets developers run applications in the cloud without having to configure an environment for the code to run in.

It enables developers with AWS accounts to deploy and manage scalable applications that run on groups of servers called clusters through API calls and task definitions.

Amazon ECS is a scalable service accessible through the AWS Management Console and SDKs.

With Amazon ECS, developers can easily use Docker containers for a range of activities like, from hosting a simple website to running complex, distributed microservices that require thousands of containers.

ECS evaluates and monitors CPU and memory output to determine the optimal deployment for a container. AWS customers can also use the service to update containers or scale them up or down.

What is ECS used for?

Amazon ECS is best used with:

1. Machine learning: Machine learning models can be easily containerized for training and inference with Amazon ECS. ML models can be created with loosely coupled, distributed services that can be placed on a variety of platforms or close to the data being analyzed by the application.

2. Microservices: Amazon ECS assists in the operation of microservices applications by providing native integration to AWS and enabling CI/CD pipelines.

3. VMs: The Amazon EC2 web service can create and operate Linux VMs in the cloud, which are called as instances. Developers can specify rules for the isolated sets of EC2 instances that run on top of a host OS which increase computing performance and portability.

4. Migrating apps to the cloud: Legacy enterprise applications can be feasibly containerized and migrated to Amazon ECS without necessitating any code changes. This is an expression of lift-and-shift application migration.

5. Batch processing: Batch workloads can run with custom or managed schedulers on AWS On-Demand Instances, Reserved Instances or Spot Instances.

Amazon ECS features

Below are the some of the features of Amazon ECS: -

1. Scheduling: Schedulers place containers over clusters according to the desired resources, such as RAM or CPU and availability requirements. This feature can be used to schedule batch jobs and long-running applications or services. Amazon ECS includes two schedulers to deploy containers based on computing needs or availability requirements. ECS also supports third-party scheduling options.

2. Docker integration: Amazon ECS supports Docker, which enables AWS users to manage Docker containers across clusters of Amazon EC2 instances. Each EC2 instance in a cluster runs a Docker daemon that deploys and runs any application packaged as a container locally on Amazon ECS without the need to make any changes to the container.

3. Networking: Amazon ECS supports Docker networking, as well as integration with Amazon Virtual Private Cloud (Amazon VPC), to provide isolation for containers. This provides developers with control over how the containers interact with other services and external traffic.

4. Cluster management: Amazon ECS handles all of the cluster management processes for the developer. This typically involves installing, operating and scaling cluster management software, monitoring solutions and configuration management systems, as well as building the architecture and managing the availability and scalability of each system. With Amazon ECS, the developer simply launches a cluster of container instances and specifies the desired tasks to perform.

5. Task definitions: Users define tasks through a declarative JSON template called a Task Definition. The Task Definition lets developers specify which containers they need for their task, including memory and CPU requirements, Docker repository and images, and shared data volumes, and also choose how the containers connect to each other. Task Definition files also enables developers to version control their application specification.

6. Load balancing: Integration with AWS ELB lets developers to distribute traffic across containers. They can specify the Task Definition and ELB to use, and then the Amazon ECS scheduler automatically adds and removes containers using the ELB.

7. Repository support: Developers can use any third-party repository, accessible private Docker registry or Docker Hub with Amazon ECS as long as it is specified in the Task Definition.

8. Local development: The AWS CLI lets users simplify the local development experience and set up an Amazon ECS cluster and its related resources. The CLI also supports Docker Compose, an open source tool used to define and run multi-container applications.

9. Programmatic control: Various simple APIs let developers integrate and extend the Amazon ECS service. With APIs, users can create or delete clusters, launch or destroy Docker containers and register or unregister tasks, as well as access detailed information about the state of the cluster and its instances. Developers can also use AWS CloudFormation to deliver Amazon ECS clusters, register Task Definitions and schedule containers.

10. Logging: Amazon CloudWatch Logs receives every container instance's ECS agent logs and Docker container logs for issue diagnosis. All Amazon ECS API calls can also be recorded and the log files will be delivered to the user through AWS CloudTrail.

11. Monitoring: Monitoring capabilities check the health of containers and clusters. Average and aggregate CPU can be supervised, as well as the memory utilization of running tasks grouped by Task Definition, service or cluster through Amazon CloudWatch. Furthermore, users can set CloudWatch alarms to alert developers whenever a container or cluster needs to be scaled up or down.

12. Container deployments: Whenever a new version of the application Task Definition is uploaded, the Amazon ECS scheduler automatically starts new containers using the updated image and disables any container running on the old version. Amazon ECS will also register and unregister the appropriate new and old containers from the AWS ELB.

13. Container auto-recovery: Amazon ECS service scheduler automatically recovers unhealthy containers. This ensures the necessary number of containers are constantly supporting the application.

14. Container security: EC2 instances reside in the Amazon VPC and a user can specify which instances are exposed to the internet. EC2 instances and ECS tasks also adhere to IAM roles, while security groups and network access control lists limit access to instances.

How does Amazon ECS works?

With Amazon ECS, developers can pull the necessary Docker images and resources from Amazon Elastic Container Registry (ECR), or other repositories, to define their application. The ECS service then ingests container images and arranges or composes containers and resources into an application. Once all the appropriate containers are gathered and services implemented, the containers are deployed either on EC2 or AWS Fargate. Finally, Amazon ECS scales the application and continuously manages the availability of containers.

AWS account holders can integrate the ECS service with other Amazon Web Services, such as: AWS CloudTrail logs, AWS CLI, Amazon EC2, AWS CloudFormation templates, AWS SDKs, AWS Tools for Windows PowerShell and Amazon ECR.

Amazon ECS Layers

Amazon ECS comprises three key layers, each serving a distinct role in the orchestration of containerized applications:

1. Capacity Layer: This foundational layer encompasses the infrastructure on which your containers execute, providing the essential computing resources for seamless operation.

2. Controller Layer: At the core of Amazon ECS, the Controller layer is responsible for deploying and overseeing the applications that run within the containers. It acts as the orchestrator, ensuring efficient deployment and management.

3. Provisioning Layer: The Provisioning layer introduces tools designed for interfacing with the scheduler, offering a user-friendly means to deploy and manage applications and containers. This layer plays a crucial role in facilitating the seamless interaction between the user and the orchestration system.

Together, these layers establish a robust framework for deploying, managing, and scaling containerized applications on Amazon ECS, streamlining the entire process from infrastructure provision to application deployment and orchestration. As an example, a user can leverage the Provisioning layer.

ECS Application Lifecycle

The application lifecycle in Amazon ECS (Elastic Container Service) involves the management of tasks and services throughout their lifecycle.

1. Task Definition Creation: The application lifecycle starts with creating a task definition. A task definition is a blueprint that defines how containers should run within ECS.

2. Task Scheduling: Once a task definition is created, tasks based on that definition can be scheduled on ECS container instances or as Fargate tasks.

3. Task Execution: When a task is scheduled to run, ECS provisions the necessary resources, such as EC2 instances or Fargate resources, to host the containers.

4. Task Monitoring: During task execution, you can monitor the health and resource utilization of tasks using Amazon CloudWatch.

5. Task Updates: If you need to update a running task, you can create a new task definition version with the desired changes and update the service to use the new version.

6. Scaling: ECS provides scaling capabilities to manage the number of tasks running in a service.

7. Service Management: Services in ECS provide higher-level abstractions for managing long-running tasks.

8. Task Termination: When a task is no longer needed or needs to be stopped, you can manually terminate the task or update the service to a desired count of zero.

Amazon ECS Use Cases

Two most common use case in Amazon ECS is: -

1. Fargate is suitable for the following workloads:

• Large workloads that need to be optimized for low overhead

• Small workloads that have the occasional burst

• Tiny workloads

• Batch workloads

2. EC2 is suitable for the following workloads:

• Workloads that require consistently high CPU core and memory usage

• Large workloads that need to be optimized for price

• Your applications need to access persistent storage

• You must directly manage your infrastructure

ECS also integrates with other AWS services, such as Elastic Load Balancing, Auto Scaling, and Amazon VPC, allowing you to build scalable and highly available applications. Additionally, ECS has support for Docker Compose and Kubernetes, making it easy to adopt existing container workflows.

Difference between EKS and ECS

TASK: Set up ECS by setting up Nginx on ECS

Let's start with the steps👇

Step 1: Login to AWS Console and on search bar type "ECS".

Step 2: Click on "Elastic Container Service". You will see the below page.

Step 3: Click on "Create Cluster". Give Cluster named as NginxCluster. And leave the rest settings as it is.

Step 4: Click on "Create".

Step 5: In the left Panel of ECS, Select "Task Definitions".

Step 6: Click on Create new task definition, Select Create new task definition from dropdown.

Step 7: Now, Configure task definition and container.

1. Task definition family: nginx-task

2. Under Container details: -

   Name: nginx-container

   Image URI: public.ecr.aws/nginx/nginx:mainline-apline

   You can get this Image URL from Amazon ECR Public Gallery.

3. Let other things be as it is.

   

   

Step 8: Click on Create.

Step 9: Now Go to ECS, Select & open the Cluster you created.

Step 10: Click on "Create Service". Let the Environment section be the default.

Step 11: In the Deployment Configuration section > Select Service > Give the Service Name > Select the task definition you created.

Step 12: In the Networking Tab > Security Group > Click on Create new SG.

1. Security group name: nginx-SG

2. Security group description: Security Group for Nginx Cluster

3. Inbound Rule for SG : Select Type as "HTTP" and Source as "Anywhere"

4. And leave the rest configuration as it is

Step 13: Click on Create.

Step 14: Let’s test by accessing the Nginx container using the Public IPv4 of Fargate.

For the IP go to the Tasks tab in the ECS Cluster dashboard > Select the Task number of your task required > You can find the Public IPv4.

This can be further done by exposing Nginx publicly, by setting up an Application Load Balancer (ALB). And then can be reached by the public IP address of your load balancer.

Conclusion

In Conclusion, Elastic Container Service(ECS) is a fully managed container orchestration service provided by Amazon Web Services (AWS). It simplifies the deployment, management, and scaling of containerized applications using Docker containers.

ECS allows users to easily run and scale applications in a flexible and efficient manner, making it a valuable tool for organizations looking to leverage container technology for their software deployment needs. With features like task definitions, service definitions, and integration with other AWS services

In this blog, We have seen about ECS, how it works, where it is used, its features, different layers and lifecycle of ECS, difference between EKS & ECS and then how to set up an Nginx cluster in ECS.

Hope you find it helpful🤞 So I encourage you to try this on your own and let me know in the comment section👇 about your learning experience.✨

👆The information presented above is based on my interpretation. Suggestions are always welcome.😊

~Smriti Sharma✌