Section 8_High Availability and Scalability: ELB & ASG

-Scalability means that an application / system can handle greater loads by adapting.

There are two kinds of scalability:

• Vertical Scalability

• Horizontal Scalability (= elasticity)

 

Vertical Scalability
- Increasing the size of the instance
- Vertical scalability is common for non-distributed systems such as database
- RDS, Elasticache are vertical scalability
- Usually a limit to how much you can vertically scale (hardware)

Horizontal scalability
- Increasing the number of instances
- Common for web applications / modern application
- Easy to horizontally scale thanks to cloud offerings such as Amazon EC2

 

High Availability
- Usually goes hand in hand with horizontal scaling and runs your application / system in at least 2 data centers
- The goal of high availability is to survive a data center loss
- High availability can be passive and can be active (horizontal scaling)

High Availability & scalability for EC2
- Vertical scaling : Increase instance size
- Horizontal scaling : Increase no. of instances
- High availability : Run instances for the same application across multi AZ

 

What is load balancing?
- Servers that forward traffic to multiple servers downstream (More users you have, the more load it gets balanced across EC2 instances but users do not know which EC2 it gets connected to)

Why use a load balancer?
- Spread load across multiple downstream instances
- Expose a single point of access (DNS) to your application
- Seamlessly handle failures of downstream instances
- Do regular health checks to your instances
- Provide SSL termination (HTTPS) for your websites
- Enforce stickiness with cookies
- High availability across zones
- Separate public traffic from private traffic

Why use an Elastic Load Balancer?
- ELB is a managed load balancer which costs less to setup your own load balancer but it will be a lot more effort on your end
- Integrated with many AWS offerings / services

Health Checks
- Crucial for Load balancers
- Enable the load balancer to know if the instances it forwards traffic to are available to reply to requests
- Health check is done on a port and a route and if the response is not 200, then the instance is unhealthy

Types of load balancers on AWS
- AWS has 4 kinds of managed Load balancers :
- Classic load balancer – CLB (HTTP, HTTPS, TCP, SSL) (Deprecated at AWS and not on the Exam)
- Application Load Balancer – ALB (HTTP, HTTPS, WebSocket)
- Network Load Balancer – NLB (TCP, TLS, UDP)
- Gateway Load Balancer – GWLB (Operates at layer 3 IP protocol)
Recommended to use the newer generation load balancers

Load Balancer Security Groups
- Users <-> Load Balancer <-> (HTTP restricted to Load Balancer) EC2

Application Load Balancer
- Layer 7 (HTTP) load balancer
- Load balancing to multiple HTTP applications across machines and multiple applications on the same machine (containers)
- Supports for HTTP/2 and websocket and supports redirects from HTTP to HTTPS
- Routing tables to different target groups :
  - routing based on path in URL
  - Routing based on hostname in URL
  - Routing based on Query String, Headers
- ALB are great fit for micro services & container-based application
- Has a port mapping feature to redirect to a dynamic port in ECS

 

ALB – Target groups
- EC2 instances – HTTP
- ECS tasks – HTTP
- Lambda functions – HTTP request is translated into a JSON event
- Ip addresses – must be a private IPs
- ALB can route to multiple target groups and health checks are at the target group level

ALB – Query Strings / Parameters Routing

 

ALB – Good to know
- Fixed hostname
- The application servers don’t see the IP of the client directly

ALB – Hands on
- Launch 2 EC2 Instances -> Create a ALB (in all AZ) -> Create security group for ALB (Inbound rule for HTTP and outbound traffic for all traffic) -> Create a target group (instances) and register the instances
- DNS name is available and once you copy the name into the web browser and keep on refreshing then it will alternate between the two IP instances
- If you stop one of the instances and go to target group for health checks it will change from healthy to unused and if you go back to the DNS page and refresh you will only get the IP address on the working instance.

ALB – Hands on part 2
- Under security groups, we can change inbound rules -> delete 80 port and instead of CIDR we can select a select the security group of the load balancer -> tighten network security
- Customize listener rules -> add rules -> add conditions (select condition types) -> path /error -> select action types -> select priority -> create rule
- Copy DNS name from Load balancer -> Rule of error will be shown on the web browser

 

Network Load Balancer
- Layer 4 allow to forward TCP & UDP traffic to your instances and handle millions of requests per second + Less latency
- NLB has one static IP per AZ, and supports assigning Elastic IP
- NLB are used for extreme performance, TCP or UDP traffic and not included in the AWS free tier

TCP Based traffic

Target groups :
- EC2 instances
- IP addresses – must be private IPs
- Application Load balancer
- Health checks support the TCP, HTTP and HTTPS protocols

NLB – Hands on
- Create a load balancer -> create a NLB -> Deploy in multiple AZ -> create a target group for NLB -> Target group protocol is TCP
- Open web browser with DNS name but it comes up as loading as instances are unhealthy -> NLB doesn’t define a security group for it that means all traffic goes through the network load balancer and goes straight into the instances -> check the security group for instances -> add a rule for inbound rules for HTTP from anywhere -> instances will change to healthy -> NLB DNS will open
- To delete you can delete inbound rules in security groups

Gateway Load Balancer
- Deploy, scale, and manage a fleet of 3rd part network virtual appliances such as firewalls, prevention systems, deep packet inspection system

- Operates at Layer 3 (network layer) – IP packets
- Combines the following functions :
 - Transparent Network Gateway – single entry/exit for all traffic
 - Load balancer – distributes traffic to your virtual appliances
- Uses the Geneve protocol on port 6081

Target groups
- EC2 instances
- IP addresses – must be private IPs

Elastic Load Balancer – Sticky Sessions
- It is possible to implement stickiness so that the same client is always redirected to the same instance behind a load balancer

- This works for Classic Load Balancers & Application Load Balancers

- The “cookie” used for stickiness has an expiration date you control

- Use case: make sure the user doesn’t lose his session data

- Enabling stickiness may bring imbalance to the load over the backend EC2 instances

Sticky Sessions – Cookie Names
- Application-based cookies
 - Custom cookie
  - generated by the target
  - Can include any custom attributes
  - Cookie name must be specified individually for each target group
  - Don’t use AWSALB, AWSALBBAPP or AWSALBTG

 - Application Cookie
  - Generated by the load balancer
  - Cookie name is AWSALBAPP
- Duration-based Cookies
 - Generated by load balancer and name is AWSALB for ALB, AWSELB for CLB

Hands on
- Target group -> select stickiness -> select stickiness type -> refresh the DNS web browser then it will come back to only one instance instead of 3 -> to return select target group and edit attributes

 

Cross-Zone Load Balancing
- Each load balancer instance distributes evenly across all registered instances in all AZ

Without Cross zone balancing
- Requests are distributed in the instances of the node of the ELB

 

Application Load balancer
- Enabled by default (can be disabled at target group level)
- No charges for inter AZ data

Network Load Balancer & Gateway Load Balancer
- Disabled by default
- Pay charges for inter AZ data if enabled

Classic load balancer
- Disabled by default
- No charges

Hands-on
- Load balancer -> select load balancer and go to attributes -> press edit and turn on cross zone load balancer or go to target group and select cross zone load balancer

SSL/TLS – Basics
- An SSL Certificate allows traffic between your clients and your load balancer to be encrypted in transit (in-flight encryption)

- SSL refers to Secure Sockets Layer, used to encrypt connections

- TLS refers to Transport Layer Security, which is a newer version

- Nowadays, TLS certificates are mainly used, but people still refer as SSL

- Public SSL certificates are issued by Certificate Authorities (CA)

- Comodo, Symantec, GoDaddy, GlobalSign, Digicert, Letsencrypt, etc…
- SSL certificates have an expiration date (you set) and must be renewed

 

Load balancer – SSL certificates

- Load balancer uses an X.509 certificate
- Manage certificates using ACM
- Create upload your own certificates
- HTTPS listener
 - must specify a default certificate
 - Add an optional list of certs to support multiple domains
- clients can use SNI to specify the hostname they reach
- Ability to specify a security policy to support older versions of SSL/TLS

SSL – Server name Indication
- SNI solves the problem of loading multiple SSL certificates onto one web server
- It’s a new protocol and requires the client to indicate the hostname of the target server in the initial SSL handshake
- The server will then find the correct certificate or return the default one
- Only works for ALB and NLB

ELB – SSL Certificates
- Classic Load Balancer (v1)

• Support only one SSL certificate

• Must use multiple CLB for multiple hostname with multiple SSL certificates

- Application Load Balancer (v2)

• Supports multiple listeners with multiple SSL certificates

• Uses Server Name Indication (SNI) to make it work
- Network Load Balancer (v2)

• Supports multiple listeners with multiple SSL certificates

• Uses Server Name Indication (SNI) to make it work

 

SSL certificates – Hands on
Select the load balancer -> set to HTTPS port 443 -> select Target group and set security policy -> Select default SSL from ACM / IAM / Import

Connection Draining
- Connection draining for CLB and Deregistration Delay for ALB & NLB
- Time to complete “in-flight requests” while the instance is de-registering or unhealthy

- Stops sending new requests to the EC2 instance which is de-registering

- Between 1 to 3600 seconds (default: 300 seconds)

- Can be disabled (set value to 0)
- Set to a low value if your requests are short

 

What is an auto Scaling Group?
- In real-life, the load on your websites and application can change

- In the cloud, you can create and get rid of servers very quickly

The goal of an Auto Scaling Group (ASG) is to:

• Scale out (add EC2 instances) to match an increased load

• Scale in (remove EC2 instances) to match a decreased load

• Ensure we have a minimum and a maximum number of EC2 instances running

• Automatically register new instances to a load balancer

• Re-create an EC2 instance in case a previous one is terminated (ex: if unhealthy)
- ASG are free (you only pay for the underlying EC2 instances)

Auto scaling Group attributes
A Launch Template (older “Launch Configurations” are deprecated)

• AMI + Instance Type

• EC2 User Data

• EBS Volumes

• Security Groups

• SSH Key Pair

• IAM Roles for your EC2 Instances

• Network + Subnets Information

• Load Balancer Information

- Min Size / Max Size / Initial Capacity

- Scaling Policies

Auto scaling – Cloud Watch alarms & scaling
It is possible to scale an ASG based on CloudWatch alarms

- An alarm monitors a metric (such as Average CPU, or a custom metric)

- Metrics such as Average CPU are computed for the overall ASG instances

Based on the alarm:

• We can create scale-out policies (increase the number of instances)

• We can create scale-in policies (decrease the number of instances

Auto Scaling Groups – Hands on
- Create auto scaling group -> create a launch template -> select instance type and ami, key pair and security group
- Choose instance launch option -> choose subnets -> next
- Attach the existing load balancer -> configure group size and scaling policies
- As soon as you create an ASG, a new instance will be made
- If the EC2 instance becomes unhealthy the old instance will be terminated and a new one will be made. (check through activity history) The reasons why this is happening is due to misconfiguring the EC2 instance or EC2 user data script issue
- Select ASG and you can edit the capacity (scale down and scale up)

ASG – Dynamic scaling policies
- Target Tracking Scaling

• Most simple and easy to set-up

• Example: I want the average ASG CPU to stay at around 40%

- Simple / Step Scaling

• When a CloudWatch alarm is triggered (example CPU > 70%), then add 2 units

• When a CloudWatch alarm is triggered (example CPU < 30%), then remove 1

- Scheduled Actions

• Anticipate a scaling based on known usage patterns

• Example: increase the min capacity to 10 at 5 pm on Fridays

ASG – Predictive scaling
- Continuously forecast load and schedule scaling ahead

Good metrics to scale on

• CPUUtilization: Average CPU utilization across your instances

• RequestCountPerTarget: to make sure the number of requests per EC2 instances is stable

• Average Network In / Out (if you’re application is network bound)

• Any custom metric (that you push using CloudWatch

ASG – Scaling Cooldowns
After a scaling activity happens, you are in the cooldown period (default 300 seconds)

• During the cooldown period, the ASG will not launch or terminate additional instances (to allow for metrics to stabilize)

• Advice: Use a ready-to-use AMI to reduce configuration time in order to be serving request fasters and reduce the cooldown period

ASG – scaling policies hands on
- You can create a scheduled action when you know the traffic (promotion event)
- You can create a predictive scaling policy (Machine learning) and set the metrics
- Dynamic scaling policy -> Simple scaling, tracking scaling, step scaling policies -> select target value
- Stress the EC2 CPU -> Alarm will trigger and create 1 more EC2 instance for ASG -> to check go to cloudwatch -> check alarms -> Delete the ASG for clean up