Containers

Deep dive into cluster networking for Amazon EKS Hybrid Nodes

Amazon Elastic Kubernetes Service (Amazon EKS) Hybrid Nodes enables organizations to integrate their existing on-premises and edge computing infrastructure into EKS clusters as remote nodes. EKS Hybrid Nodes provides you with the flexibility to run your containerized applications wherever needed, while maintaining standardized Kubernetes management practices and addressing latency, compliance, and data residency needs.

EKS Hybrid Nodes accelerates infrastructure modernization by repurposing existing hardware investments. Organizations can harness the elastic scalability, high availability, and fully managed advantages of Amazon EKS, while making sure of operational consistency through unified workflows and toolsets across hybrid environments.

One of the key aspects of the EKS Hybrid Nodes solution is the hybrid network architecture between the cloud-based Amazon EKS control plane and your on-premises nodes. This post dives deep into the cluster networking configurations, guiding you through the process of integrating an EKS cluster with hybrid nodes in your existing infrastructure. In this walkthrough, we set up different Container Network Interface (CNI) options and load balancing solutions on EKS Hybrid Nodes to meet your networking requirements.

Architecture overview

EKS Hybrid Nodes needs private network connectivity between the cloud-hosted Amazon EKS control plane and the hybrid nodes running in your on-premises environment. This connectivity can be established using either Amazon Web Services (AWS) Direct Connect or AWS Site-to-Site VPN, through an AWS Transit Gateway or the Virtual Private Gateway into your Amazon Virtual Private Cloud (Amazon VPC).

For an optimal experience, AWS recommends reliable network connectivity with at least 100 Mbps bandwidth, and a maximum of 200ms round-trip latency, for hybrid nodes connecting to the AWS Region. This is general guidance rather than a strict requirement, and specific bandwidth and latency requirements may differ based on the quantity of hybrid nodes and your application’s unique characteristics.

The node and pod Classless Inter-Domain Routing (CIDR) blocks for your hybrid nodes and container workloads must be within the IPv4 RFC-1918 ranges. Make sure that these CIDR blocks are globally unique and routable across your hybrid network environment. When creating an EKS cluster with hybrid nodes enabled, you provide these CIDRs as inputs through the RemoteNodeNetwork and RemotePodNetwork settings for the cluster.

For this walkthrough, we use the following CIDRs for the demo setup:

  • Amazon EKS VPC CIDR: 10.250.0.0/16
  • On-premises Node CIDR (RemoteNodeNetwork): 192.168.200.0/24
  • On-premises Pod CIDR (RemotePodNetwork): 192.16.32.0/23

The following diagram is a high-level network architecture overview of our EKS Hybrid Nodes demo environment.

Demo network architecture for EKS Hybrid Nodes

Figure 1: Demo network architecture for EKS Hybrid Nodes

CNI considerations

The Amazon VPC CNI is not compatible with EKS Hybrid Nodes deployments. To make sure that hybrid nodes become operational and ready to handle workloads, you must install a compatible CNI (such as Calico or Cilium). Newly provisioned hybrid nodes in an EKS cluster remain in a NotReady state until a CNI is deployed.

If you are running webhooks on hybrid nodes, then you must make sure that on-premises Pod CIDRs are routable across the entire hybrid network environment. This is essential to allow the Amazon EKS control plane to communicate to the webhook pods running on your hybrid nodes.

However, if you cannot make your on-premises pod CIDRs routable at the local network, then you can alternatively run webhooks on cloud nodes in a mixed mode cluster. Refer to the Amazon EKS documentation for more details on design considerations for webhooks and mixed mode clusters.

There are several techniques you can use to make your on-premises pod CIDRs routable or reachable across the hybrid network, such as BGP routing, static routing, or ARP proxying. This post focuses on how to use BGP and static routing methods to make your on-premises pod CIDRs routable on the on-premises network.

Load balancing considerations

The CNI component is responsible for the in-cluster pod networking. However, a load balancer resource is also needed to enable external access to the microservices running on your hybrid nodes.

For services intended for local on-premises traffic, a load balancer controller such as MetalLB can be deployed. It supports the advertisement of load balancer external IP addresses using either Layer 2 (ARP) or Layer 3 (BGP) mode.

When operating in the Layer 2 mode, MetalLB needs you to reserve an address block from within the node subnet as the load-balancing external IP pool. Then, it automatically chooses one node to respond to ARP requests for a load balancer address allocated from the IP pool. This streamlines the load balancer address advertisement and eliminates the complexity of deploying and managing BGP at the remote or edge locations.

Alternatively, if you are planning to publish the service externally to the Region or to the internet, then we recommend using a cloud-native solution such as AWS Load Balancer Controller. It streamlines load balancer configuration and management by natively integrating with Kubernetes Services type of Load Balancer using Network Load Balancers (NLBs), and Ingress resources through Application Load Balancers (ALBs).

Prerequisites

The following prerequisites are necessary to complete this solution:

  • Amazon VPC with two private and two public subnets, across two Availability Zones (AZs).
  • Tag the two public subnets with the following format, as the prerequisites for integration with ALBs:
    • Key – kubernetes.io/role/elb
    • Value – 1
  • Deploy an EKS cluster with hybrid nodes – follow this AWS containers post for a detailed walkthrough.
  • On-premises compute nodes running a compatible operating system.
  • Private connectivity between the on-premises network and Amazon VPC (through VPN or Direct Connect).
  • Two routable CIDR blocks for RemoteNodeNetwork and RemotePodNetwork.
  • Configure the on-premises firewall and the EKS cluster security group to allow bi-directional communications between the Amazon EKS control plane and remote node and pod CIDRs, as per the networking prerequisites.
  • The following tools:

Walkthrough

In this walkthrough, we will set up cluster networking for hybrid nodes by exploring different routing methods, including BGP routing with the Cilium CNI and static routing with the Calico CNI. Next, we will install two load balancer options to the Amazon EKS cluster with hybrid nodes, including an on-premises load balancer example using MetalLB, and an external load balancer solution using the AWS Load Balancer Controller.

BGP routing (Cilium example)

BGP is a recommended solution for enhanced scalability and streamlined route distribution for the on-premises Pod CIDRs. In the following example, we deploy the Cilium CNI and configure the on-premises router as a BGP Route Reflector (RR). This enables the on-premises router to dynamically learn Pod CIDRs through BGP without participating in the data path for intra-cluster communications.

BGP routing for EKS Hybrid Nodes (Cilium example)

Figure 2: BGP routing for EKS Hybrid Nodes (Cilium example)

  1. Before we start, the hybrid nodes are currently showing NotReady because this is a newly provisioned cluster.
$ kubectl get nodes 
NAME                   STATUS     ROLES    AGE    VERSION
mi-060cf84b2fb8eb550   NotReady   <none>   112m   v1.32.1-eks-5d632ec
mi-0a5409bb63b61fff0   NotReady   <none>   111m   v1.32.1-eks-5d632ec
  1. First, we create a cilium-values.yaml file to specify the whole Pod CIDR range (192.168.32.0/23), with a /26 block size allocated for each node. This CIDR range needs to match the RemotePodNetwork as defined at the EKS cluster provision.
$ cat cilium-values.yaml 
kaffinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: eks.amazonaws.com/compute-type
          operator: In
          values:
          - hybrid
ipam:
  mode: cluster-pool
  operator:
    clusterPoolIPv4MaskSize: 26
    clusterPoolIPv4PodCIDRList:
    - 192.168.32.0/23
operator:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: eks.amazonaws.com/compute-type
            operator: In
            values:
              - hybrid
  unmanagedPodWatcher:
    restart: false
envoy:
  enabled: false
  1. Next, we deploy Cilium using the preceding values, and make sure to set bgpControlPlane.enabled to true.
$ helm repo add cilium https://helm.cilium.io/
$CILIUM_VERSION=1.16.7

$ helm install cilium cilium/cilium \
    --version ${CILIUM_VERSION} \
    --namespace kube-system \
    --values cilium-values.yaml \
    --set bgpControlPlane.enabled=true
  1. Verify the cilium-operator and the cilium-agent running on each of the hybrid nodes.
$ kubectl get pods -n kube-system -o wide | grep cilium
cilium-kw6ng                       1/1     Running   0              7m40s   192.168.200.12   mi-0a5409bb63b61fff0   <none>           <none>
cilium-operator-6dd8d6d489-x6zqt   1/1     Running   0              7m40s   192.168.200.12   mi-0a5409bb63b61fff0   <none>           <none>
cilium-operator-6dd8d6d489-xdw29   1/1     Running   0              7m40s   192.168.200.11   mi-060cf84b2fb8eb550   <none>           <none>
cilium-x4qjq                       1/1     Running   0              7m40s   192.168.200.11   mi-060cf84b2fb8eb550   <none>           <none>
  1. Apply BGP cluster and peer configurations, such as Autonomous System (AS) numbers, peer-router address, and BGP timer configurations.
$ cat cilium-bgp-cluster.yaml 
apiVersion: cilium.io/v2alpha1
kind: CiliumBGPClusterConfig
metadata:
  name: cilium-bgp
spec:
  nodeSelector:
    matchExpressions:
    - key: eks.amazonaws.com/compute-type
      operator: In
      values:
      - hybrid
  bgpInstances:
  - name: "home_lab"
    localASN: 65432
    peers:
    - name: "VyOS_RT01"
      peerASN: 65432
      peerAddress: 192.168.200.254
      peerConfigRef:
        name: "cilium-peer"
        
$ cat cilium-bgp-peer.yaml
apiVersion: cilium.io/v2alpha1
kind: CiliumBGPPeerConfig
metadata:
  name: cilium-peer
spec:
  timers:
    holdTimeSeconds: 30
    keepAliveTimeSeconds: 10
  gracefulRestart:
    enabled: true
    restartTimeSeconds: 120
  families:
    - afi: ipv4
      safi: unicast
      advertisements:
        matchLabels:
          advertise: "bgp"

$ kubectl apply -f cilium-bgp-cluster.yaml 
$ kubectl apply -f cilium-bgp-peer.yaml
  1. Configure the cluster to advertise Pod CIDRs from each node to the upstream router (BGP RR).
$ cat cilium-bgp-advertisement.yaml 
apiVersion: cilium.io/v2alpha1
kind: CiliumBGPAdvertisement
metadata:
  name: bgp-advertisements
  labels:
    advertise: bgp
spec:
  advertisements:
    - advertisementType: "PodCIDR"

$ kubectl apply -f cilium-bgp-advertisement.yaml
  1. Finally, apply the BGP configuration at the on-premises router (VyOS example).
set protocols bgp system-as '65432'
set protocols bgp neighbor 192.168.200.11 address-family ipv4-unicast route-reflector-client
set protocols bgp neighbor 192.168.200.11 address-family ipv4-unicast soft-reconfiguration inbound
set protocols bgp neighbor 192.168.200.11 graceful-restart 'enable'
set protocols bgp neighbor 192.168.200.11 remote-as '65432'
set protocols bgp neighbor 192.168.200.11 timers holdtime '30'
set protocols bgp neighbor 192.168.200.11 timers keepalive '10'
set protocols bgp neighbor 192.168.200.12 address-family ipv4-unicast route-reflector-client
set protocols bgp neighbor 192.168.200.12 address-family ipv4-unicast soft-reconfiguration inbound
set protocols bgp neighbor 192.168.200.12 graceful-restart 'enable'
set protocols bgp neighbor 192.168.200.12 remote-as '65432'
set protocols bgp neighbor 192.168.200.12 timers holdtime '30'
set protocols bgp neighbor 192.168.200.12 timers keepalive '10'
  1. Using Cilium CLI, we can verify the BGP session status at the EKS Hybrid Nodes.
$ cilium bgp peers
Node                   Local AS   Peer AS   Peer Address      Session State   Uptime   Family         Received   Advertised
mi-060cf84b2fb8eb550   65432      65432     192.168.200.254   established     57m30s   ipv4/unicast   2          2    
mi-0a5409bb63b61fff0   65432      65432     192.168.200.254   established     57m32s   ipv4/unicast   2          2    
  1. This can also be confirmed at the on-premises router as we are learning the two /26 Pod CIDRs from the hybrid nodes.
vyos@VyOS-RT01:~$ show ip bgp 
BGP table version is 2, local router ID is 192.168.200.254, vrf id 0
Default local pref 100, local AS 65432
Status codes:  s suppressed, d damped, h history, u unsorted, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 *>i 192.168.32.0/26  192.168.200.12                100      0 i
 *>i 192.168.32.64/26 192.168.200.11                100      0 i
  1. The status of the hybrid nodes are now changed to Ready with Cilium CNI installed.
$ kubectl get nodes 
NAME                   STATUS   ROLES    AGE    VERSION
mi-060cf84b2fb8eb550   Ready    <none>   132m   v1.32.1-eks-5d632ec
mi-0a5409bb63b61fff0   Ready    <none>   131m   v1.32.1-eks-5d632ec

Static routing (Calico example)

For smaller-scale hybrid node deployments, or if using an on-premises gateway lacking BGP support, static routing can be configured to direct traffic for specific Pod CIDRs to their respective nodes. This approach necessitates static assignment of Pod CIDRs to individual nodes. The following steps demonstrate how to achieve this using the Calico CNI.

Static routing for EKS Hybrid Nodes (Calico example)

Figure 3: Static routing for EKS Hybrid Nodes (Calico example)

  1. To allocate per-node based Pod CIDR, we first apply custom labels to each node.
$ kubectl label nodes mi-060cf84b2fb8eb550 zone=node-01
node/mi-060cf84b2fb8eb550 labeled

$ kubectl label nodes mi-0a5409bb63b61fff0 zone=node-02
node/mi-0a5409bb63b61fff0 labeled
  1. We create a Calico configuration file that maps specific Pod CIDR block to each node using the matching labels. These CIDRs must fall into the RemotePodNetwork as defined at the EKS cluster provision stage.
$ cat calico-values.yaml 
installation:
  enabled: true
  cni:
    type: Calico
    ipam:
      type: Calico
  calicoNetwork:
    bgp: Disabled
    ipPools:
    - cidr: 192.168.32.0/26
      encapsulation: VXLAN
      natOutgoing: Enabled
      nodeSelector: zone == "node-01"
    - cidr: 192.168.32.64/26
      encapsulation: VXLAN
      natOutgoing: Enabled
      nodeSelector: zone == "node-02"  
  controlPlaneReplicas: 1
  controlPlaneNodeSelector:
    eks.amazonaws.com/compute-type: hybrid
  calicoNodeDaemonSet:
    spec:
      template:
        spec:
          nodeSelector:
            eks.amazonaws.com/compute-type: hybrid
  csiNodeDriverDaemonSet:
    spec:
      template:
        spec:
          nodeSelector:
            eks.amazonaws.com/compute-type: hybrid
  calicoKubeControllersDeployment:
    spec:
      template:
        spec:
          nodeSelector:
            eks.amazonaws.com/compute-type: hybrid
  typhaDeployment:
    spec:
      template:
        spec:
          nodeSelector:
            eks.amazonaws.com/compute-type: hybrid
  1. Now we use Helm to install Calico with the configuration file.
$ helm repo add cilium https://helm.cilium.io/
$ helm install calico projectcalico/tigera-operator --version 3.29.2 --namespace kube-system -f calico-values.yaml
  1. Two Pod IP pools are automatically created, each containing a /26 CIDR block assigned to a specific node using the custom labels defined previously.
$ kubectl get ippools.crd.projectcalico.org 
NAME               AGE
192.168.32.0-26    5d3h
192.168.32.64-26   5d3h

$ kubectl describe ippools 192.168.32.0-26
Name:         192.168.32.0-26
[...]
  Block Size:     26
  Cidr:           192.168.32.0/26
  Ipip Mode:      Never
  Nat Outgoing:   true
  Node Selector:  zone == "node-01"
  Vxlan Mode:     Always

$ kubectl describe ippools 192.168.32.64-26
Name:         192.168.32.64-26
[...]
  Block Size:     26
  Cidr:           192.168.32.64/26
  Ipip Mode:      Never
  Nat Outgoing:   true
  Node Selector:  zone == "node-02"
  Vxlan Mode:     Always
  1. Lastly, we configure two static routes for the Pod CIDRs at the on-premises router targeting the two nodes, and redistribute them across the hybrid network environment.
vyos@VyOS-RT01# set protocols static route 192.168.32.0/26 next-hop 192.168.200.11
vyos@VyOS-RT01# set protocols static route 192.168.32.64/26 next-hop 192.168.200.12

On-premises load balancer (MetalLB example)

In the following example, we show how to implement a MetalLB solution in Layer 2 mode on our EKS cluster with hybrid nodes.

  1. First, we install MetalLB using Helm, and confirm the controller and speakers are in the Running status.
$ helm install metallb metallb/metallb \
  --namespace metallb-system \
  --create-namespace \
  --set "loadBalancerClass=metallb.io/metallb"

$ kubectl get pods -n metallb-system 
NAME                          READY   STATUS    RESTARTS   AGE
controller-74b6dc8f85-ngswz   1/1     Running   0          3d5h
speaker-qg8ps                 1/1     Running   0          3d5h
speaker-sgpp2                 1/1     Running   0          3d5h
  1. Next, we reserve a small address block (192.168.200.201-220) from the node subnet, and assign it for the load balancer external IP pool using the Layer 2 mode. MetalLB now responds to all ARP requests targeting the load balancer addresses allocated from the Layer 2 IP pool, thus eliminating the need of any Layer 3 advertisement.
$ cat metallb-config.yaml 

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default
  namespace: metallb-system
spec:
  addresses:
  - 192.168.200.201-192.168.200.220
  autoAssign: true
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system
spec:
  ipAddressPools:
  - default

$ kubectl apply -f metallb-config.yaml
  1. We create an Nginx pod for quick testing.
$ kubectl run nginx —image=nginx —restart=Never —labels="app=nginx" —port=80 -n default
  1. We then create a load balancer service to expose the Nginx pod using the MetalLB load balancer controller.
$ cat nginx-lb-metallb.yaml
 
apiVersion: v1
kind: Service
metadata:
  name: nginx-lb
spec:
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80
  type: LoadBalancer
  loadBalancerClass: metallb.io/metallb   
 
$ kubectl apply -f nginx-lb-metallb.yaml
  1. We can see a LoadBalancer service is deployed with an external IP (192.168.200.201) automatically assigned from the previously defined Layer 2 IP pool.
$ kubectl get svc nginx-lb
NAME       TYPE           CLUSTER-IP       EXTERNAL-IP       PORT(S)        AGE
nginx-lb   LoadBalancer   172.16.208.141   192.168.200.201   80:32373/TCP   112s
  1. From the on-premises network, we should be able to access the Nginx service through the load balancer’s external IP.
$ curl http://192.168.200.201
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
[...]

External load balancer (AWS Load Balancer Controller example)

The following section demonstrates how to install the AWS Load Balancer Controller onto our EKS cluster with hybrid nodes. Then, we deploy a demo application using the Kubernetes Ingress through the integration with ALB.

  1. First, we set up an OpenID Connect (OIDC) provider for our EKS cluster and create an AWS Identity and Access Management (IAM) policy for deploying the AWS Load Balancer Controller.
$ cluster_name=<my-cluster>
$ eksctl utils associate-iam-oidc-provider --cluster $cluster_name --approve

$ curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.12.0/docs/install/iam_policy.json
$ aws iam create-policy  --policy-name AWSLoadBalancerControllerIAMPolicy  --policy-document file://iam_policy.json
  1. Second, we create an Amazon EKS service account for the AWS Load Balancer Controller.
$ eksctl create iamserviceaccount \
  --cluster=<my-cluster> \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --attach-policy-arn=arn:aws:iam::<my-account-id>:policy/AWSLoadBalancerControllerIAMPolicy \
  --override-existing-serviceaccounts \
  --region <aws-region-code> \
  --approve 
  1. Next, we install the AWS Load Balancer Controller using Helm and verify the deployment status.
$ helm repo add eks https://aws.github.io/eks-charts
$ helm repo update eks

$ helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=<my-cluster> \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set region=<my-vpc-region> \
  --set vpcId=<my-vpc-id>

$ kubectl get deployment -n kube-system aws-load-balancer-controller
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   2/2     2            2           39s
  1. We use a 2048 game demo application to test the controller. To register pods running on hybrid nodes as remote targets for the ALB, we need to specify target-type as IP mode. This is configured using the alb.ingress.kubernetes.io/target-type: ip annotation as defined in the 2048 deployment file under the Ingress section.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: game-2048
  name: ingress-2048
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
  1. Deploy the demo application using Kubernetes Ingress through the ALB integration.
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.12.0/docs/examples/2048/2048_full.yaml
  1. We see there are five (replica) pods running across our hybrid nodes, and an ALB is automatically created as the Kubernetes Ingress for publishing the demo application to the internet.
$ kubectl get pods -n game-2048 -o wide
NAME                               READY   STATUS    RESTARTS   AGE   IP              NODE                   NOMINATED NODE   READINESS GATES
deployment-2048-7df5f9886b-78gw5   1/1     Running   0          15m   192.168.32.25   mi-060cf84b2fb8eb550   <none>           <none>
deployment-2048-7df5f9886b-bp5kj   1/1     Running   0          15m   192.168.32.81   mi-0a5409bb63b61fff0   <none>           <none>
deployment-2048-7df5f9886b-hp4fh   1/1     Running   0          15m   192.168.32.26   mi-060cf84b2fb8eb550   <none>           <none>
deployment-2048-7df5f9886b-mtrrp   1/1     Running   0          15m   192.168.32.24   mi-060cf84b2fb8eb550   <none>           <none>
deployment-2048-7df5f9886b-wp67m   1/1     Running   0          15m   192.168.32.82   mi-0a5409bb63b61fff0   <none>           <none>

$ kubectl get ingress -n game-2048 
NAME           CLASS   HOSTS   ADDRESS                                                                      PORTS   AGE
ingress-2048   alb     *       k8s-game2048-ingress2-184c5765e8-32714527.ap-southeast-2.elb.amazonaws.com   80      8m29s
  1. We can now access the 2048 game through the ALB DNS address, as shown in the following figure.
Accessing the demo app hosted on EKS Hybrid Nodes through ALB

Figure 4: Accessing the demo app hosted on EKS Hybrid Nodes through ALB

Cleaning up

To remove the demo resources created in the preceding steps, use the following commands:

$ kubectl delete pod nginx 
$ kubectl delete svc nginx-lb
$ kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.12.0/docs/examples/2048/2048_full.yaml

If the demo environment is no longer needed, then delete the EKS cluster to avoid incurring charges.

$ eksctl delete cluster --name <my-cluster> --region <aws-region-code>

Conclusion

In this post, we went through detailed steps for setting up the cluster networking for Amazon EKS Hybrid Nodes. We explored different CNI and load balancing solutions for various use cases and scenarios.

On-premises and edge environments often vary, and you should choose the approach that best fits your networking needs. If it’s possible for your environment, then it’s recommended to use the CNI’s built-in BGP functionality to advertise Pod CIDRs and to use AWS services such as ALB to remove undifferentiated heavy lifting.

To learn more about EKS Hybrid Nodes and apply these techniques to your own environments, see the following resources:

About the author

Sheng Chen is a Sr. Specialist Solutions Architect at AWS Australia, bringing over 20 years of experience in IT infrastructure, cloud architecture and multi-cloud networking. In his current role, Sheng helps customers accelerate cloud migrations and application modernization by leveraging cloud-native technologies. His recent focus includes Kubernetes, K8s networking, Agentic AI and platform engineering.

Frank Fan is a Sr. Container Solutions Architect at AWS Australia. As a passionate advocate for application modernization, Frank specializes in containerization and overseeing large-scale migration and modernization initiatives. Frank is a frequent speaker at prominent tech events including AWS reInvent, AWS Summit and Kubernetes Community Day. You can get in touch with Frank via his LinkedIn page, and his presentations are available at his YouTube channel.