Containers

Scaling beyond IPv4: integrating IPv6 Amazon EKS clusters into existing Istio Service Mesh

Organizations are increasingly adopting IPv6 for their Amazon Elastic Kubernetes Service (Amazon EKS) deployments, driven by three key factors: depletion of private IPv4 addresses, the need to streamline or eliminate overlay networks, and improved network security requirements on Amazon Web Services (AWS). In IPv6-enabled EKS clusters, each pod receives a unique IPv6 address from the Amazon Virtual Private Cloud (Amazon VPC) IPv6 range, with seamless compatibility facilitated by the Amazon EKS VPC Container Network Interface (CNI). This solution effectively addresses two major IPv4 limitations: the scarcity of private addresses and the security vulnerabilities created by overlapping IPv4 spaces that need Network Address Translation (NAT) at the node level.

When transitioning to IPv6, you likely need to run both IPv4 and IPv6 EKS clusters simultaneously. This is particularly important for organizations using Istio Service Mesh with Amazon EKS, because IPv6 clusters must integrate with the existing Service Mesh and work smoothly alongside IPv4 clusters. To streamline this transition, you can configure your Istio Service Mesh to support both your current IPv4 EKS clusters and your new IPv6 EKS clusters. If Istio Service Mesh isn’t part of your infrastructure, then we suggest exploring Amazon VPC Lattice as an alternative solution to speed up your IPv6 implementation on AWS.

This post provides a step-by-step guide for combining IPv6-enabled EKS clusters with your existing Istio Service Mesh and IPv4 workloads, enabling a graceful transition to IPv6 on AWS. This guide covers detailed instructions for enabling communication between IPv6 and IPv4 EKS clusters, along with recommended practices for implementing IPv6 across both single and multiple VPC configurations.

Amazon EKS IPv6 interoperability with IPv4 in Istio Service Mesh

The functionality of Amazon EKS IPv6 builds on the native dual-stack capabilities of VPC. When you enable IPv6 in your VPC, it receives both IPv4 prefixes and a /56 IPv6 prefix. This IPv6 prefix can come from three sources: Amazon’s Global Unicast Address (GUA) space, your own IPv6 range (BYOIPv6), or a Unique Local Address (ULA) space. Then, you can allocate /64 prefixes to your VPC subnets for dual-stack operations. For details on the steps to add IPv6 support to your existing VPCs, refer to the documentation.

In IPv6-enabled EKS clusters, the infrastructure operates in a hybrid mode. Although nodes support both protocols, pods default to IPv6-only addressing. The system allocates a /80 IPv6 prefix to each EKS node from its subnet range, which distributes IPv6 addresses to its pods. For smooth protocol interoperability, the Amazon EKS VPC CNI provides two key features: ENABLE_V6_EGRESS for IPv4 clusters (version 1.13.0 and newer) and ENABLE_V4_EGRESS for IPv6 clusters (version 1.15.1 and newer).

Istio is an open source service mesh that helps organizations connect, secure, and observe services in modern cloud-native applications. It provides essential capabilities for managing communication between services, making it an ideal choice for environments where IPv4 and IPv6 workloads need to coexist. Istio’s protocol-agnostic service discovery and traffic management features enable seamless communication between services, regardless of their underlying IP protocol. For detailed information about Istio’s architecture and capabilities, refer to the Istio documentation.

Throughout this post, we focus on practical implementation details and operational considerations when running IPv4 and IPv6 workloads together. We examine the configuration requirements for cross-protocol communication, explore how Istio handles service discovery across different IP protocols, and demonstrate working examples of applications communicating seamlessly across IPv4 and IPv6 boundaries. These insights help you make informed decisions as you plan your IPv6 adoption strategy.

Solution overview

We automated two Istio multi-cluster deployment models to streamline the transition between IPv4 and IPv6 networks. This automation eliminates the complexity typically associated with manual network configuration and reduces potential implementation errors.

Istio Multi-Primary Multicluster deployment model on a single network

In this deployment model, both IPv4 and IPv6 EKS clusters are deployed in a shared VPC, using Istio’s single network architecture, as shown in the following figure. Each cluster operates an independent Istiod control plane as primary.

  • Cross-cluster service discovery works through each cluster’s control plane (Istiod) maintaining a local registry of services by watching the API Servers of remote clusters for Service and Endpoint updates.
  • Services across clusters can communicate directly without needing an Istio gateway because they share the same network.

One challenge we faced and overcame was managing IP address family differences between network endpoints across clusters. For example, when an Istiod component in one cluster needs to communicate with a Kubernetes API Server endpoint in another cluster within the same network, one endpoint might use IPv4 while the other uses IPv6. To enable this cross-protocol communication, we configured the VPC CNI settings on both clusters:

  • For IPv4 EKS clusters: set ENABLE_V6_EGRESS flag to true to allow reaching IPv6 endpoints
  • For IPv6 EKS clusters: ENABLE_V4_EGRESS flag is by default set to true to allow reaching IPv4 endpoints
Figure 1: Istio Multi-Primary Multicluster deployment model on a single network

Figure 1: Istio Multi-Primary Multicluster deployment model on a single network

Istio Multi-Primary Multicluster deployment model on multi-network

In this deployment model, the IPv4 and IPv6 EKS clusters are deployed in separate VPCs, with each EKS cluster running its own Istiod control plane as primary.

  • Cross-cluster service discovery works through each cluster’s control plane (Istiod) maintaining a local registry of services by watching the API Servers of remote clusters for Service and Endpoint updates.
  • Services of one EKS cluster can communicate with services of the other EKS cluster through their eastwest gateway dedicated for inter-cluster communication.

The difference between this deployment model on multi-network over the single-network one is the use of dedicated eastwest gateways (as shown in the following figure) to handle the cross-cluster communication. The gateway endpoints and the Kubernetes API server endpoints need to be public for the other cluster to discover and connect. The eastwest gateways can be configured with a public facing network load balancer and with an ability to handle dualstack traffic so that we don’t have tweak the default behavior of the VPC CNI of either EKS cluster.

Figure 2: Istio Multi-Primary Multicluster deployment model on multi network

Figure 2: Istio Multi-Primary Multicluster deployment model on multi network

Walkthrough

The following sections walk you through this solution.

Initial setup

Before we proceed with deploying either pattern, you need the following components set up locally on your device:

The solution highlighted in this post can be found in this GitHub repository. We walk you through the overview of the deployment. When the solution has been deployed, we guide you on the ways to test the cross-cluster communication.

The code sample is for demonstration purpose only and should not be considered or used in production environments.

Both deployment models we discussed above follow a similar deployment workflow, except in the setup of network and/or gateways for each EKS cluster. For this post, we use the multi-cluster multi-primary on multi-network deployment model.

Infrastructure setup

The deployment begins with certificate management. First, we generate a root certificate and create two intermediate certificates. These intermediate certificates serve as dedicated Istio Certificate Authorities (CAs), one for each EKS cluster. For the network, we set up two distinct VPCs: a dual-stack VPC that supports both IPv4 and IPv6 for one cluster, and a single-stack IPv4 VPC for the other. Each VPC is configured with the necessary security groups to control network traffic and AWS Identity and Access Management (IAM) roles to manage permissions. Next, we deploy two EKS clusters with different IP families: one based on IPv4 and the other on IPv6. Both clusters are equipped with the AWS Load Balancer Controller to manage the provisioning of elastic load balancers.

Istio setup

When the infrastructure is in place, we install core Istio components (istiod and istio-ingressgateway) on both clusters. To enable cross-cluster communication, we configure remote reader secrets for authentication and service discovery and deploy eastwest gateways to facilitate communication between clusters across different VPCs.

Download the multi-cluster multi-primary Terraform pattern

Clone the repository, change the directory into multi-cluster-multi-primary as shown in the following example, and navigate to either multi-network or single-network folders based on the specific deployment model you want to implement.

git clone https://github.com/aws-samples/istio-on-eks.git

cd istio-on-eks/patterns/multi-cluster-multi-primary/

# cd single-network <-- uncomment for single-network deployment model
cd multi-network

Choose the IP version for each EKS cluster

Edit the locals.tf file and set the eks_1_IPv6 and eks_2_IPv6 variables to either true if you want the specific EKS cluster to be IPv6, or false for the specific EKS cluster to be based on IPv4. This file also allows you to customize the deployment further by modifying the following attributes:

  • AWS Region
  • VPC specific settings such as CIDR ranges
  • Amazon EKS specific settings such as name and cluster version
  • Istio specific settings such as cluster names, network names, and Istio version

We are highlighting the workload transition from IPv4 to IPv6 clusters, thus we choose the first EKS cluster to be based on IPv4 and the second EKS cluster to be based on IPv6. To reflect our election, update the locals.tf file and set eks_1_IPv6=false and eks_2_IPv6=true

These variables are independent of one another and do not need to be the same. This flexibility allows you to test various combinations of IPv4 and IPv6 networking across your clusters.

Run the deployment script

./scripts/deploy.sh

Set up kubeconfig for both EKS clusters

source scripts/set-cluster-contexts.sh

Verify your deployment

Review the two EKS clusters and the IP details of the nodes using the following commands.

echo "\n>> EKS Cluster 1 Nodes:"
kubectl get nodes --context="${CTX_CLUSTER1}" -o json | \
jq -r '.items[] | "\(.metadata.name) \(.status.addresses[0].address) \n"'

echo ">> EKS Cluster 2 Nodes:"
kubectl get nodes --context="${CTX_CLUSTER2}" -o json | \
jq -r '.items[] | "\(.metadata.name) \(.status.addresses[0].address)"'

Output:

You should see that first cluster (eks-1) is using IPv4 addressing while the other (eks-2) is using IPv6.

>> EKS Cluster 1 Nodes:
i-0e7859d009bcd7543 10.1.37.142

>> EKS Cluster 2 Nodes:
i-09cfd610cb29ffff4 2600:1f14:d68:105:ee30:1e17:b50b:2ed3

Check eastwest gateway network load balancer readiness (applied to multi-network deployment model only)

./scripts/check-lb-readiness.sh

You should see output similar to the following:

Updated context arn:aws:eks:us-west-2:XXXXXXXXXXXX:cluster/eks-1 in ~/.kube/config
Updated context arn:aws:eks:us-west-2:XXXXXXXXXXXX:cluster/eks-2 in ~/.kube/config

Readiness check for arn:aws:eks:us-west-2:XXXXXXXXXXXX:cluster/eks-1:
{
  "Target": {
    "Id": "10.1.44.114",
    "Port": 15443,
    "AvailabilityZone": "us-west-2c"
  },
  "HealthCheckPort": "15443",
  "TargetHealth": {
    "State": "healthy"
  },
  "AdministrativeOverride": {
    "State": "no_override",
    "Reason": "AdministrativeOverride.NoOverride",
    "Description": "No override is currently active on target"
  }
}

Readiness check for arn:aws:eks:us-west-2:XXXXXXXXXXXX:cluster/eks-2:
{
  "Target": {
    "Id": "2600:1f14:d68:105:ce02::2",
    "Port": 15443,
    "AvailabilityZone": "us-west-2c"
  },
  "HealthCheckPort": "15443",
  "TargetHealth": {
    "State": "healthy"
  },
  "AdministrativeOverride": {
    "State": "no_override",
    "Reason": "AdministrativeOverride.NoOverride",
    "Description": "No override is currently active on target"
  }
}

Test cross-cluster communication

We test communication between the clusters using a helloworld application. We deploy helloworld version v1 to cluster1 and version v2 to cluster2. The helloworld service responds to requests by including its version number, which allows us to identify which cluster processed the request. To test service-to-service communication within the mesh, we deploy a sleep application to both clusters. The sleep pods send HTTP requests to the helloworld service, which demonstrates cross-cluster service discovery and routing.

The helloworld and sleep components were deployed as part of overall deployment. Review the deployed components using the following commands:

kubectl get pods,svc -n sample --context=$CTX_CLUSTER1

kubectl get pods,svc -n sample --context=$CTX_CLUSTER2

You should see output similar to the following:

NAME                                 READY   STATUS    RESTARTS   AGE
pod/helloworld-v1-67b946b687-z9hrl   2/2     Running   0          41m
pod/sleep-868c754c4b-zgd59           2/2     Running   0          41m

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/helloworld   ClusterIP   172.20.150.81   <none>        5000/TCP   41m
service/sleep        ClusterIP   172.20.122.44   <none>        80/TCP     41m
NAME                                 READY   STATUS    RESTARTS   AGE
pod/helloworld-v2-7767c88b4c-f4xqz   2/2     Running   0          41m
pod/sleep-868c754c4b-t6zct           2/2     Running   0          41m

NAME                 TYPE        CLUSTER-IP            EXTERNAL-IP   PORT(S)    AGE
service/helloworld   ClusterIP   fd47:6d9:5c62::2a6f   <none>        5000/TCP   41m
service/sleep        ClusterIP   fd47:6d9:5c62::1a14   <none>        80/TCP     41m

Now that the sample apps are deployed, run the following command from each cluster to test the cross-cluster communication

From Cluster 1:

for i in {1..10}
do 
kubectl exec --context="${CTX_CLUSTER1}" -n sample -c sleep \
    "$(kubectl get pod --context="${CTX_CLUSTER1}" -n sample -l \
    app=sleep -o jsonpath='{.items[0].metadata.name}')" \
    -- curl -sS helloworld.sample:5000/hello
done

From Cluster 2:

for i in {1..10}
do 
kubectl exec --context="${CTX_CLUSTER2}" -n sample -c sleep \
    "$(kubectl get pod --context="${CTX_CLUSTER2}" -n sample -l \
    app=sleep -o jsonpath='{.items[0].metadata.name}')" \
    -- curl -sS helloworld.sample:5000/hello
done

The response from the helloworld service should have its version toggle between v1 and v2 similar to the following output:

Hello version: v2, instance: helloworld-v2-7767c88b4c-f4xqz
Hello version: v2, instance: helloworld-v2-7767c88b4c-f4xqz
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl
Hello version: v1, instance: helloworld-v1-67b946b687-z9hrl

When successful cross-cluster communication is established, you have now implemented a fully functional Istio multi-cluster environment. This configuration enables seamless service discovery and traffic routing between IPv4 and IPv6 workloads, whether they are deployed in the same network (VPC) or across multiple networks. The alternating responses from both clusters demonstrate that Istio correctly manages service discovery, load balancing, and traffic routing regardless of the underlying IP protocol. This approach provides a practical migration path that allows you to gradually adopt IPv6 for new workloads while maintaining compatibility with existing IPv4 services.

Conclusion

In this post, we demonstrated a practical approach to integrate IPv6-enabled Amazon EKS clusters with existing Istio Service Mesh and IPv4 EKS deployments. This allows us to streamline and accelerate IPv6 adoption for service-to-service communication on AWS. We showed two practical deployment models—single VPC and multi-VPC—that enable seamless communication between IPv4 and IPv6 EKS workloads. This approach allows you to gradually adopt IPv6 while maintaining operational stability and controlling costs. These deployment models allow you to scale beyond IPv4 limitations, streamline network management, and future-proof your infrastructure. To learn more about IPv6 adoption on AWS, refer to the AWS re:Post guide Get started with IPv6 on AWS and best practices guide on Running IPv6 EKS Clusters.

Special thanks to Vijay Chintalapati, Senior Container Specialist SA.

About the authors

Praseeda Sathaye is a Principal Product Manager – Technical for Edge Computing and Traffic Engineering at Amazon, based in the Bay Area in California. In her current role, she leads the product strategy and roadmap for edge computing solutions within Amazon’s Traffic Engineering organization, focusing on building innovative managed compute and networking solutions. Prior to this role, she served as a Principal Specialist for Containers and Open-source software (OSS) at Amazon Web Services, where she helped customers accelerate cloud-native adoption through modernizing their infrastructure and architecture using microservices, containerization, GitOps, Kubernetes, and service mesh. She has been recognized for her expertise in leveraging Generative AI on Amazon EKS to unlock the potential of cutting-edge technologies for developing AI-powered applications.

Alexandra Huides is a Principal Networking Specialist Solutions Architect for Strategic Accounts at Amazon Web Services. She focuses on helping customers build and develop networking architectures for highly scalable and resilient AWS environments. Alex is also a public speaker for AWS and is helping customers adopt IPv6. Outside work, she loves sailing, especially catamarans, traveling, discovering new cultures, and reading.

Goutham Annem is a Technical Account Manager at AWS, worked previously as Systems Reliability Engineer supporting customer’s infrastructure deployed on Cloud and OnPrem. He is sports enthusiast, with a particular fondness for badminton and cricket. Residing in the Bay Area, he frequently indulges in weekend hikes to connect with nature.