Enabling out-of-band management for third-party appliances in AWS Cloud WAN

Introduction

AWS Cloud WAN enables organizations to build and manage a global network across multiple AWS Regions. Through AWS Cloud WAN service insertion, you can integrate security appliances, either AWS-managed (such as AWS Network Firewall) or third-party solutions, to inspect and control traffic between network segments or outbound to the internet.

Although AWS Cloud WAN streamlines the process of inserting security appliances into the traffic path, it also presents a design challenge: these appliances aren’t directly accessible by default for administrative tasks such as software updates or configuration changes. To address this, many deployments need out-of-band (OOB) access for device management.

In this post, we explore different approaches to enabling direct OOB management for third-party security appliances deployed with AWS Cloud WAN, by separating the data plane from your management traffic.

Prerequisites

Before proceeding, you should be familiar with the following concepts:

• AWS Cloud WAN building blocks: core network, segment, network function group, and attachments
• AWS Cloud WAN service insertion
• AWS Gateway Load Balancer (GWLB) inspection architectures

AWS Cloud WAN and OOB management challenge

When deploying third-party security appliances in AWS, they are typically set up as Amazon Elastic Compute Cloud (Amazon EC2) instances in an inspection Amazon Virtual Private Cloud (Amazon VPC). Each appliance has a data plane Elastic Network Interface (ENI) that processes and inspects traffic based on administrator-configured security policies.

For management purposes, administrators use protocols such as SSH or HTTPS to connect to the appliance. This can be accomplished in two ways:

• Using the same ENI that handles data plane traffic.
• Using a separate management ENI dedicated to administrative access.

However, when the inspection VPC hosting the appliances is attached to a network function group (NFG) in an AWS Cloud WAN core network, its CIDR ranges aren’t automatically advertised anywhere. This means two things:

• Data plane traffic not directly destined to the inspection VPC can pass through the VPC for inspection (for example between AWS Cloud WAN segments or traffic to the internet).
• The appliance’s management interface, on the same or a separate ENI, isn’t automatically reachable from outside of the VPC, preventing direct administrative access.

Figure 1 shows an AWS Cloud WAN architecture with an inspection VPC attached to the NFG. Although the data plane flow functions correctly, the NFG attachment does not automatically grant direct access to the appliance’s management interface. This is because the inspection VPC routes are not present in any of the segments.

Figure 1: Data plane ENIs and management plane ENIs in the same VPC

Data plane traffic flow:

1.VPC A (10.1.0.0/16) sends traffic to VPC B (10.2.0.0/16) through AWS Cloud WAN.

2. Service insertion policy is enabled on AWS Cloud WAN to enforce inspection between Segments A and B by sending it to a designated inspection VPC.

• The only sources and destinations in the IP packets are VPC A and VPC B. The IP range of the Inspection VPC (172.16.0.0/16) is never used.

3. Inspection VPC hosts all GWLB infrastructure, such as the endpoint and the appliances. To understand the flow of traffic in the Inspection VPC, refer to the preceding prerequisites reference.

Management traffic flow:

4. VPC A (10.1.0.0/16) tries to connect to the management interface of one of the third-party appliances (172.16.0.0/16)

• traffic from the VPC can get to the AWS Cloud WAN core network, but segment A has no route to the Inspection VPC. The CIDR ranges of VPCs attached to an NFG aren’t automatically propagated anywhere.

The best way to solve this challenge is to separate the data plane from your management traffic. This approach not only enhances operational efficiency but also by hosting the appliances in a distinct AWS account, we can effectively isolate them from the data plane responsible for policy enforcement and inspection. The following three deployment approaches are determined by the appliance’s capabilities:

·      Approach 1: For appliances that support GWLB

·      Approach 2: For appliances that don’t support GWLB, using Multi-VPC ENI

·      Approach 3: Manually add static routes for the Inspection VPC ranges (included for completeness, but doesn’t separate data plane from management plane)

AWS strongly recommends using GWLB deployment when possible.

Approach 1: GWLB-supported appliance architecture

This approach implements a two-VPC design that separates service and management layers. This separation makes sure of automatic propagation of management routes in a dedicated segment, while maintaining efficient traffic inspection. Figure 2 shows this approach:

Inspection VPC:

• Attached to an AWS Cloud WAN NFG dedicated to traffic inspection
• Uses GWLB endpoints for data plane traffic steering

Firewall/Management VPC:

• Houses the GWLB and connects to the AWS Cloud WAN segment for administrative access to security appliances
• Hosts GWLB and security appliances, enabling direct administrative control for each one

Figure 2: Deployment OOB management using GWLB with AWS Cloud WAN and service insertion

Data plane traffic flow:

1. VPC A (10.1.0.0/16) sends traffic to VPC B (10.2.0.0/16) through AWS Cloud WAN.

2. Service insertion policy is enabled on AWS Cloud WAN to enforce inspection between Segments A and B by sending it to a designated inspection VPC

• The only sources and destinations in the IP packets are VPC A and VPC B. The IP ranges of the Inspection VPC (172.16.0.0/16) and Management VPC (192.168.0.0/16) aren’t used in the flow.

3. Inspection VPC only hosts the GWLB Endpoints that act as a data plane extension of the GWLB and the appliances hosted in the Management VPC.

4. GWLB Endpoint used AWS PrivateLink to forward data plane traffic to the GWLB.

• Data plane traffic from the GWLB to third-party appliance is inside of a GENEVE tunnel allowing it to not rely on the VPC routing.

Management traffic flow:

5. VPC A (10.1.0.0/16) tries to connect to the management interface of one of the third-party appliances (192.168.0.0/16).

6. Traffic from the VPC can get to the AWS Cloud WAN core network and segment A has a route to the management VPC.

7. Management traffic doesn’t go through the GWLB.

Segment A is used as an example, and the management VPC can be connected to any segment. For example, you could create a dedicated segment for out of band management.

Benefits

• Clear separation between inspection (data plane) and management access
• Enables role-based team management: Security team manages firewall VPC, and Networking team manages inspection VPC endpoints
• Management routes are automatically populated across all AWS Regions in the relevant segment (segment A in the example)

Bonus benefit: AWS Cloud WAN migration support

Another advantage of this model is its ability to streamline the migration to AWS Cloud WAN. This can be done by utilizing either a pure AWS Transit Gateway model or AWS Cloud WAN without service insertion.

The detailed migration process is documented in the post Migration to AWS Cloud WAN multi-Region inspection using service insertion, using AWS Network Firewall. The same migration steps documented in that post apply to third-party appliances. When using third-party solutions, maintain a single firewall deployment with endpoints distributed across inspection VPCs.

Figure 3 shows the interim state of a migration from AWS Transit Gateway to AWS Cloud WAN. The interim stage necessitates that inspection is available both on the AWS Transit Gateway and AWS Cloud WAN. GWLB makes it easy to keep the firewall stack in a single VPC but deploy endpoints to different inspection VPCs.

Figure 3: AWS Transit Gateway to AWS Cloud WAN migration with third-party appliances: interim state

The IP ranges of the Inspection VPCs can overlap because they aren’t used in the data plane forwarding. This means you can save IP space by using the same range in your inspection VPCs. GWLB and GWLB Endpoints track where traffic comes from to make sure it’s returned the same way.

Approach 2: appliance without GWLB support

For appliances that don’t support GWLB, Multi-VPC ENI attachments can be used as an alternative solution. This AWS feature allows you to do the following:

• Launch an EC2 instance with a primary ENI in one VPC
• Attach secondary ENIs from another VPC in the same AWS account

Figure 4 shows this approach:

• Inspection VPC (attached to AWS Cloud WAN NFG segment): hosts the appliance’s data plane ENI for traffic inspection
• Management VPC (attached to management segment): hosts the appliance’s management ENI, for direct administrative access

This architecture enhances security by separating administrative access through the management ENI while keeping data plane traffic isolated within the inspection VPC.

Figure 4: Deployment OOB management with no GWLB using Multi-VPC ENI

Data plane traffic flow:

1. VPC A (10.1.0.0/16) sends traffic to VPC B (10.2.0.0/16) through AWS Cloud WAN.

2. Service insertion policy is enabled on AWS Cloud WAN to enforce inspection between Segments A and B by sending it to a designated inspection VPC.

• The only sources and destinations in the IP packets are VPC A and VPC B. The IP ranges of the Inspection VPC (172.16.0.0/16) and Management VPC (192.168.0.0/16) aren’t used in the flow.

3. Inspection VPC only hosts the Data Plane ENIs for each firewall appliance.

Management traffic flow:

4.VPC A (10.1.0.0/16) tries to connect to the management interface of one of the third-party appliances (192.168.0.0/16).

5. Traffic from VPC can get to AWS Cloud WAN core network and segment A has a route to the management VPC.

We only show a single firewall appliance.

This approach is ideal for appliances that need direct ENI management but don’t support GWLB integration.

Approach 3: Manually add static routes for the Inspection VPC ranges

This is the least recommended approach due to its downsides. However, we include it for completeness.

If you want to use the same VPC for both data plane traffic and management while using service insertion with AWS Cloud WAN, then you can do so by manually adding the necessary static routes.

However, this necessitates adding routes for every inspection VPC that hosts third-party appliances. It also doesn’t provide a clear separation between data plane and management traffic.

Another downside is that it forces the management segment to also participate in the data plane traffic flow (AWS Cloud WAN service insertion policy). Without this, there would be no return path from the NFG to the management segment.

Figure 5 depicts how this setup looks.

Figure 5: Manual static routes

Data plane traffic flow:

1. VPC A (10.1.0.0/16) sends traffic to VPC B (10.2.0.0/16) through AWS Cloud WAN.

2. Service insertion policy is enabled on AWS Cloud WAN to enforce inspection between Segments A and B by sending it to a designated inspection VPC.

• The only sources and destinations in the IP packets are VPC A and VPC B. The IP range of the Inspection VPC (172.16.0.0/16) isn’t used in the flow.

3. Inspection VPC receives the traffic and performs policy enforcement.

Management traffic flow:

4. VPC A (10.1.0.0/16) tries to connect to the management interface of one of the third-party appliances (172.16.0.0/16).

5. Traffic from the VPC can get to AWS Cloud WAN core network and segment A has a static route to the inspection VPC that also hosts the management interfaces.

6. Return traffic only works if Segment A is also configured to use service insertion for its data plane flow. If Segment A isn’t configured in any service insertion policy, then there is no return path to it and no way to configure static routes in the NFG.

Conclusion

Network inspection is a fundamental component of cloud security architectures. This post outlines best practices for integrating third-party security appliances with AWS Cloud WAN, focusing on OOB management designs using either Gateway Load Balancer or Multi-VPC ENI. These solutions help make sure of the following:

• Secure management plane access
• Seamless data plane integration using GWLB or Multi-VPC ENI
• Improved operational efficiency while maintaining security posture

Implementing these architectures allows organizations to confidently deploy and manage third-party security appliances within AWS Cloud WAN environments.

To learn more about this solution and AWS Cloud WAN, contact your AWS account team.

About the authors