Redirecting internet bound traffic through a transparent forward proxy

Centralized egress is the principle of using a single, common inspection point for all network traffic destined for the internet. This approach is beneficial from a security perspective because it limits exposure to externally accessible malicious resources, such as malware command and control (C&C) infrastructure. This inspection is generally done by a firewall like AWS Network Firewall and customers often also want to insert a forward proxy in the path with or without firewalls. Proxies can work in two modes: explicit proxy mode, where all clients needing access to the internet are configured with explicit proxy and send supported outbound traffic using explicit proxy configuration. One such implementation is described in the post, How to set up an outbound VPC proxy with domain whitelisting and content filtering. However, there are systems that cannot be configured with explicit proxy, and customers often look for a way to transparently redirect traffic from such systems going toward the internet through a proxy. In this post I explain an architecture that can be used to deploy a proxy in internet egress path and redirect traffic to it transparently using Web Cache Communication Protocol (WCCP). I’ll walk through a conceptual overview of implementing transparent proxies with WCCP in an AWS Transit Gateway based architecture. Specific implementation details may vary based on your organization’s requirements and chosen technologies.

Real-world use cases

The following are three real-world use cases.

1. Secure internet access: Organizations can use this architecture to make sure that all outbound internet traffic passes through security controls without needing explicit proxy configuration on each client device.
2. Data loss prevention: Inspecting traffic transparently allows companies to identify and block sensitive data leaving the network, even when the endpoints do not support explicit proxy configuration.
3. Regulatory compliance: Industries with strict compliance requirements can implement consistent inspection and logging of all network traffic to meet regulatory standards.

Prerequisites:

Before proceeding, you should be familiar with the following AWS networking concepts and services:

• Amazon Virtual Private Clouds (Amazon VPCs)
• Route tables
• AWS Transit Gateway
• Transit Gateway Connect attachments
• AWS Network Firewall
• NAT Gateway
• Third party virtual appliances (WCCP-compliant routers and proxy servers) WCCP functionality.

Architecture overview

The architecture centers around an Egress VPC containing WCCP-compliant virtual routers and proxy servers running on Amazon Elastic Compute Cloud (Amazon EC2). These virtual routers redirects traffic to proxy using WCCP . This setup allows for transparent inspection and filtering of traffic without needing client-side configuration changes. Figure 1 shows the diagram, followed by the details of the architecture.

Figure 1: Overall architecture diagram

Key components

Transit Gateway:  Transit Gateway serves as the central hub for network connectivity, allowing for simplified management of connections between VPCs and on-premises networks. In our architecture, Transit Gateway routes traffic between multiple Spoke VPCs, as wells as between Spoke VPCs and the internet through an Egress VPC for inspection.

Transit Gateway Connect attachments: Connect attachments provides the capability to integrate third-party virtual appliances with the Transit Gateway. In our design, these attachments:

• Link the Transit Gateway to the WCCP-compliant routers through Generic routing encapsulation (GRE) over the Connect attachment
• Enable traffic redirection without modifying routing tables in each VPC
• Provide flexibility for scaling and high availability

You can refer to the post, Simplify SD-WAN connectivity with AWS Transit Gateway Connect, as a foundation for implementing Transit Gateway with Connect attachments. Then, you can modify the implementation to align with the requirements outlined in this post.

Egress VPC: The Egress VPC is a specialized environment that:

• Houses WCCP-compliant routers and proxy servers on Amazon EC2 in private subnets
• Contains firewalls in separate private subnets
• Implements centralized security policies using proxies and firewalls
• Uses NAT Gateways in public subnets for internet connectivity through Internet Gateway (IGW)

WCCP-compliant routers running on Amazon EC2: These routers are the cornerstone of our transparent proxy implementation. They do the following:

• Sit inline on the path between traffic from Spoke VPCs to the internet
• Intercept traffic based on configured policies
• Redirect specific traffic to proxy servers
• Return processed traffic to its original path

Proxy servers running on Amazon EC2: These are Amazon EC2 based virtual appliances functioning as forward proxy (for example Squid) that interact with and receive redirected traffic from the WCCP router appliance mentioned previously. Using WCCP, they enforce configured policies for all egress traffic and forward it to the next security function in the path. In this example, we are using AWS Network firewalls . However, these proxies can daisy chain other security/inspection systems, such as Data Loss Protection (DLP) systems, in the path before the traffic is routed to the internet.

Connectivity

As shown in the preceding Figure 1, Spoke VPCs are connected to the Transit Gateway using VPC attachments. These attachments are associated with a Spoke VPC Route Table (routing will be explained in the next section). The Egress VPC connects to the same Transit Gateway using VPC attachment, and a Connect attachment is provisioned between the Transit Gateway and Egress VPC over this VPC attachment. This Connect attachment is used for provisioning GRE tunnels between the Transit Gateway and the virtual appliance in the Egress VPC, which serves as a WCCP router. This Egress VPC has connectivity to the internet through an attached IGW.

Routing (refer to Figure 1)

• Spoke VPCs: Each Spoke VPC needing internet access has subnets associated with a route table containing a default route pointing to its attached Transit Gateway. This configuration routes all internet-bound traffic to the Transit Gateway, where the Spoke VPC attachment is associated with the Transit Gateway Spoke Route Table.
• Transit Gateway Configuration:
  • The Transit Gateway Spoke Route Table contains propagated routes for all Spoke VPCs and a default route propagated from the Connect attachment. The default route is generated by the WCCP virtual appliance in Egress VPC and is advertised to the Transit Gateway through BGP peering over the GRE tunnel.
  • On the Transit Gateway, the Egress VPC attachment is associated with the Egress Route Table. The table contains the propagated route to the Egress VPC CIDR from the Egress VPC attachment, as well as propagated routes to all Spoke VPC CIDRs from their respective attachments.
  • The Connect attachment is associated with its own route table. This route table contains propagated Spoke VPC routes as well as the default route advertised by the WCCP routers over BGP session through the GRE tunnel.
• Egress VPC Configuration: The Egress VPC needs more routing configuration to enable transparent proxy functionality.
  • WCCP router appliances in private subnets establish GRE tunnels with the Transit Gateway and create BGP peering sessions.
  • Through these BGP sessions, WCCP router appliances advertise a default route to the Transit Gateway and receive all Spoke VPC prefixes.
  • The WCCP router appliance redirects internet bound traffic to forward proxies in the same subnet using WCCP (detailed discussion follows later in this post).
• Subnet-specific Routing: private subnets (WCCP Router and Proxy servers).
  • Route table entries for Transit Gateway CIDR connect attachment peers pointing to Transit Gateway.
  • Default route pointing to firewalls in a separate private subnet.
• Private subnets (firewalls):
  • Default route pointing to NAT Gateways in the corresponding AWS Availability Zone’s (AZ’s) public subnet.
  • Routes to WCCP router ENI in the same AZ for return traffic not matching WCCP redirection rules.
• Public Subnets (NAT Gateways):
  • Default route pointing to IGW.
  • Routes to firewall endpoint in the same AZ for return traffic not matching WCCP redirection rules.

WCCP redirection

All internet-bound traffic arriving on WCCP routers is evaluated against the configured WCCP redirection rules and forwarded to proxies in the same subnets. For redundancy, these WCCP routers have WCCP redirection configured to connect with proxies in other AZs. WCCP redirection can be configured to handle either all supported traffic or chosen traffic types. The configuration mechanism varies depending on the vendor of the WCCP virtual appliance. Traffic arriving at the proxies is evaluated and processed according to the configuration and then forwarded to the firewall while it’s on the way to the internet. As previously mentioned, proxies can be chained with other functions, such as DLP, if needed.

Packet walkthrough

The following figure outlines the packet walkthrough.

Figure 2: Architecture diagram with packet flow

Forward path: This section explains how the packet traverses from resources in a Spoke VPC to destination on Internet via the proxies and firewalls in the Egress VPC.

• (A) Traffic from sources in Spoke VPCs toward the internet follow the subnet route table’s default route and enter the Transit Gateway.
• (B) When it is in the Transit Gateway, it follows the route in the Spoke Route Table on Transit Gateway, which forwards the traffic to the Transit Gateway Connect attachment. There are multiple GRE tunnels over the Connect attachment, all advertising default routes to Transit Gateway, thus the Transit Gateway uses Equal Cost Multi-Pathing to send traffic to the WCCP virtual appliance in Egress VPC.
• (C) The WCCP-virtual appliance intercepts the traffic and determines if it needs inspection. Traffic needing inspection is redirected to proxy servers, which apply security policies, content filtering, or other controls before forwarding it to Network Firewalls.
• (CD) Traffic not matching the WCCP redirection rules exit out of the WCCP virtual appliance and go directly to Network firewall, following the subnet route table.
• (D) Proxied traffic is Source NATed to its IP address and exits the proxy server toward Network Firewall following the subnet route table. This has a default route pointing to Network Firewall endpoint in the same AZ of Egress VPC.
• (E) Network Firewall inspects the traffic and, if allowed, sends it to NAT Gateway in the same AZ of Egress VPC following the default route configured in its subnet route table.
• (F) NAT Gateway performs Source NAT on the traffic and sends it to the destination on internet.

Reverse path: This section explains how the packet for return traffic traverses from Internet to the resources in a Spoke VPC via the proxies and firewalls in the Egress VPC.

• (G) Return traffic from the internet lands on the same NAT Gateway because of the Source NAT.
• (H) Following the subnet route table, NAT Gateway forwards the traffic to Network Firewall in the same AZ.
• (I) Network firewall sends the traffic to the proxy server in the same AZ, if this flow on its forward path matched the WCCP redirection rule and was Source NATed by the proxy server.
• (J) The proxy server sends the returned traffic of the proxied traffic flow to the WCCP router using the WCCP configuration on the proxy server.
• (IJ) Return traffic for the flow that did not match the redirection rule is sent to the WCCP router by Network firewall following the subnet route table.
• (K) There are multiple GRE tunnels from the WCCP router to Transit Gateway over the Connect attachment. Spoke VPC CIDRs are advertised to WCCP router via these tunnels. The WCCP router uses Equal Cost Multi-Pathing to send traffic to the Transit Gateway over these GRE tunnels.
• (L) Transit Gateway uses the Connect attachment route table to forward the traffic to corresponding Spoke VPC attachment.
• (M) In the spoke VPC traffic follows the local routes to arrive at the destination from the Transit Gateway ENI.

Considerations

Consider the following four points when undertaking this solution.

1. WCCP virtual appliance sizing
   • Make sure of proper sizing of the WCCP virtual appliance to handle the expected traffic volume. You can refer the documentation at Amazon EC2 instance network bandwidth to choose an appropriate EC2 instance that supports your traffic volume requirements.
   • Plan for redundancy by deploying across multiple AZs.
   • Component level redundancy (for WCCP router or Proxy servers) can be implemented using automatic instance recovery.
2. WCCP configuration
   • Define service groups to categorize traffic for different inspection policies as needed. Different service groups can be used for primary and secondary proxy servers.
   • Configure redirection according to your requirements. You can use multiple attributes such as ports, protocols, and source/destination addresses.
   • Implement authentication between WCCP routers and proxy servers (out of scope for this post).
3. Proxy selection
   • Choose proxy solutions that support WCCP integration.
   • Consider performance requirements based on expected traffic volume. You can refer to documentation at Amazon EC2 instance network bandwidth to choose an EC2 instance that can support your traffic volume.
4. Monitoring and logging
   • Implement comprehensive logging for both WCCP routers and proxy servers. The following are some examples of logs to be collected. Given that WCCP and proxy implementation vary based on the vendor/solution chosen, these logs might be known using different names. Consult the vendor documentation for details:

On the WCCP router:

1. 1. 1. WCCP service status, packet redirection, and router-to-proxy communication logs
      2. Interface traffic statistics
      3. ACL hit counts for WCCP-related rules
      4. Netflow/sFlow data for traffic analysis

On the proxy server:

1. 1. 1. Access logs and Error logs (client IP, URL, status code, bytes transferred, timing, connection failures, timeouts)
      2. WCCP service logs (service registration, group membership)
      3. Cache performance logs (hit/miss ratio)
      4. Connection handling logs (TCP connections, handshakes)
      5. Authentication logs (if applicable)
      6. SSL/TLS inspection logs (for HTTPS traffic)
      7. System resource usage (CPU, memory, disk I/O)

• • Set up alerts for traffic anomalies or proxy health issues and create dashboards to visualize traffic patterns and security events. You can export the previously mentioned logs to Amazon CloudWatch to set alerts and create dashboards.

Conclusion

Implementing transparent proxies using WCCP with AWS Transit Gateway provides a powerful solution for organizations seeking to enhance their network security posture without disrupting user experience. Centralizing security controls in an egress VPC and using WCCP-compliant routers enables organizations to achieve comprehensive traffic inspection while maintaining network performance and scalability.

This architecture offers flexibility to adapt to changing security requirements and growing network demands, making it an excellent choice for enterprises looking to strengthen their cloud security infrastructure. For more information and architecture options for centralized egress, go to the AWS Whitepaper, Centralized egress to internet.

Vijay Menon Vijay Menon is a Principal Solutions Architect based out of Singapore with a background in large scale networks and communications infrastructure. He enjoys learning new technologies and helping customers solve complex technical problems by providing solutions using AWS products and services. When he is not helping customers, he likes to go on long runs and spend time with family and friends.