← All writing · Article · September 21, 2025 · 2 min read

Allowed Isn't Reachable: Debugging Hybrid Network Paths

When the firewall says traffic is allowed but nothing gets through, stop reading rules and check the path.

Networking
AWS
Transit Gateway
Incident response

The trap

When traffic between a cloud network and an on-premises network stops, the firewall is the obvious suspect. If its policy says the traffic is permitted, investigations tend to loop: re-read the rules, re-check the ports, ask the network team, repeat.

Firewall policy answers one question: is this traffic permitted? It says nothing about whether a working path exists. From the application's side, a missing route and a blocked port look identical: the connection times out either way.

Check the path in a fixed order

For hybrid connectivity through an AWS Transit Gateway, work from the control plane outward and don't skip ahead:

Attachment state. Is each attachment (VPC, VPN or Direct Connect) available? An attachment that is deleted, failed or stuck in pending makes every rule irrelevant.
Route table associations and propagations. An association decides which Transit Gateway route table an attachment uses. A propagation decides which route tables learn its routes. Both must be right, for both sides of the connection.
VPC subnet routes. Do the subnets' route tables send the on-premises ranges to the Transit Gateway?
Return routes. Does the on-premises side route the cloud ranges back? The outbound side can look perfectly healthy while replies have nowhere to go.

# 1. Attachment state
aws ec2 describe-transit-gateway-attachments \
  --filters Name=transit-gateway-id,Values=tgw-0123456789abcdef0 \
  --query 'TransitGatewayAttachments[].[TransitGatewayAttachmentId,ResourceType,State]' \
  --output table

# 2. Associations and propagations
aws ec2 get-transit-gateway-route-table-associations \
  --transit-gateway-route-table-id tgw-rtb-0123456789abcdef0
aws ec2 get-transit-gateway-route-table-propagations \
  --transit-gateway-route-table-id tgw-rtb-0123456789abcdef0

# 3. Subnet routes pointing at the Transit Gateway
aws ec2 describe-route-tables \
  --filters Name=vpc-id,Values=vpc-0123456789abcdef0 \
  --query 'RouteTables[].Routes[?TransitGatewayId]'

Identifiers are AWS documentation-style placeholders.

Let a tool find the broken hop

VPC Reachability Analyzer tests whether a path exists between two endpoints in AWS and names the component that blocks it. It's the fastest way to answer "allowed, but is it reachable?" for the AWS side. The on-premises side still needs its own checks.

Prevent the next one

Alert on changes to Transit Gateway attachments and route tables, for example from their CloudTrail events, so a broken path is found by monitoring rather than by users.
Keep attachments and routes in infrastructure-as-code with drift detection, so an unexpected deletion shows up as drift.
Write the check order down as a runbook, so the next investigation starts at the right layer.

Checklist

[ ] Attachment state checked before any rule is read
[ ] Associations and propagations verified in both directions
[ ] Subnet routes send the right ranges to the Transit Gateway
[ ] On-premises return routes confirmed
[ ] Reachability Analyzer run between the two endpoints
[ ] Alerts on attachment and route-table changes