aws-cdk: VPC: allow configuring NAT instances instead of gateways (and a 0 NAT gateways bug)

Hi all. @rix0rrr, I believe the 0 NAT gateway issue is not fixed.

If I do

    const vpc = new Vpc(this, 'Vpc', {
      cidr: '10.0.0.0/16',
      maxAzs: 1,
      natGateways: 0,
    });

I get the following CDK error:

If you do not want NAT gateways (natGateways=0), make sure you don’t configure any PRIVATE subnets in ‘subnetConfiguration’ (make them PUBLIC or ISOLATED instead)

And if I do

    const vpc = new Vpc(this, 'Vpc', {
      cidr: '10.0.0.0/16',
      maxAzs: 1,
      natGateways: 0,
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'Public',
          subnetType: SubnetType.PUBLIC,
        }
      ],
    });

I get the following CDK error:

There are no ‘Private’ subnet groups in this VPC. Available types: Public

What am I missing ?

This seems to be possible with a recent commit #4898 in version 1.16.0 thanks to rix0rrr.

new Vpc(this, `vpc`, {
  cidr: '10.40.0.0/16',
  maxAzs: 2,
  natGateways: 2,
  natGatewayProvider: NatProvider.instance({
    instanceType: InstanceType.of(InstanceClass.T3A, InstanceSize.NANO),
  }),
  gatewayEndpoints: {
    s3: { service: GatewayVpcEndpointAwsService.S3 },
  },
});

@Pwntus I think this might actually be a gap in the ScheduledFargateTask construct.

It’s lacking a subnetSelection: ec2.SubnetSelection property. if you had that, you could say subnetSelection: { subnetType: ec2.SubnetType.PUBLIC } when creating it (right now, it’s trying to bind to the private subnets of your VPC, which is the default behavior).

@0xdabbad00 @Pwntus @sallar

I’m pretty much in the same boat as you and I figured out how to get rid of the nat gateways. The trick is to put a cloudformation condition on the AWS::EC2::Route resource that references the gateway and force it to evaluate to false. You can do this when nat gateway count = 0. Here it is in Python syntax:

from aws_cdk.aws_ec2 import Vpc, CfnRoute
from aws_cdk.core import CfnCondition, Fn

vpc = Vpc(self, 'vpc', nat_gateways=0)
exclude_condition = CfnCondition(
    self, 'exclude-default-route-subnet', expression=Fn.condition_equals(True, False)
)
for subnet in vpc.private_subnets:
    for child in subnet.node.children:
        if type(child) == CfnRoute:
            route: CfnRoute = child
            route.cfn_options.condition = exclude_condition # key point here

@rix0rrr I think the above is a pretty reasonable workaround for this issue.

I have this same problem. I’m trying to deploy a CDK app to spin up an ECS on a nightly job and my biggest cost is going to be the NAT Gateway that I don’t need to be using.

Hi @0xdevalias

I tried following your example, but it doesn’t work in the newest version of the cdk clients. I did some changes to make the typing work as follows:

    const vpc = new VpcNetwork(stack, "MyApp", {
        natGateways: 0
    });

    const natSecurityGroup = new SecurityGroup(stack, "NATSecurityGroup", {
        vpc,
        groupName: "NATSecurityGroup",
        description: "NAT Instance Security Group",
        allowAllOutbound: true
    });

    natSecurityGroup.tags.setTag("Name", natSecurityGroup.groupName);
    natSecurityGroup.connections.allowFromAnyIPv4(new TcpAllPorts());

    const natInstance = new CfnInstance(stack, "NATInstance", {
        imageId: "ami-d03288a3",
        instanceType: new InstanceTypePair(
            InstanceClass.T2,
            InstanceSize.None
        ).toString(),
        subnetId: vpc.publicSubnets[0].subnetId,
        securityGroupIds: [natSecurityGroup.securityGroupId],
        sourceDestCheck: false, // Required for NAT
        keyName: "myapp-ssh"
    });

    natInstance.propertyOverrides.tags = [
        { key: "Name", value: `${natInstance.stackPath}` }
    ];

    vpc.privateSubnets.forEach(subnet => {
        const defaultRoute = subnet.node.findChild("DefaultRoute") as CfnRoute;
        defaultRoute.propertyOverrides.instanceId = natInstance.instanceId;
    });

This yields the following error:

MyAppInfrastructureStack failed: ValidationError:
Circular dependency between resources: [NATSecurityGroupDB004F3B, MyAppVPCProductionPrivateSubnet2DefaultRoute2D986C86, NATInstance, MyAppPrimaryDBInstance2DD5D4CDB, MyAppVPCProductionPrivateSubnet3DefaultRoute710E04C8, MyAppPrimaryDBInstance171D1C604, MyAppVPCProductionPrivateSubnet1DefaultRouteC69EC64E]
Circular dependency between resources: [NATSecurityGroupDB004F3B, MyAppVPCProductionPrivateSubnet2DefaultRoute2D986C86, NATInstance, MyAppPrimaryDBInstance2DD5D4CDB, MyAppVPCProductionPrivateSubnet3DefaultRoute710E04C8, MyAppPrimaryDBInstance171D1C604, MyAppVPCProductionPrivateSubnet1DefaultRouteC69EC64E]

Any suggestions to how should I remove the default NAT Gateways and instead use my own NAT instance here? The default configuration is too expensive for us.

We are trying to deploy a ECS stack (available via an ALB) which doesn’t have much use for so many NAT Gateways.

Thanks a lot

Based on @sallar’s snippet - this stack is working for me on AWS CDK 0.33. It creates a VPC with a t2.nano NAT instance without SSH access. I grabbed the instance ID from this page and looked up my corresponding AWS deployment region for the ‘HVM (NAT) EBS-Backed 64-bit’ instance.

import cdk = require("@aws-cdk/cdk");
import ec2 = require("@aws-cdk/aws-ec2");

export class VpcStack extends cdk.Stack {
  public readonly vpc: ec2.Vpc;

  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    this.vpc = new ec2.Vpc(this, "VPC", { natGateways: 0 });
    const natSecurityGroup = new ec2.SecurityGroup(this, "NATSecurityGroup", {
      vpc: this.vpc,
      groupName: "NATSecurityGroup",
      description: "NAT Instance Security Group",
      allowAllOutbound: true
    });

    natSecurityGroup.connections.allowFromAnyIPv4(new ec2.TcpAllPorts());

    const natInstance = new ec2.CfnInstance(this, "NATInstance", {
      imageId: "ami-00c1445796bc0a29f",
      instanceType: new ec2.InstanceTypePair(
        ec2.InstanceClass.T2,
        ec2.InstanceSize.Nano
      ).toString(),
      subnetId: this.vpc.publicSubnets[0].subnetId,
      securityGroupIds: [natSecurityGroup.securityGroupId],
      sourceDestCheck: false // Required for NAT
    });

    natInstance.addPropertyOverride("Name", natInstance.stackPath);

    this.vpc.privateSubnets.forEach(subnet => {
      const defaultRoute = subnet.node.findChild(
        "DefaultRoute"
      ) as ec2.CfnRoute;
      defaultRoute.addPropertyOverride("instanceId", natInstance.instanceId);
    });
  }
}

@0xdevalias – for the cost conscious finding this issue we should also mention cross AZ network fees. If you really want to be least cost and run a single NAT (instance or GW) you should really use only one AZ. Depending on your data usage and instance sizes the cost implications will shift as well.