Gateways

Introduction

Enclave Gateway allows you to provide:

1. Safe access to on-premise devices that can't run Enclave, like printers and webcams
2. IP whitelisted and conditional access to SaaS platforms like Office 365 and Sharepoint
3. DNS filtering of Internet traffic to remove ads and help protect against threats like malware

1. Access to the LAN

Enclave is designed to be installed directly onto every client, server, cloud instance, virtual machine, container, mobile phone and workstation in your organisation. That way, Enclave can provide peer-to-peer connections between users and resources without VPN servers, while also applying Zero Trust Network Access controls at the edge to enforce policy and end-to-end encryption.

However, in some situations, you can’t or might not want to install Enclave on all devices or systems:

• On domain controllers where two or more network interfaces can be problematic
• On networks where the physical infrastructure is not allowed to be changed
• On embedded systems, like firewalls, webcams or printers which prohibit external software
• When accessing legacy systems which are too old to run, or are incompatible, with the agent
• When accessing cloud native services like AWS RDS, which don't run third party software
• With large numbers of devices in a single subnet, like a single AWS VPC

In these cases, you can set up an Enclave Gateway to provide access to devices and systems which don't, or can't, run Enclave.

Enclave Gateways allows you to route traffic from systems running Enclave to systems and devices not running Enclave (like RDS databases, webcams, printers and IoT sensors) in subnets the Enclave Gateway can reach.

2. Access to SaaS platforms

Enclave Gateway can also be used as a trusted pathway to reach common SaaS and cloud platforms like Office 356, Azure, AWS, Google and Salesforce with IP-based whitelisted access.

By routing all traffic for one or more selected SaaS platforms via an Enclave Gateway on a static IP address, SaaS platforms can be configured to restrict access to customer accounts by whitelisting access from a known static IP address. If used in conjunction with identity providers like Azure, Enclave Gateway can also be used in conjunction with conditional access polices to add additional security controls.

3. Access to the public Internet

Enclave Gateway can also be used to route all traffic out to the public Internet through fixed network locations, enabling customers to ensure network traffic egresses via points of presence under their regulatory control before it reaches the public Internet.

This can be useful in a number of scenarios: remote users working in other geographies or forcing the use of a single static public IP address for all users, for example.

Enclave Gateway optionally includes the open source PiHole project, making it simple to deploy DNS filtering that removes ads, bolsters protection against malware and other online threats and stops users from viewing inappropriate or undesirable content, such as streaming or adult websites.

Deployment options

1. Self-hosted Gateway

Customers can enable any enrolled Linux or WSL system running Enclave to act as an Enclave Gateway. The rest of this guide describes how customers can deploy self-hosted Enclave Gateways. Be sure to familiarise yourself with the prerequisites and limitations of self-hosting Enclave Gateway.

2. Managed Gateway

Customers may optionally purchase our fully managed Enclave Gateway service, enabling them to route all Internet traffic through hardware we manage on their behalf and deploy DNS filtering without having to manage or maintain the gateway themselves. Contact sales@enclave.io for more information about adding a managed Enclave Gateway to your account.

Prerequisites

Before you begin this guide, you’ll need an Enclave account set up with at least two devices enrolled. Read our getting started guide if you need help with this.

Limitations

In order to enable an enrolled system to act as an Enclave Gateway, Enclave must be installed onto a Linux operating system.

An Enclave Gateway is just a normal installation of Enclave which has been set as "Allowed to act as a Gateway" in the Portal. The following steps will show you how to configure an Enclave Gateway and configure Policies which will allow other Enclave systems to use it.

Step 1: Install and enrol

To get started, download and install Enclave onto the System which you want to use as an Enclave Gateway, provide an enrolment key and join it to your account. We support a variety of Linux distros.

Step 2: Enable the gateway

Once enrolled, you’ll need to "Allow the system to act as a Gateway" in the Portal. To do this, log into the Enclave Portal and open the Systems page. Find your enrolled system in the list and click on it to open the details pane.

Next, edit the system using the edit pen icon (), find the “Allow this system to act as a gateway” section and switch the toggle to enable this system to act as a Gateway.

Once enabled, Enclave will automatically configure that system to act as a gateway.

Info

When a system is enabled to act as a gateway the Enclave agent running on that system will automatically enable IP forwarding in the Linux kernel and create an iptables chain named ENCLAVE to hold the relevant source NAT rules.

Step 3: Configure subnets

An Enclave Gateway can act as a route to one or more subnets for other Enclave peers. It might provide access to a single subnet, for example to its local 192.168.1.0/24 network, or the Enclave Gateway may be running in AWS or Azure and provide access to a number of subnets. You may even decide to route all traffic for the entire Internet via your Enclave Gateway by configuring it to advertise a default route (0.0.0.0/0).

Enclave needs to know which subnets can be accessed via the new gateway. For convenience, Enclave will auto-discover all local ethernet or wireless network interfaces on the new gateway with IPv4 unicast addresses and automatically show them in the Portal (where the adapters are up and their assigned IP addresses are not broadcast, multicast or auto-configured addresses).

In the example below, Enclave has discovered there is a network interface with an IP address in the subnet 172.19.64.0/20 on this system. That means this system can be used as a gateway to access other devices not running Enclave in that subnet.

Against each subnet, you can optionally specify a subnet name. This name will be displayed when using the subnet in a Gateway Access Policy, making it easier to understand which subnet is being referred to.

It’s important to note that on this screen the administrator is defining all of the possible subnets available via this gateway, but not actually determining which systems can (or cannot) route traffic to those subnets. Controlling which Systems can reach which subnets behind specific Enclave Gateways is done later by creating one or more Gateway Access Policies. Use the () button to remove any unwanted subnets from the gateway's configuration.

In some scenarios, you may want an Enclave Gateway to forward traffic towards a subnet it can reach but is not directly connected to (like 0.0.0.0/0). In these situations, use the "Add subnet" link to manually define additional subnets that this gateway is capable of routing traffic into.

Subnet Priority

When adding a subnet to a gateway, you can choose whether the gateway is a preferred route to the given subnet. For example, if you have two gateways that can both reach a given subnet (perhaps over an underlying global network), but one of the gateways is known to be a more efficient route to the specified subnet, you can indicate it has a Preferred priority level:

By default, gateway policies will use 'preferred' gateways first, and then fall back to other gateways if no preferred gateways is unavailable. Gateway selection can be further controlled by using the Gateway Priority setting when creating a Gateway Access Policy.

Step 4: Create an access policy

Now you have one or more configured Enclave Gateways, create one or more Gateway Access Policies to determine which of your enrolled systems can route traffic to which subnets.

Gateway Access Policies use Tags in exactly the same way as Direct Access Policies, but only on the sender side of the policy. Each policy enables connectivity between tagged systems and destination subnets via one or more selected gateways. Traffic can flow from Senders to destination IP addresses (subject to policy-based access controls), but devices on the routed subnets cannot initiate connectivity back to the sender systems.

In practice, this means that if you use Enclave together with Enclave Gateways to route traffic to webcams or printers (for example), any tagged Enclave systems will be able to send traffic to those devices, but those devices won’t be able to send unsolicited traffic back to the Systems inside your Enclave network.

To create a Gateway Access Policy select one or more Tags on the Senders side of the policy and select one or more subnets on the Gateway / Subnet side of the policy.

Once a policy is created, Enclave automatically configures the routing table for appropriate sender systems according to Tag membership and Trust Requirements.

Tip

Subnets can be large. In some situations, you may not want to provide access to an entire subnet and instead prefer to allow partial access to a smaller set of IP addresses, devices or systems in each subnet. In such cases, see the Subnet Filtering section below.

Subnet filtering

With at least one subnet attached to a Gateway Access Policy it is possible to enable and configure Subnet Filtering on that policy.

Subnet Filtering allows administrators to avoid providing full access to entire subnets and instead restrict Senders access to a limited set of IP addresses.

For example, you may have an Enclave Gateway which provides access to 10.0.0.0/24 but only want to allow systems on the Senders side of the policy to access two servers in that subnet: 10.0.0.8 and 10.0.0.9. By adding a Subnet Filter to the policy you can restrict access to the wider subnet and provide only the required access.

In the example below the policy grants the users tag access to the 172.16.1.0/24 subnet advertised by the Enclave Gateway named Menlo Park Office. Having then enabled subnet filtering, a rule has been added which restricts Senders access to only two IP addresses, 172.16.1.10 to 172.16.1.11, two domain controllers in the Menlo Park Office.

Gateways as the default route

Enclave Gateways can also route all non-Enclave traffic through specific systems on your network.

Normally Enclave doesn’t interact with traffic destined for the public Internet, instead defaulting to a split-tunnel overlay network and only routing traffic between systems running Enclave.

However there may be times when you do want Enclave to route your public Internet traffic. For example, to ensure a predictable and static IP address is used to access to trusted SaaS services such as Office 365 or Salesforce.

To setup an Enclave Gateway as a default route on your network you should:

1. Enrol a Linux system to act as your Enclave Gateway

2. In the Portal, allow that enrolled system to act as an Gateway

3. Manually add the subnet 0.0.0.0/0 to that system in the Portal

4. Create a Gateway Access Policy which includes the Gateway system

Once setup, all public Internet traffic for systems on the Senders side of the policy will be routed via that Enclave Gateway.

You can also opt to block a subset of this traffic by modifying the Gateway Access Policy's Access Control Rules. For example, you could choose to allow all traffic other than SSH.

Gateway Priority, failover and redundancy

It’s possible to have several Enclave Gateways all advertising the same logical subnets to the same set of Senders. In such cases Enclave will automatically pick one of the available Enclave Gateways for each logical subnet.

Which gateway wins is determined by the Gateway Priority setting in each Gateway policy. We have three Gateway Priority settings; Balanced, Ordered and Geographic.

Balanced

With a Balanced priority, senders will be distributed amongst the available configured gateways on the policy. If one or more gateways are the preferred route to a subnet, they will be used first before balancing to other gateways.

For example, an administrator might allow two systems in the Menlo Park Office to act as Gateways which both advertise access to the subnet 192.168.1.0/24.

With this gateway priority setting, clients connected via the policy will randomly choose one of the available gateways to move traffic through.

If a sender system in the policy loses its connection to one of the Enclave Gateways in the Menlo Park office, connectivity will automatically failover to the alternative Enclave Gateway.

In this way, by running multiple Enclave Gateways to provide access to the same group of logical subnets, Enclave will automatically enable multi-path failover and increase redundancy.

If one or more of the gateways are marked as a preferred route to a subnet, they will be prioritised, with traffic only failing over to other gateways when none of the preferred gateways are available.

With this priority mode, if a preferred gateway comes back online, traffic will automatically switch back to flowing through the preferred gateway.

Ordered

With an Ordered gateway priority setting, gateways are explicitly consumed in the order they appear in the receivers list. If a gateway is unavailable, the next available gateway will be used, in an Active/Passive configuration. When a higher-priority gateway becomes available, traffic will switch to it.

You can re-order gateways in the receivers list using the arrows displayed when hovering over a gateway.

An Ordered policy ignores any gateway subnet-level preference settings, so can be useful if you want to ensure that traffic is always routed through a specific gateway unless it is unavailable, regardless of the subnet.

If a gateway higher in the ordered list comes online again after being unavailable, traffic will automatically switch back to flowing through the higher-priority gateway.

Geographic

In this mode, gateways are selected based on approximate distance from sender to gateway, based on the sender and gateway's public IP address. Use this if you want users to connect via the gateway closest to them. Preferred gateways are used to choose between gateways that are less than ~10km apart.

A common use-case for this mode is when you want to route all internet access (i.e. 0.0.0.0/0) for your global users back through your own infrastructure. By using geographic priority, you can ensure that users are routed to the closest gateway to them, reducing latency and improving performance.

If the distance to multiple gateways on the policy differs by less than ~10km (~6 miles), for example if the gateways are in the same cloud datacenter, the policy will randomly route clients between these gateways, as with a Balanced priority, including considering subnet priority.

If a closer gateway comes online, senders will automatically switch to using it.

Gateway Resource Requirements

Gateways are devices running any of our supported Linux distributions, with the Enclave Agent installed. This could be a VM, a Raspberry Pi, a mini-PC or any other system that can run Linux.

Tip

If you cannot deploy a Linux VM or Linux-compatible device, Enclave can run as a gateway inside WSL2 on a Windows Server.

WSL2 runs as a form of VM within Windows, so the server in question must have virtualisation enabled.

If running the host server itself as a VM, nested virtualisation must be available on your hypervisor. If running this VM in Azure, check the Azure Compute Unit documentation for details on which VM sizes support nested virtualisation.

The resource requirements of an Enclave Gateway are highly dependent on traffic volumes, number of users and execution environment. It's very hard for us to prescribe exactly the minimum requirements of your Enclave Gateway (not least because Enclave can run in very low-resource environments if required), so we highly recommend trialling your use-case on a small VM, and increase the size of the VM used for the gateway once you notice continuous high CPU usage.

However, there are a few rules that you can use when guiding your thinking about how much resource your gateway may need.

• The amount of network traffic moving through the gateway has a much bigger impact on the resources required than the number of users; you may find that 500 users connecting to (for example) an Active Directory server and a file share for intermittent use will run on a single vCPU and 1GB of RAM, whereas those same users all continuously streaming HD video 24/7 through the gateway may need 4 load-balanced gateways with 4vCPUs but still 1GB of RAM each.

• When deploying multiple Enclave Gateways for load-balancing/performance, connections are split randomly between the deployed gateways on a per-client basis. That is, all the traffic for a single client device will flow through a single gateway. Use the formula (number_of_users / number_of_gateways) * average_bandwidth_per_user to calculate the average bandwidth each gateway will have to handle.

• An insufficiently-resourced or overloaded gateway will likely exhibit traffic slowdown / throttling evenly across all the users connected to it, coupled with high CPU usage on the gateway.

• In a high-availability scenario, if a single gateway falls over, all traffic to that gateway will automatically switch over to one of the remaining gateways available. Consider whether you want each gateway to be able to comfortably handle the entirety of the total bandwidth in each case for an uninterrupted user experience (in which case each gateway will need spare resource sufficient to handle the failure of another gateway), or whether temporarily reduced overall performance is acceptable until the failed gateway is recovered.

Feel free to get in touch on slack if you want to discuss resource requirements for your usecase in more detail.

Trust requirements

Gateway Access Policies obey trust requirements and ACLs the same was as a regular Direct Access Policies, so you can easily apply Zero Trust requirements to the Senders connecting into your subnets, like multi-factor authentication.

DNS name resolution

In many circumstances, you may prefer to use DNS names to access devices behind an Enclave Gateway rather than the local IP address of those devices. In most situations, devices on the LAN will already have a hostname, and one or more DNS names assigned and we want to ensure parties reaching the LAN via the Enclave Gateway are able to resolve device IP addresses using the existing DNS names.

Consider a developer working from home who needs access to the company database server in the office. The office network is 10.0.0.0/24 and the database server on that network has the IP address 10.0.0.30. The database server's hostname is dbserver1 and it's joined to a domain called corp.net, so it's fully qualified domain name is dbserver1.corp.net.

To provide access via a Gateway, there must be an Enclave system in the office network that is "Allowed to act as a Gateway" (set by an Administrator in the Enclave Portal). Once enabled, and with a Gateway Access Policy has in place which shares access to the 10.0.0.0/24 subnet (or a part of it) to the developer working from home, Enclave will be able to resolve dbserver1.corp.net on the developer's system without any further configuration.

This is possible because Enclave runs a local stub resolver which provides name resolution services to all enrolled systems. When the local stub resolver on the developer's laptop doesn't know how to resolve the name dbserver1.corp.net, the operating system will fall back to asking it's other configured DNS resolvers, but when a system is connect to an Enclave Gateway it will automatically forward that DNS query to the Gateway.

If the Gateway can successfully resolve a forwarded DNS query into an IP address, it will do and send the answer back. In this example, the developer's system may attempt to resolve dbserver1.corp.net to an IP address. The local Enclave stub resolver on the developer's laptop won't know the answer, but as the developer's system is connected to an Enclave Gateway the local stub resolver will forward the query to see if the Gateway can opportunistically resolve it.

As the Enclave Gateway is on the same subnet in the office as the fileserver, it's likely that the Enclave Gateway itself will be able to resolve dbserver1.corp.net into a local IP address, 10.0.0.30. That answer is sent back to the developer's laptop, and since the developers laptop is configured to access the 10.0.0.0/24 subnet via the same gateway, the access is automatic and seamless.

Security behaviour

An Enclave Gateway will only provide an answer for a given DNS query if the Gateway Access Policy that allowed it to be forwarded in the first place allows the querying party to reach the returned IP addresses.

So, for example if a DNS query is forwarded to an Enclave Gateway, and the query yields an answer of 10.0.0.30 and 192.168.0.30, but Gateway Access Policy only allows 10.0.0.1 to be reached, then the returned result will NOT include 192.168.0.30 so as to prevent information disclosure from the Gateway about systems in adjacent networks.

A Gateway may attempt to resolve forwarded DNS queries in one of two ways:

1. Using static or DHCP assigned nameservers to query the forwarded name verbatim.

2. Using the /etc/hosts file in to resolve the forwarded name

   10.0.0.30 dbserver1
   10.0.0.30 dbserver1.corp.net
   

DNS and the .local domain

If you are deploying a gateway to reach a domain with a .local suffix (for example, if you are using Active Directory as your DNS server, and have defined your corporate domain as ending with .local), then DNS resolution through the Enclave Gateway may fail.

If you connect to the gateway with SSH, and attempt to ping a workstation in the network, you may see this error:

enclave-admin@gateway1:~$ ping workstation1.mydomain.local
ping: workstation1.mydomain.local: Temporary failure in name resolution

The cause of this issue is that by default a number of Linux distributions treat the .local suffix as special, reserved for mDNS, and will not resolve queries to it.

In order to resolve this you must add the local domain as a search domain on the specific adapter (or adapters) that provide connectivity to your network. So, if you have an ethernet connection you may need to configure eth0 with that search domain.

How you configure the additional search domain will vary by OS version and distribution.

---

Example: Configure DNS search suffix using Netplan on Ubuntu Server

We provide here an example to configure Ubuntu 18.04 and later using Netplan. It's important to note that the when using Netplan, the local search domain should be applied to the interface, rather than as a global setting; otherwise local names will not resolve.

Tip

The following Netplan configuration example is for Ubuntu Server, do not use these instructions with desktop flavours of Linux such as Ubuntu Desktop, XUbuntu, or Linux Mint as the steps required are not the same.

Run the following command to create a file at /etc/netplan/60-local-domain.yaml with the appropriate content. If your network interface is not called eth0, replace it with the name of your network interface.

sudo tee /etc/netplan/60-local-domain.yaml > /dev/null <<EOF
# Configure an additional 'local' search domain
network:
    ethernets:
        eth0:
            match:
                name: eth0
            nameservers:
                search: ['local']
    version: 2
EOF

Running netplan try will apply the new configuration, waiting for user confirmation and backing out if none is given or the resulting changes destabilise the network.

sudo netplan try

Testing the Gateway

Check that you can ping your new Enclave Gateway from your Sender policy machine, using the Enclave Gateway's Enclave Virtual address (i.e. 100.64.69.156 in the example below). You can find the Enclave Virtual address of your Enclave Gateway by running enclave status on either system once connected by policy.

You should also notice in the output of the enclave status CLI command on the sender-side systems that any routes advertised via your Enclave Gateway are listed under the appropriate peer, as shown below under Gateway for for the Menlo Park Office peer.

Peer: WZL94 (Menlo Park Office)

   Peer state. . . . . : Up
   Certificate . . . . : CN=WZL94 Expires=Never (Perpetual Issue)
   Endpoint. . . . . . : Udp/10.1.10.115:40161
   Last activity . . . : 0.75 seconds ago
   Transfer. . . . . . : 3.462 KB received, 4.071 KB sent, link rtt 0.59 ms
   Virtual network . . : 100.64.0.0/10 (255.192.0.0)
   Virtual address . . : 100.64.0.70
   Gateway for . . . . : 172.26.0.0/20
   Dns . . . . . . . . : wzl94.id.enclave, ubuntu-dev.enclave
   ACLs. . . . . . . . : allow [icmp] from peer -> local, allow [any] from local -> peer

If you can ping the Enclave Gateway (i.e. ping 100.64.0.70 in this example) via Enclave and your local system is showing 172.26.0.0/20 (or your equivalent subnet) as the Gateway for in CLI output for the correct peer, you should be able to start sending traffic directly to non-Enclave devices that subnet.

Knowing that the Enclave Gateway itself is reachable, try sending some traffic past the Enclave Gateway to a device in the subnet behind it. If the any of the host-based firewalls on the target systems not running Enclave allow it, you may be able to send pings to test end-to-end connectivity.

In the example below, we've successfully sent a ping from a Windows laptop in a coffee shop, via the Enclave Gateway at 100.64.0.70 in the office, out to a printer with an IP address of 172.26.0.250 in the Gateway's local subnet.

C:\> ping 172.26.0.250

Pinging 172.26.0.250 with 32 bytes of data:
Reply from 172.26.0.250: bytes=32 time=29ms TTL=64
Reply from 172.26.0.250: bytes=32 time=30ms TTL=64
Reply from 172.26.0.250: bytes=32 time=29ms TTL=64
Reply from 172.26.0.250: bytes=32 time=38ms TTL=64

Ping statistics for 172.26.0.250:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 29ms, Maximum = 38ms, Average = 31ms

Troubleshooting

If you find an Enclave Gateway isn't working as expected, please see the Gateway section of our troubleshooting guide.

---

Having problems? Contact us at support@enclave.io or get help and advice in our community support channels.

Last updated Nov 21, 2022