Skip to content

Documentation / Guides / Kubernetes

Accessing a Kubernetes cluster through an Enclave Gateway

This guide describes the recommended pattern for providing remote users with routed access and DNS name resolution into a Kubernetes cluster using an Enclave Gateway. It covers architecture decisions, DNS resolution strategies, and the Enclave configuration needed to make Kubernetes services reachable by name.

Overview

When remote developers or administrators need to reach Kubernetes pods, services, or the control plane without exposing them to the public internet, an Enclave Gateway provides private, policy-controlled access with automatic DNS resolution.

The pattern places the Enclave Gateway on a dedicated VM in the same network as the cluster - not inside it. Connected clients receive routes to the cluster's network ranges and have their DNS queries forwarded to the Gateway, which resolves Kubernetes service and pod names on their behalf.

Architecture

Place the Enclave Gateway on a dedicated VM in the same VPC (or underlying network) as the Kubernetes cluster. The Gateway VM is enrolled in Enclave and enabled to act as a Gateway in the Portal. It advertises the cluster's routable network ranges so that connected clients can reach nodes, pods, and services through the encrypted Enclave tunnel.

Remote user ──Enclave tunnel──▶ Gateway VM ──VPC routing──▶ K8s nodes / pods / services
                                    │
                                    └──DNS query──▶ Cluster DNS (kube-dns / CoreDNS)

DNS queries from connected clients are forwarded to the Gateway automatically by Enclave's stub resolver. The Gateway resolves them against the cluster's DNS and returns answers, filtered by policy so that clients only receive addresses they are authorised to reach.

Why the Gateway sits outside the cluster

Stability. A standalone VM is not subject to pod eviction, node drain, or rolling updates. DNS resolution and routing remain available during cluster upgrades and maintenance windows.

Simplicity. No Kubernetes operator, custom resource definition, or privileged DaemonSet is required inside the cluster. Enclave configuration is limited to the Gateway VM and the Portal.

Security. The Gateway VM is the single point of entry. It does not require elevated pod capabilities (NET_ADMIN, SYS_ADMIN), and Enclave's policy-based DNS filtering ensures the Gateway only returns DNS answers for addresses the querying client is authorised to reach.

How other products handle this

Other overlay network products typically recommend deploying a subnet router or routing peer as a pod inside the cluster. While this avoids provisioning a separate VM, it introduces pod lifecycle concerns - routing and DNS resolution are interrupted when pods are evicted, drained, or rescheduled - and requires granting privileged capabilities to the routing pod. Placing the Gateway outside the cluster avoids these trade-offs.

Network prerequisites

The Gateway VM must be able to reach the cluster resources it will advertise. In most cloud-hosted, VPC-native Kubernetes distributions (GKE with alias IPs, EKS with the VPC CNI, AKS with Azure CNI), the following ranges are routable from other VMs in the same VPC:

Target Example CIDR Notes
Node network 10.60.0.0/20 Always VPC-routable
Pod network 10.60.0.0/24 per node VPC-routable with VPC-native networking
Control plane 172.21.8.2/32 Private endpoint, VPC-routable

Kubernetes Service IPs (ClusterIPs) are virtual addresses managed by kube-proxy on each node. They are not VPC-routable by default. See Making ClusterIPs reachable if clients need to access standard Kubernetes services by their ClusterIP.

The .local domain. Kubernetes uses cluster.local as its default DNS zone. Most Linux distributions reserve the .local suffix for mDNS (multicast DNS) and will not send .local queries to standard DNS resolvers. Because the Enclave Gateway runs Linux, you must configure the Gateway VM to resolve .local via standard DNS, or DNS resolution for Kubernetes names will silently fail. See The .local domain for the required configuration. The same workaround is needed on any Linux client machines that will resolve cluster.local names.

Making ClusterIPs reachable from the Gateway VM

When a remote user resolves a Kubernetes service name like my-app.default.svc.cluster.local, the cluster's DNS returns a ClusterIP (e.g. 10.52.12.7). The remote user's traffic to that address travels through the Enclave tunnel to the Gateway VM, which then needs to forward it into the cluster. The problem is that ClusterIPs are virtual addresses - they are not real IPs on any network interface. They only work on Kubernetes nodes, where kube-proxy intercepts packets destined for a ClusterIP and translates them to a real pod IP.

Because the Gateway VM is not a Kubernetes node, it has no kube-proxy rules and cannot forward ClusterIP traffic on its own. There are three ways to handle this:

  • Option A: Route ClusterIPs through a node is the simplest option and works with all existing Kubernetes services without changing their definitions. Start here unless you have a reason not to.
  • Option B: Internal LoadBalancer services give each service a stable, VPC-routable IP without any static routes on the Gateway VM, but require changing service definitions. Prefer this for production services that need dedicated, health-checked endpoints.
  • Option C: Headless services return pod IPs directly, bypassing ClusterIPs entirely. Useful for development access or stateful workloads where clients connect to specific pods, but pod IPs change when pods restart.

These approaches are per-service, not per-cluster, so you can mix them - for example, route ClusterIPs through a node for general access while exposing a handful of critical production services via Internal LoadBalancers.

Option A: Route ClusterIPs through a node

Add a static route on the Gateway VM that sends service CIDR traffic to one or more Kubernetes node IPs. Kube-proxy on those nodes translates the ClusterIP to a real pod IP and forwards the traffic:

sudo ip route add 10.52.0.0/17 via 10.60.0.10

This makes all ClusterIP services reachable through the Gateway without changing any Kubernetes service definitions. For resilience, point the route at multiple nodes or use an internal load balancer as the next hop.

Note

The static route above is added on the Gateway VM. Persist it across reboots by adding it to the Gateway VM's network configuration (e.g. Netplan on Ubuntu or a systemd-networkd route file).

Option B: Internal LoadBalancer services

For production services, create a Kubernetes Service of type: LoadBalancer with an internal annotation. The cloud provider assigns a stable, VPC-routable IP that the Gateway can reach directly:

apiVersion: v1
kind: Service
metadata:
  name: my-app
  annotations:
    # GKE
    networking.gke.io/load-balancer-type: "Internal"
    # EKS: service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
    # AKS: service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 8080

Option C: Headless services

Headless services (clusterIP: None) cause the cluster DNS to return individual pod IPs instead of a ClusterIP. Since pod IPs are VPC-routable in VPC-native clusters, the Gateway can forward traffic directly without additional routes:

apiVersion: v1
kind: Service
metadata:
  name: my-app-headless
spec:
  clusterIP: None
  selector:
    app: my-app
  ports:
  - port: 80

This is well-suited to development access or stateful workloads where clients connect to specific pods.

DNS resolution on the Gateway

The Enclave stub resolver on each client forwards DNS queries to connected Gateways automatically. The Gateway resolves those queries using its configured upstream DNS. For Kubernetes, the Gateway needs an upstream DNS server that is authoritative for the cluster.local zone.

Use one of the following approaches:

Option A: Internal Load Balancer in front of kube-dns

Create a Kubernetes Service that exposes the cluster's DNS (kube-dns / CoreDNS) on a VPC-routable IP. This requires no additional software on the Gateway VM:

apiVersion: v1
kind: Service
metadata:
  name: kube-dns-gateway
  namespace: kube-system
  annotations:
    # GKE
    networking.gke.io/load-balancer-type: "Internal"
    # EKS: service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
    # AKS: service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  selector:
    k8s-app: kube-dns
  ports:
  - name: dns-udp
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP

Apply the manifest and note the assigned IP:

kubectl apply -f kube-dns-gateway.yaml
kubectl get svc -n kube-system kube-dns-gateway \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

Then configure the Gateway VM to use it as its upstream DNS:

sudo enclave set-config gateway-dns-upstream-servers "LOAD_BALANCER_IP"
sudo enclave restart

The load balancer provides a stable IP with health checks against kube-dns pods, requiring no operational overhead on the Gateway VM itself.

Option B: Cloud provider DNS

Some managed Kubernetes services make the cluster.local zone resolvable from the VPC's default DNS resolver. When available, this is the simplest approach because the Gateway VM's OS DNS already resolves Kubernetes names with no additional configuration.

GKE clusters with Cloud DNS enabled expose the cluster.local zone through the VPC metadata DNS resolver. Leave gateway-dns-upstream-servers unset (so the Gateway uses its OS-configured DNS) or set it explicitly:

sudo enclave set-config gateway-dns-upstream-servers "169.254.169.254"
sudo enclave restart

EKS and AKS do not natively expose cluster.local through the VPC DNS resolver. Use Option A or Option C instead.

Option C: CoreDNS on the Gateway VM

Choose this option when no cloud-managed DNS integration is available (self-hosted or bare-metal clusters), when cluster policy prohibits exposing kube-dns outside the cluster network, or when you want the Gateway to be fully self-contained with no dependency on services running inside the cluster for DNS. It is the most operationally involved option - you are running and maintaining a DNS server on the Gateway VM - but it keeps the cluster's DNS surface completely closed.

CoreDNS is installed directly on the Gateway VM. Its kubernetes plugin connects to the Kubernetes API server using a kubeconfig file, authenticates as a Kubernetes service account with read-only permissions, and watches for changes to Services, Endpoints, Pods, and Namespaces. It builds DNS records from that API data in real time. It does not forward queries to kube-dns inside the cluster - it is its own authoritative DNS server for cluster.local, populated directly from the API. The steps below create the service account, grant it the minimum required permissions, and generate the kubeconfig that CoreDNS uses to authenticate.

  1. Install CoreDNS on the Gateway VM:

    COREDNS_VERSION=1.12.1
curl -sL "https://github.com/coredns/coredns/releases/download/v${COREDNS_VERSION}/coredns_${COREDNS_VERSION}_linux_amd64.tgz" \
  | sudo tar xz -C /usr/local/bin/

  2. Create RBAC and kubeconfig. CoreDNS needs read-only access to Services, Endpoints, Namespaces, and Pods:

    kubectl create serviceaccount coredns-gateway -n kube-system
kubectl create clusterrole coredns-gateway-reader --verb=list,watch --resource=services,endpoints,namespaces,pods
kubectl create clusterrolebinding coredns-gateway-reader --clusterrole=coredns-gateway-reader --serviceaccount=kube-system:coredns-gateway

    Generate a kubeconfig for the service account and copy it to /etc/coredns/kubeconfig on the Gateway VM.

  3. Create a Corefile at /etc/coredns/Corefile:

    cluster.local {
    kubernetes cluster.local {
        kubeconfig /etc/coredns/kubeconfig
    }
    cache 30
    errors
}

. {
    forward . /etc/resolv.conf
    cache 30
    errors
}

  4. Run CoreDNS as a systemd service and configure Enclave to use it:

    sudo enclave set-config gateway-dns-upstream-servers "127.0.0.1"
sudo enclave restart

Tip

CoreDNS resolves Service names to ClusterIPs by default. To return pod IPs instead (avoiding the need for a ClusterIP route), add endpoint_pod_names to the kubernetes block in the Corefile.

Enclave configuration

Enable the Gateway and add subnets

In the Enclave Portal, open the enrolled Gateway VM and enable "Allow this system to act as a Gateway". See Gateway deployment for detailed steps.

Add the cluster network ranges that clients should be able to reach. Enclave auto-discovers the Gateway VM's directly connected subnets; use "Add subnet" to include ranges reachable via VPC routing:

Subnet Name Purpose
10.60.0.0/20 K8s pod network Node and pod access
10.52.0.0/17 K8s services ClusterIP access (if using service CIDR routing)
172.21.8.2/32 K8s control plane kubectl / API server access

Create a Gateway Access Policy

Create a Gateway Access Policy with:

  • Senders: a tag representing users who need cluster access (e.g. k8s-developers)
  • Gateway / Subnet: the Gateway VM and the subnets listed above

Tune DNS settings

Adjust the upstream query timeout if the DNS path involves the Kubernetes API or a cloud DNS resolver with higher latency than a local nameserver:

sudo enclave set-config gateway-dns-upstream-timeout-ms 1000

If you want the Gateway to answer all DNS queries - not just those whose answers fall within advertised subnets - enable reply-for-all:

sudo enclave set-config gateway-dns-reply-for-all enabled

This is useful when combining Kubernetes DNS with DNS filtering, or when some resolved addresses fall outside the subnets advertised by this Gateway.

Client configuration

Search domains

Kubernetes service names are fully qualified as service.namespace.svc.cluster.local. To resolve short names like my-app or my-app.default, configure DNS search suffixes on client machines:

Add search suffixes via Group Policy, NRPT, or adapter settings:

  • default.svc.cluster.local
  • svc.cluster.local
  • cluster.local

Add search domains in System Settings > Network > DNS > Search Domains:

  • default.svc.cluster.local
  • svc.cluster.local
  • cluster.local

Add to /etc/systemd/resolved.conf or a drop-in file:

[Resolve]
Domains=default.svc.cluster.local svc.cluster.local cluster.local

Then restart systemd-resolved:

sudo systemctl restart systemd-resolved

Verification

From a connected client machine:

  1. Check routes. Run enclave status and confirm the Gateway peer shows the expected subnets:

    Gateway for . . . . : 10.60.0.0/20, 10.52.0.0/17, 172.21.8.2/32

  2. Resolve a Kubernetes name. The default server should be the Enclave stub resolver:

    nslookup kubernetes.default.svc.cluster.local

  3. Access a service:

    curl http://my-app.default.svc.cluster.local

  4. Verify kubectl (if the control-plane endpoint is advertised):

    kubectl get nodes

Further considerations

High availability. Deploy two Gateway VMs in different availability zones, both advertising the same subnets. Enclave automatically fails over between them. See Priority, redundancy and failover.

Multi-cluster access. Each cluster gets its own Gateway VM and its own upstream DNS configuration. If multiple clusters share the same cluster.local zone, assign each Gateway a distinct Enclave tag and create separate Gateway Access Policies so that clients connect to the correct Gateway for the cluster they need.

Monitoring. Use enclave status on the Gateway VM to view DNS query metrics. For deeper visibility into what traffic flows through the Gateway, enable Network Flow Metadata export to an IPFIX collector.

DNS filtering. The Gateway can combine Kubernetes DNS with DNS filtering by chaining upstream resolvers - for example, forwarding to Pi-hole for filtering, with Pi-hole configured to forward cluster.local queries to kube-dns.

Having problems? Contact us at support@enclave.io or visit our support options.

Last updated July 30, 2026