esddns

MetalLB Deployment Guide for ESDDNS

Overview

This guide explains how to deploy ESDDNS on bare-metal (on-premises) Kubernetes clusters using MetalLB for LoadBalancer support.

Why MetalLB?

MetalLB provides LoadBalancer service type capability for Kubernetes clusters that don’t run on cloud providers (AWS, GCP, Azure). This is essential for on-premises deployments.

MetalLB Benefits for ESDDNS

✅ LoadBalancer Service Type - Enables cloud-like LoadBalancer on bare-metal
✅ Layer 2 Mode - Simple ARP-based IP advertisement (same-subnet only)
✅ BGP Mode - Enterprise-grade routing with multi-subnet support
✅ STUN Integration - Works perfectly with ESDDNS’s STUN-based WAN IP detection
✅ No Vendor Lock-in - Open-source, works on any Kubernetes cluster

How It Works with ESDDNS

On-Premises Network
┌────────────────────────────────────────────────────────┐
│ Kubernetes Cluster                                     │
│                                                        │
│  ┌──────────────────────────────────────────────┐      │
│  │ ESDDNS Operator (DaemonSet)                  │      │
│  │ • Detects WAN IP via STUN protocol           │      │
│  │ • Updates Gandi DNS with public IP           │      │
│  └──────────────────────────────────────────────┘      │
│                                                        │
│  ┌──────────────────────────────────────────────┐      │
│  │ ESDDNS Service (LoadBalancer)                │      │
│  │ • Type: LoadBalancer                         │      │
│  │ • MetalLB assigns IP: 192.168.1.100          │      │
│  └──────────────────────────────────────────────┘      │
│                                                        │
│  ┌──────────────────────────────────────────────┐      │
│  │ MetalLB Speaker (DaemonSet)                  │      │
│  │ • Advertises 192.168.1.100 on local network  │      │
│  │ • Layer 2: ARP responses                     │      │
│  │ • BGP: Routes to upstream routers            │      │
│  └──────────────────────────────────────────────┘      │
│                                                        │
└────────────────┬───────────────────────────────────────┘
                 │
                 ▼
         Local Network Router
         (receives ARP/BGP)
                 │
                 ▼
            NAT Gateway
                 │
                 ▼
            Internet
         (public WAN IP)
                 │
      ┌──────────┴──────────┐
      │                     │
   STUN Servers      Gandi.net API
(detect WAN IP)   (update DNS records)

Key Points:

MetalLB: Provides LoadBalancer capability on-prem (assigns local IP)
STUN: Detects actual public WAN IP behind NAT
ESDDNS: Updates DNS with the public IP, not the LoadBalancer IP

Quick Start (Automated)

One-Command Deployment

export GANDI_API_KEY="your-gandi-api-key"
cd k8s
./deploy-metallb.sh production 192.168.1.100-192.168.1.110

This script:

✅ Checks prerequisites (kubectl, helm)
✅ Installs MetalLB (via Helm or kubectl)
✅ Creates IP address pool with your IP range
✅ Configures Layer 2 advertisement
✅ Deploys ESDDNS operator with LoadBalancer service
✅ Verifies deployment and shows status

Script Arguments

./deploy-metallb.sh [environment] [ip-range]

# Examples:
./deploy-metallb.sh production 192.168.1.100/32              # Single IP
./deploy-metallb.sh production 192.168.1.100-192.168.1.110   # IP range
./deploy-metallb.sh development 10.0.0.100/28                # CIDR notation

Supported IP Range Formats

Format	Example	Description
Single IP	`192.168.1.100/32`	Only one IP address
Range	`192.168.1.100-192.168.1.110`	Range of IPs (11 addresses)
CIDR	`192.168.1.0/28`	CIDR notation (16 addresses)

Important: Choose IPs that are:

On your local network subnet
NOT assigned by DHCP
NOT already in use by other devices

Manual Deployment

If you prefer manual control or need to customize the configuration:

Step 1: Install MetalLB

Option A: Using Helm (Recommended)

helm repo add metallb https://metallb.github.io/metallb
helm repo update
helm install metallb metallb/metallb \
  --namespace metallb-system \
  --create-namespace

Option B: Using kubectl

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.5/config/manifests/metallb-native.yaml

Step 2: Wait for MetalLB to be Ready

kubectl wait --namespace metallb-system \
  --for=condition=ready pod \
  --selector=app=metallb \
  --timeout=90s

Step 3: Create IP Address Pool

cat <<EOF | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: esddns-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.100-192.168.1.110  # Your IP range
  autoAssign: true
EOF

Step 4: Configure Layer 2 Advertisement

cat <<EOF | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: esddns-l2-advertisement
  namespace: metallb-system
spec:
  ipAddressPools:
  - esddns-pool
EOF

Step 5: Deploy ESDDNS

export GANDI_API_KEY="your-gandi-api-key"
cd k8s
./deploy.sh production

Step 6: Verify LoadBalancer IP

kubectl get svc -n esddns-production esddns-service

# Should show something like:
# NAME             TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)
# esddns-service   LoadBalancer   10.96.123.45    192.168.1.100    80:30123/TCP

Layer 2 vs BGP Mode

MetalLB supports two modes for IP advertisement:

Layer 2 Mode (Default - Simpler)

How it works:

MetalLB responds to ARP requests for the LoadBalancer IP
Traffic is sent directly to the node running the MetalLB speaker
Works on same subnet only

Use when:

Simple network setup
All nodes on same subnet
No BGP infrastructure
Small to medium deployments

Configuration:

apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: esddns-l2
  namespace: metallb-system
spec:
  ipAddressPools:
  - esddns-pool

BGP Mode (Advanced - Enterprise)

How it works:

MetalLB announces routes via BGP to your network routers
Traffic is routed by your network infrastructure
Supports multiple subnets and advanced routing

Use when:

Enterprise network with BGP routers
Multi-subnet deployments
Need advanced routing features
Large-scale deployments

Configuration:

# 1. Create BGP peer
apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
  name: router-peer
  namespace: metallb-system
spec:
  myASN: 64500          # Your ASN
  peerASN: 64501        # Router's ASN
  peerAddress: 192.168.1.1  # Router IP

---
# 2. Create BGP advertisement
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: esddns-bgp
  namespace: metallb-system
spec:
  ipAddressPools:
  - esddns-pool

Verification & Testing

Check MetalLB Status

# All MetalLB resources
kubectl get all -n metallb-system

# IP address pools
kubectl get ipaddresspool -n metallb-system

# L2 advertisements
kubectl get l2advertisement -n metallb-system

# MetalLB speaker logs
kubectl logs -n metallb-system -l component=speaker -f

Check ESDDNS LoadBalancer

# Get service status
kubectl get svc -n esddns-production esddns-service

# Get LoadBalancer IP
EXTERNAL_IP=$(kubectl get svc -n esddns-production esddns-service \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

echo "LoadBalancer IP: $EXTERNAL_IP"

# Test endpoint
curl http://$EXTERNAL_IP/

Check WAN IP Detection

# View operator logs (should show STUN detection)
kubectl logs -n esddns-production -l app=esddns-operator -f

# Look for:
# INFO STUN protocol enabled for IP discovery
# INFO [stun] Successfully obtained IP: X.X.X.X

Troubleshooting

LoadBalancer IP Stuck in “Pending”

Symptoms:

kubectl get svc -n esddns-production esddns-service
# EXTERNAL-IP shows <pending>

Solutions:

Check MetalLB is running:
```
kubectl get pods -n metallb-system
```

Check IP address pool exists:

kubectl get ipaddresspool -n metallb-system esddns-pool

Check speaker logs:

kubectl logs -n metallb-system -l component=speaker

Verify IP range is available on network

MetalLB Not Installing

Symptoms:

Helm install fails
Pods crash or don’t start

Solutions:

Check Kubernetes version (1.19+ required)
Verify RBAC is enabled
Check cluster has sufficient resources
Review installation logs:
```
kubectl describe pod -n metallb-system
```

No Traffic Reaching Service

Symptoms:

LoadBalancer IP assigned
But curl timeout or connection refused

Solutions:

Check ESDDNS pods are running:
```
kubectl get pods -n esddns-production
```

Port-forward to test pod directly:

kubectl port-forward -n esddns-production svc/esddns-service 8080:80
curl http://localhost:8080/

Check firewall rules on nodes
Verify service port configuration

ARP Issues (Layer 2 Mode)

Symptoms:

LoadBalancer IP assigned
Only works from some network locations

Solutions:

Check all nodes on same subnet
Verify no network policies blocking ARP
Check switch/router ARP settings
Try pinging LoadBalancer IP from different hosts

BGP Not Peering

Symptoms:

BGP peer shows “not established”

Solutions:

Verify router BGP configuration
Check ASN numbers match
Verify peer IP addresses
Check firewall allows BGP (TCP 179)

Review speaker logs:

kubectl logs -n metallb-system -l component=speaker | grep BGP

WAN IP Detection with STUN

Why STUN is Perfect for On-Prem

ESDDNS uses STUN (Session Traversal Utilities for NAT) protocol to detect your public WAN IP address. This is ideal for on-premises deployments because:

✅ NAT Traversal - Designed specifically for discovering public IPs behind NAT
✅ Fast - Typically ~2x faster than HTTP-based detection
✅ Reliable - RFC 8489 standard, widely supported
✅ Free Public Servers - Google, Cloudflare, Twilio provide STUN infrastructure

How It Works

On-Prem Network         NAT Gateway         Internet
┌──────────────┐       ┌──────────┐       ┌─────────────────┐
│ ESDDNS Pod   │──────▶│  NAT     │──────▶│ STUN Server     │
│ Private IP:  │       │  Maps:   │       │ stun.google.com │
│ 10.0.0.5     │       │  10.0.0.5│       │                 │
│              │       │  ──▶     │       │ Response:       │
│              │       │  X.X.X.X │       │ "Your IP is     │
│              │◀──────│          │◀──────│  X.X.X.X"       │
└──────────────┘       └──────────┘       └─────────────────┘

ESDDNS pod sends STUN request to public STUN server
Request passes through your NAT gateway
STUN server sees the public source IP (your WAN IP)
STUN server responds with the detected public IP
ESDDNS updates DNS with this public IP

STUN Configuration

STUN is enabled by default in ESDDNS. Configuration is in the ConfigMap:

[STUNConfig]
udp_host_list_url = https://raw.githubusercontent.com/pradt2/always-online-stun/master/valid_hosts.txt
tcp_host_list_url = https://raw.githubusercontent.com/pradt2/always-online-stun/master/valid_hosts_tcp.txt
bind_request = 1
magic_cookie = 554869826
xor_mapped_address = 32
udp_limit = 10
tcp_limit = 10
retry_attempts = 3
retry_cooldown_seconds = 5

Fallback Chain

ESDDNS uses multiple methods with automatic fallback:

STUN Protocol (primary) - Fast, designed for NAT traversal
HTTP Services (fallback) - api.ipify.org, checkip.amazonaws.com
Direct Detection (last resort) - If external services fail

Advanced Configuration

Custom IP Pool Naming

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: my-custom-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.100-192.168.1.110
  autoAssign: true
  avoidBuggyIPs: true  # Skip .0 and .255

Multiple IP Pools

# Production pool
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: production-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.100-192.168.1.110

---
# Development pool
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: development-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.200-192.168.1.210

Service-Specific IP Assignment

apiVersion: v1
kind: Service
metadata:
  name: esddns-service
  annotations:
    metallb.universe.tf/address-pool: production-pool  # Specific pool
    metallb.universe.tf/loadBalancerIPs: 192.168.1.100  # Specific IP
spec:
  type: LoadBalancer
  # ... rest of service spec

Production Checklist

Useful Commands Reference

# MetalLB
kubectl get all -n metallb-system
kubectl get ipaddresspool -n metallb-system
kubectl get l2advertisement -n metallb-system
kubectl logs -n metallb-system -l component=speaker -f

# ESDDNS
kubectl get all -n esddns-production
kubectl get svc -n esddns-production esddns-service
kubectl logs -n esddns-production -l app=esddns-operator -f
kubectl logs -n esddns-production -l app=esddns-service -f

# Get LoadBalancer IP
EXTERNAL_IP=$(kubectl get svc -n esddns-production esddns-service \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl http://$EXTERNAL_IP/

# Restart ESDDNS
kubectl rollout restart daemonset/esddns-operator-daemon -n esddns-production
kubectl rollout restart deployment/esddns-service -n esddns-production

Resources

MetalLB Documentation: https://metallb.universe.tf
ESDDNS Documentation: README.md
STUN RFC: https://datatracker.ietf.org/doc/html/rfc8489
Quick Start: QUICKSTART.md
Kubernetes Operator: k8s/README.md

Support

For issues or questions:

Check QUICKSTART.md troubleshooting
Review MetalLB speaker logs: kubectl logs -n metallb-system -l component=speaker
Review ESDDNS operator logs: kubectl logs -n esddns-production -l app=esddns-operator
Open GitHub issue: https://github.com/sqe/esddns/issues

This site is open source. Improve this page.