Skip to content
Menu

Networking13 min read

Advanced DNS

Split-horizon, anycast, DNSSEC, and encrypted DNS

Split-Horizon DNS

Split-horizon (also called "split-brain" or "views") is a DNS configuration where the same domain name resolves to different addresses depending on who is asking. The DNS server examines the source IP of the query and returns different answers for internal vs external clients.

External Client (Public Internet)
console
$ dig api.example.com @8.8.8.8
;; ANSWER SECTION:
api.example.com. 300 IN A 203.0.113.50
# Public load balancer IP
Internal Client (VPC / Office)
console
$ dig api.example.com @10.0.0.2
;; ANSWER SECTION:
api.example.com. 60 IN A 10.0.1.15
# Private IP — bypasses LB, NAT

Use Cases

Internal Service Discovery

  • Internal microservices resolve to private IPs
  • Avoids hairpin NAT through the public load balancer
  • Lower latency, reduced attack surface
  • Example: db.example.com resolves to 10.0.2.5 internally, NXDOMAIN externally

VPN/Office Access

  • Office users resolve internal tools to private IPs
  • Remote users on VPN get the same internal view
  • Public users get the external (or no) answer
  • Common with Active Directory environments

Implementation Approaches

  1. BIND views: Classic approach. Define ACLs for internal/external networks, and provide different zone files per view. One BIND instance serves both views.
  2. AWS Route 53 Private Hosted Zones: Create a private hosted zone associated with your VPC(s) and a public hosted zone with the same domain name. VPC resolvers use the private zone; public DNS uses the public zone.
  3. CoreDNS with conditional forwarding: In K8s, CoreDNS can forward specific zones to internal DNS servers for private resolution, while forwarding everything else upstream.

Route 53 Split-Horizon Example

console
# Public hosted zone (zone ID: Z1234PUBLIC)
api.example.com.  A  203.0.113.50   # ALB public IP

# Private hosted zone (zone ID: Z5678PRIVATE, associated with vpc-abc123)
api.example.com.  A  10.0.1.15      # Internal NLB IP
db.example.com.   A  10.0.2.5       # RDS private endpoint
cache.example.com. CNAME my-redis.abc123.cache.amazonaws.com.

Warning

Debugging split-horizon: When troubleshooting DNS, always note where you're resolving from. dig api.example.com from your laptop may give a different answer than from inside a pod. Use dig @specific-server to explicitly control which resolver you're querying.

GeoDNS

GeoDNS returns different DNS answers based on the geographic location of the querying resolver's IP address (determined via GeoIP databases). This enables routing users to their nearest datacenter or CDN edge.

  1. User in Tokyo

    Queries recursive resolver

  2. GeoDNS Server

    Checks resolver IP geolocation

  3. Returns ap-northeast-1

    Nearest datacenter IP

GeoDNS Behavior

  • User in US-East -> api.example.com resolves to 52.1.2.3 (us-east-1)
  • User in Europe -> api.example.com resolves to 34.5.6.7 (eu-west-1)
  • User in Asia -> api.example.com resolves to 13.8.9.10 (ap-northeast-1)

Note

EDNS Client Subnet (ECS): Traditional GeoDNS uses the recursive resolver's IP for geolocation, which can be inaccurate (e.g., Google's 8.8.8.8 anycast resolvers are globally distributed). RFC 7871 (ECS) allows the recursive resolver to send a truncated version of the client's IP to the authoritative server, enabling more accurate geolocation.

Route 53 Geolocation Routing

console
# Route 53 geolocation routing policy
api.example.com.  A  52.1.2.3     # Geolocation: North America
api.example.com.  A  34.5.6.7     # Geolocation: Europe
api.example.com.  A  13.8.9.10    # Geolocation: Asia Pacific
api.example.com.  A  52.1.2.3     # Geolocation: Default (fallback)

Tip

CDN routing: CDN providers like Cloudflare, Akamai, and CloudFront use GeoDNS (or anycast) to route users to the nearest edge server. When you CNAME to a CDN, the CDN's GeoDNS handles the geographic routing transparently.

Anycast DNS

Anycast is a network addressing and routing methodology where multiple servers share the same IP address. BGP routing directs each client to the topologically nearest instance. This is the backbone of how the DNS root server system and major public resolvers operate.

How Anycast Works

  • Multiple physical servers in different locations are configured with the same IP address.
  • Each server announces the same IP prefix via BGP to its upstream providers.
  • Internet routing naturally directs packets to the closest announcement (shortest AS path, lowest metric).
  • The client has no idea there are multiple servers — it just sees one IP.
User
Client
Queries 1.1.1.1
BGP
Internet Routing
Shortest path wins
DC
Nearest Instance
Same IP, closest server

DNS Root Servers and Anycast

The Root Server Myth: "Only 13 Root Servers"

There are 13 root server addresses (a.root-servers.net through m.root-servers.net), but well over 1,700 physical instances worldwide. The 13-address limit exists because all root server NS records and their glue records must fit in a single 512-byte UDP response (the original DNS limit).

Root Server Operator Instances (approx.)
a.root-servers.net Verisign ~50+ sites
f.root-servers.net ISC (BIND developers) ~250+ sites
k.root-servers.net RIPE NCC ~80+ sites
l.root-servers.net ICANN ~200+ sites

Each "site" is a full copy of the root zone, deployed at an IXP or datacenter, announcing the same anycast IP via BGP.

Public Resolver Anycast

Service IP Addresses Anycast PoPs
Cloudflare DNS 1.1.1.1, 1.0.0.1 300+ cities worldwide
Google Public DNS 8.8.8.8, 8.8.4.4 20+ locations
Quad9 9.9.9.9 200+ locations

Note

Anycast + UDP = natural fit: Anycast works perfectly with DNS-over-UDP because each query is a single stateless packet. TCP-based protocols are trickier — if BGP routing changes mid-connection, packets may reach a different anycast instance that has no knowledge of the TCP state, causing a reset. This is why anycast is primarily used for UDP services.

Encrypted DNS

Traditional DNS queries are sent in plaintext over UDP. Anyone on the network path (ISP, coffee shop WiFi, corporate proxy) can see every domain you query. Encrypted DNS protocols solve this.

DNS over TLS (DoT)
  • Port 853 (dedicated)
  • DNS queries wrapped in a TLS session
  • Easy to identify and block (distinct port)
  • Easy for network operators to allow/deny
  • Supported by: Android (Private DNS), systemd-resolved, Unbound, Knot Resolver
  • Defined in RFC 7858
DNS over HTTPS (DoH)
  • Port 443 (same as regular HTTPS)
  • DNS queries sent as HTTPS requests
  • Indistinguishable from normal HTTPS traffic
  • Very hard to block without blocking all HTTPS
  • Supported by: browsers (Chrome, Firefox, Safari), curl, cloudflared
  • Defined in RFC 8484
console
console

    # DoH query using curl

    $ curl -s -H 'accept: application/dns-json' \

      'https://1.1.1.1/dns-query?name=example.com&type=A' | jq

    {

      "Status": 0,

      "Answer": [

        { "name": "example.com", "type": 1, "TTL": 86400, "data": "93.184.216.34" }

      ]

    }

    

    # DoT using kdig (knot-dns utility)

    $ kdig +tls @1.1.1.1 example.com A
  

Tradeoffs: Privacy vs Network Visibility

The Operator's Dilemma

Perspective Encrypted DNS Benefit Encrypted DNS Problem
End User ISP/network can't see queries. Privacy from surveillance. Bypasses local DNS filtering (parental controls, corporate policy).
Network Operator Users trust your network more. Lose visibility for security monitoring, logging, threat detection.
Security Team Prevents DNS spoofing on the wire. Malware can use DoH to bypass DNS-based threat blocking. C2 channels over DoH are hard to detect.
Enterprise IT Protects remote workers on untrusted networks. Browsers using DoH bypass corporate DNS policies (content filtering, split-horizon).

Warning

Enterprise concern: Firefox and Chrome enable DoH by default in some regions, bypassing your corporate DNS resolver (and its split-horizon, logging, and security filtering). Enterprises can disable this via Group Policy (Chrome), canary domain detection (Firefox checks for use-application-dns.net — if it resolves, DoH is disabled), or network-level policies.

Note

DNS over QUIC (DoQ): A newer protocol (RFC 9250) that uses QUIC transport (UDP-based, encrypted, multiplexed). Combines the privacy of DoT/DoH with lower latency (0-RTT connection establishment). Still early in adoption.

DNSSEC (DNS Security Extensions)

DNS was designed in the 1980s with no authentication. Any response that arrives with the correct transaction ID is accepted. DNSSEC adds cryptographic signatures to DNS records, allowing resolvers to verify that responses haven't been tampered with.

The Problem: DNS Spoofing

The Kaminsky Attack (2008)

Dan Kaminsky discovered that an attacker can flood a recursive resolver with forged responses for a target domain. The attack exploits the weak entropy of DNS transaction IDs (16 bits = 65,536 possibilities) and the race between legitimate and forged responses.

  • Attacker triggers a query for random123.example.com (guaranteed cache miss)
  • Simultaneously floods the resolver with forged responses claiming to be from example.com's authoritative NS
  • The forged responses include a spoofed authority section pointing to the attacker's nameserver
  • If the forged response arrives first with the right transaction ID, the resolver caches it
  • Result: the entire example.com zone is poisoned in the resolver's cache

Mitigations (source port randomization) made this harder but not impossible. DNSSEC is the proper fix.

Chain of Trust

DNSSEC builds a chain of trust from the DNS root zone down to the record you're querying. Each level cryptographically vouches for the next.

  1. Root Zone

    Signs .com DS record with root KSK

  2. .com TLD

    Signs example.com DS with .com KSK

  3. example.com

    Signs A record with zone ZSK

  4. Validating Resolver

    Verifies entire chain

DNSSEC Record Types

Record Name Purpose
DNSKEY DNS Public Key Contains the public key for a zone. Two types: KSK (Key Signing Key, signs DNSKEY RRset) and ZSK (Zone Signing Key, signs all other records).
RRSIG Resource Record Signature Cryptographic signature over a record set (RRset). Each signed RRset has a corresponding RRSIG. Contains: algorithm, signer name, signature expiry, and the signature itself.
DS Delegation Signer Hash of the child zone's KSK, stored in the parent zone. This is the link in the chain of trust: the parent signs the DS record, vouching for the child's key.
NSEC Next Secure Authenticated denial of existence. Proves that a name does not exist by showing the "gap" between existing names in the zone (sorted alphabetically). Drawback: allows zone enumeration (walking).
NSEC3 Next Secure v3 Same as NSEC but uses hashed names to prevent zone enumeration. More complex but standard practice.

DNSSEC Validation Process

  1. Trust anchor: The validating resolver has the root zone's public key (KSK) built in. This is the trust anchor — the starting point for all validation.
  2. Verify root -> TLD delegation: The root zone's DNSKEY signs the DS record for .com. The resolver verifies the RRSIG on the DS record using the root's DNSKEY.
  3. Verify TLD DNSKEY: The DS record contains a hash of .com's KSK. The resolver fetches .com's DNSKEY records and verifies the KSK matches the DS hash.
  4. Verify TLD -> domain delegation: .com's ZSK signs the DS record for example.com. Resolver verifies.
  5. Verify domain DNSKEY: The DS record hashes to example.com's KSK. Resolver fetches and verifies DNSKEY records.
  6. Verify the answer: example.com's ZSK signs the A record's RRSIG. Resolver verifies the signature, confirming the record is authentic and unmodified.

KSK vs ZSK: Why Two Keys?

  • KSK (Key Signing Key): Long-lived, strong key (2048+ bit RSA or equivalent). Signs only the DNSKEY record set. Its hash (DS record) is registered in the parent zone. Changing the KSK requires updating the parent (registrar coordination).
  • ZSK (Zone Signing Key): Shorter-lived, rotated more frequently (every 1-3 months). Signs all other record sets. Can be rotated without touching the parent zone — only the DNSKEY RRset (signed by KSK) needs updating.
  • Separation of concerns: Frequent ZSK rotation limits exposure from key compromise. KSK rotation is rare and more operationally complex.

Why DNSSEC Adoption Is Slow

Operational Complexity

  • Key generation, rotation, and rollover procedures
  • Signature expiry — if RRSIG records expire and aren't re-signed, the zone becomes BOGUS (validation fails, treated as SERVFAIL)
  • Zone signing increases response sizes (4-6x typical), increasing bandwidth and causing UDP truncation more often
  • DS record management at the registrar adds a dependency

Risk of Breaking Resolution

  • A misconfigured DNSSEC zone is worse than unsigned — it returns SERVFAIL instead of answers
  • DNSSEC failures are hard to diagnose for end users and even for many operators
  • Some firewalls/middleboxes strip DNSSEC records or block large UDP responses
  • Many zones are signed but few resolvers actually validate (chicken-and-egg problem)

Warning

DNSSEC outage example: If you enable DNSSEC and then forget to re-sign the zone before RRSIG expiry (e.g., your automated signing pipeline breaks), validating resolvers will return SERVFAIL for your entire domain. This is a complete outage for all validating clients. Some high-profile domains (including NASA, Comcast, and various government sites) have experienced DNSSEC-related outages.

Tip

Managed DNSSEC: If you use Route 53, Cloudflare, or Google Cloud DNS, they handle zone signing and key rotation automatically. You only need to add the DS record at your registrar (and some registrars, like Cloudflare Registrar, automate even that). This dramatically reduces operational risk.

Tools Deep Dive: dig

dig (Domain Information Groper) is the standard CLI tool for DNS queries. It's part of the BIND utilities package (bind-utils on RHEL/CentOS, dnsutils on Debian/Ubuntu).

Basic Queries

console
console

    # Basic A record lookup

    $ dig example.com

    

    # Query a specific record type

    $ dig example.com MX

    $ dig example.com AAAA

    $ dig example.com TXT

    $ dig example.com NS

    $ dig example.com SOA

    $ dig example.com CAA

    

    # Short output (just the answer)

    $ dig example.com +short

    93.184.216.34

    

    # Query a specific nameserver

    $ dig example.com @8.8.8.8

    $ dig example.com @ns1.example.com

    

    # Query ALL record types

    $ dig example.com ANY

    # Note: many servers now refuse ANY queries (RFC 8482) for DDoS mitigation
  

Reverse DNS Lookup

console
console

    # Reverse lookup (IP -> hostname)

    $ dig -x 93.184.216.34

    

    ;; ANSWER SECTION:

    34.216.184.93.in-addr.arpa. 86400 IN PTR example.com.

    

    # Short form

    $ dig -x 93.184.216.34 +short

    example.com.
  

dig +trace — Follow the Delegation Chain

The most powerful diagnostic: +trace makes dig perform iterative resolution from the root, showing every delegation step.

console
console

    $ dig +trace api.example.com

    

    ; <<>> DiG 9.18.18 <<>> +trace api.example.com

    ;; global options: +cmd

    

    .                       518400  IN  NS  a.root-servers.net.

    .                       518400  IN  NS  b.root-servers.net.

    .                       518400  IN  NS  c.root-servers.net.

    ;; ... (13 root servers)

    ;; Received 239 bytes from 127.0.0.53#53 in 0 ms

    

    com.                    172800  IN  NS  a.gtld-servers.net.

    com.                    172800  IN  NS  b.gtld-servers.net.

    ;; ... (referral from root to .com TLD)

    ;; Received 836 bytes from 198.41.0.4#53(a.root-servers.net) in 24 ms

    

    example.com.            172800  IN  NS  ns1.example.com.

    example.com.            172800  IN  NS  ns2.example.com.

    ;; ... (referral from .com TLD to authoritative)

    ;; Received 460 bytes from 192.5.6.30#53(a.gtld-servers.net) in 32 ms

    

    api.example.com.        300     IN  A   93.184.216.34

    ;; Received 62 bytes from 198.51.100.1#53(ns1.example.com) in 15 ms
  

Tip

When to use +trace: When you need to verify that delegation is correct, diagnose where in the chain a resolution fails, confirm authoritative answers (bypassing caching), or debug DNSSEC chain-of-trust issues.

dig +dnssec — Request DNSSEC Records

console
console

    $ dig example.com +dnssec

    

    ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

    ;; ad flag = Authenticated Data (DNSSEC validation passed)

    

    ;; ANSWER SECTION:

    example.com.        86400  IN  A     93.184.216.34

    example.com.        86400  IN  RRSIG A 13 2 86400 (

                        20240415120000 20240401120000 12345 example.com.

                        aB3cD4eF5gH6iJ7kL8mN9oP0qR... )
  

DNSSEC Flags in dig Output

Flag Meaning
ad (Authenticated Data) The recursive resolver validated the DNSSEC chain successfully. The answer is authentic.
cd (Checking Disabled) The client asked the resolver to skip DNSSEC validation (used for debugging).
do (DNSSEC OK) EDNS0 flag indicating the client wants DNSSEC records in the response.

Useful dig Combinations

console
console

    # Check NS delegation at the TLD level

    $ dig example.com NS @a.gtld-servers.net +norecurse

    

    # Get all records at a name (with caveats)

    $ dig example.com ANY +noall +answer

    

    # Show only answer section, clean output

    $ dig example.com +noall +answer

    example.com.    86400   IN      A       93.184.216.34

    

    # Check if DNSSEC is configured (look for DS at parent)

    $ dig example.com DS @a.gtld-servers.net +short

    12345 13 2 aB3cD4eF5gH6iJ7kL8mN9oP0qR...

    

    # Measure query time

    $ dig example.com | grep "Query time"

    ;; Query time: 12 msec

    

    # CNAME chain following

    $ dig www.example.com +trace +nodnssec

    

    # Check zone transfer (if allowed)

    $ dig example.com AXFR @ns1.example.com

    # Usually refused: Transfer failed.
  

Other Useful DNS Tools

Tool Package Use Case
dig bind-utils / dnsutils Full-featured DNS query tool (the standard)
nslookup bind-utils / dnsutils Simpler DNS queries. Works on Windows. Less output than dig.
host bind-utils / dnsutils Quick lookups. host example.com gives clean output.
kdig knot-dnsutils Like dig, but supports DoT and DoH natively. kdig +tls @1.1.1.1
drill ldns-utils DNSSEC-focused dig alternative. Good chain validation output.
dog Rust (cargo install dog) Modern, colorized DNS client. DoT and DoH support.
resolvectl systemd Query systemd-resolved cache, stats, and configuration.

Quick Diagnostic Workflow

  1. dig example.com +short — Does it resolve at all? What IP do I get?
  2. dig example.com @8.8.8.8 — Is this a local resolver issue or global?
  3. dig example.com @ns1.example.com — Is the authoritative server returning the right answer?
  4. dig +trace example.com — Walk the delegation chain. Where does it break?
  5. dig example.com +dnssec — Is DNSSEC causing validation failures (SERVFAIL)?
  6. dig example.com +cd — Checking Disabled: if this works but step 5 doesn't, DNSSEC is broken.
Solidnines — solidnines.com