Advanced DNS
Split-horizon, anycast, DNSSEC, and encrypted DNS
Split-Horizon DNS
Split-horizon (also called "split-brain" or "views") is a DNS configuration where the same domain name resolves to different addresses depending on who is asking. The DNS server examines the source IP of the query and returns different answers for internal vs external clients.
$ dig api.example.com @8.8.8.8
;; ANSWER SECTION:
api.example.com. 300 IN A 203.0.113.50
# Public load balancer IP$ dig api.example.com @10.0.0.2
;; ANSWER SECTION:
api.example.com. 60 IN A 10.0.1.15
# Private IP — bypasses LB, NATUse Cases
Internal Service Discovery
- Internal microservices resolve to private IPs
- Avoids hairpin NAT through the public load balancer
- Lower latency, reduced attack surface
- Example:
db.example.comresolves to10.0.2.5internally, NXDOMAIN externally
VPN/Office Access
- Office users resolve internal tools to private IPs
- Remote users on VPN get the same internal view
- Public users get the external (or no) answer
- Common with Active Directory environments
Implementation Approaches
- BIND views: Classic approach. Define ACLs for internal/external networks, and provide different zone files per view. One BIND instance serves both views.
- AWS Route 53 Private Hosted Zones: Create a private hosted zone associated with your VPC(s) and a public hosted zone with the same domain name. VPC resolvers use the private zone; public DNS uses the public zone.
- CoreDNS with conditional forwarding: In K8s, CoreDNS can forward specific zones to internal DNS servers for private resolution, while forwarding everything else upstream.
Route 53 Split-Horizon Example
# Public hosted zone (zone ID: Z1234PUBLIC)
api.example.com. A 203.0.113.50 # ALB public IP
# Private hosted zone (zone ID: Z5678PRIVATE, associated with vpc-abc123)
api.example.com. A 10.0.1.15 # Internal NLB IP
db.example.com. A 10.0.2.5 # RDS private endpoint
cache.example.com. CNAME my-redis.abc123.cache.amazonaws.com.Warning
Debugging split-horizon: When troubleshooting DNS, always note where you're resolving from. dig api.example.com from your laptop may give a different answer than from inside a pod. Use dig @specific-server to explicitly control which resolver you're querying.
GeoDNS
GeoDNS returns different DNS answers based on the geographic location of the querying resolver's IP address (determined via GeoIP databases). This enables routing users to their nearest datacenter or CDN edge.
User in Tokyo
Queries recursive resolver
GeoDNS Server
Checks resolver IP geolocation
Returns ap-northeast-1
Nearest datacenter IP
GeoDNS Behavior
- User in US-East ->
api.example.comresolves to52.1.2.3(us-east-1) - User in Europe ->
api.example.comresolves to34.5.6.7(eu-west-1) - User in Asia ->
api.example.comresolves to13.8.9.10(ap-northeast-1)
Note
EDNS Client Subnet (ECS): Traditional GeoDNS uses the recursive resolver's IP for geolocation, which can be inaccurate (e.g., Google's 8.8.8.8 anycast resolvers are globally distributed). RFC 7871 (ECS) allows the recursive resolver to send a truncated version of the client's IP to the authoritative server, enabling more accurate geolocation.
Route 53 Geolocation Routing
# Route 53 geolocation routing policy
api.example.com. A 52.1.2.3 # Geolocation: North America
api.example.com. A 34.5.6.7 # Geolocation: Europe
api.example.com. A 13.8.9.10 # Geolocation: Asia Pacific
api.example.com. A 52.1.2.3 # Geolocation: Default (fallback)Tip
CDN routing: CDN providers like Cloudflare, Akamai, and CloudFront use GeoDNS (or anycast) to route users to the nearest edge server. When you CNAME to a CDN, the CDN's GeoDNS handles the geographic routing transparently.
Anycast DNS
Anycast is a network addressing and routing methodology where multiple servers share the same IP address. BGP routing directs each client to the topologically nearest instance. This is the backbone of how the DNS root server system and major public resolvers operate.
How Anycast Works
- Multiple physical servers in different locations are configured with the same IP address.
- Each server announces the same IP prefix via BGP to its upstream providers.
- Internet routing naturally directs packets to the closest announcement (shortest AS path, lowest metric).
- The client has no idea there are multiple servers — it just sees one IP.
DNS Root Servers and Anycast
The Root Server Myth: "Only 13 Root Servers"
There are 13 root server addresses (a.root-servers.net through m.root-servers.net), but well over 1,700 physical instances worldwide. The 13-address limit exists because all root server NS records and their glue records must fit in a single 512-byte UDP response (the original DNS limit).
| Root Server | Operator | Instances (approx.) |
|---|---|---|
a.root-servers.net |
Verisign | ~50+ sites |
f.root-servers.net |
ISC (BIND developers) | ~250+ sites |
k.root-servers.net |
RIPE NCC | ~80+ sites |
l.root-servers.net |
ICANN | ~200+ sites |
Each "site" is a full copy of the root zone, deployed at an IXP or datacenter, announcing the same anycast IP via BGP.
Public Resolver Anycast
| Service | IP Addresses | Anycast PoPs |
|---|---|---|
| Cloudflare DNS | 1.1.1.1, 1.0.0.1 |
300+ cities worldwide |
| Google Public DNS | 8.8.8.8, 8.8.4.4 |
20+ locations |
| Quad9 | 9.9.9.9 |
200+ locations |
Note
Anycast + UDP = natural fit: Anycast works perfectly with DNS-over-UDP because each query is a single stateless packet. TCP-based protocols are trickier — if BGP routing changes mid-connection, packets may reach a different anycast instance that has no knowledge of the TCP state, causing a reset. This is why anycast is primarily used for UDP services.
Encrypted DNS
Traditional DNS queries are sent in plaintext over UDP. Anyone on the network path (ISP, coffee shop WiFi, corporate proxy) can see every domain you query. Encrypted DNS protocols solve this.
- Port 853 (dedicated)
- DNS queries wrapped in a TLS session
- Easy to identify and block (distinct port)
- Easy for network operators to allow/deny
- Supported by: Android (Private DNS), systemd-resolved, Unbound, Knot Resolver
- Defined in RFC 7858
- Port 443 (same as regular HTTPS)
- DNS queries sent as HTTPS requests
- Indistinguishable from normal HTTPS traffic
- Very hard to block without blocking all HTTPS
- Supported by: browsers (Chrome, Firefox, Safari), curl, cloudflared
- Defined in RFC 8484
# DoH query using curl
$ curl -s -H 'accept: application/dns-json' \
'https://1.1.1.1/dns-query?name=example.com&type=A' | jq
{
"Status": 0,
"Answer": [
{ "name": "example.com", "type": 1, "TTL": 86400, "data": "93.184.216.34" }
]
}
# DoT using kdig (knot-dns utility)
$ kdig +tls @1.1.1.1 example.com A
Tradeoffs: Privacy vs Network Visibility
The Operator's Dilemma
| Perspective | Encrypted DNS Benefit | Encrypted DNS Problem |
|---|---|---|
| End User | ISP/network can't see queries. Privacy from surveillance. | Bypasses local DNS filtering (parental controls, corporate policy). |
| Network Operator | Users trust your network more. | Lose visibility for security monitoring, logging, threat detection. |
| Security Team | Prevents DNS spoofing on the wire. | Malware can use DoH to bypass DNS-based threat blocking. C2 channels over DoH are hard to detect. |
| Enterprise IT | Protects remote workers on untrusted networks. | Browsers using DoH bypass corporate DNS policies (content filtering, split-horizon). |
Warning
Enterprise concern: Firefox and Chrome enable DoH by default in some regions, bypassing your corporate DNS resolver (and its split-horizon, logging, and security filtering). Enterprises can disable this via Group Policy (Chrome), canary domain detection (Firefox checks for use-application-dns.net — if it resolves, DoH is disabled), or network-level policies.
Note
DNS over QUIC (DoQ): A newer protocol (RFC 9250) that uses QUIC transport (UDP-based, encrypted, multiplexed). Combines the privacy of DoT/DoH with lower latency (0-RTT connection establishment). Still early in adoption.
DNSSEC (DNS Security Extensions)
DNS was designed in the 1980s with no authentication. Any response that arrives with the correct transaction ID is accepted. DNSSEC adds cryptographic signatures to DNS records, allowing resolvers to verify that responses haven't been tampered with.
The Problem: DNS Spoofing
The Kaminsky Attack (2008)
Dan Kaminsky discovered that an attacker can flood a recursive resolver with forged responses for a target domain. The attack exploits the weak entropy of DNS transaction IDs (16 bits = 65,536 possibilities) and the race between legitimate and forged responses.
- Attacker triggers a query for
random123.example.com(guaranteed cache miss) - Simultaneously floods the resolver with forged responses claiming to be from
example.com's authoritative NS - The forged responses include a spoofed authority section pointing to the attacker's nameserver
- If the forged response arrives first with the right transaction ID, the resolver caches it
- Result: the entire
example.comzone is poisoned in the resolver's cache
Mitigations (source port randomization) made this harder but not impossible. DNSSEC is the proper fix.
Chain of Trust
DNSSEC builds a chain of trust from the DNS root zone down to the record you're querying. Each level cryptographically vouches for the next.
Root Zone
Signs .com DS record with root KSK
.com TLD
Signs example.com DS with .com KSK
example.com
Signs A record with zone ZSK
Validating Resolver
Verifies entire chain
DNSSEC Record Types
| Record | Name | Purpose |
|---|---|---|
DNSKEY |
DNS Public Key | Contains the public key for a zone. Two types: KSK (Key Signing Key, signs DNSKEY RRset) and ZSK (Zone Signing Key, signs all other records). |
RRSIG |
Resource Record Signature | Cryptographic signature over a record set (RRset). Each signed RRset has a corresponding RRSIG. Contains: algorithm, signer name, signature expiry, and the signature itself. |
DS |
Delegation Signer | Hash of the child zone's KSK, stored in the parent zone. This is the link in the chain of trust: the parent signs the DS record, vouching for the child's key. |
NSEC |
Next Secure | Authenticated denial of existence. Proves that a name does not exist by showing the "gap" between existing names in the zone (sorted alphabetically). Drawback: allows zone enumeration (walking). |
NSEC3 |
Next Secure v3 | Same as NSEC but uses hashed names to prevent zone enumeration. More complex but standard practice. |
DNSSEC Validation Process
- Trust anchor: The validating resolver has the root zone's public key (KSK) built in. This is the trust anchor — the starting point for all validation.
- Verify root -> TLD delegation: The root zone's DNSKEY signs the DS record for
.com. The resolver verifies the RRSIG on the DS record using the root's DNSKEY. - Verify TLD DNSKEY: The DS record contains a hash of
.com's KSK. The resolver fetches.com's DNSKEY records and verifies the KSK matches the DS hash. - Verify TLD -> domain delegation:
.com's ZSK signs the DS record forexample.com. Resolver verifies. - Verify domain DNSKEY: The DS record hashes to
example.com's KSK. Resolver fetches and verifies DNSKEY records. - Verify the answer:
example.com's ZSK signs the A record's RRSIG. Resolver verifies the signature, confirming the record is authentic and unmodified.
KSK vs ZSK: Why Two Keys?
- KSK (Key Signing Key): Long-lived, strong key (2048+ bit RSA or equivalent). Signs only the DNSKEY record set. Its hash (DS record) is registered in the parent zone. Changing the KSK requires updating the parent (registrar coordination).
- ZSK (Zone Signing Key): Shorter-lived, rotated more frequently (every 1-3 months). Signs all other record sets. Can be rotated without touching the parent zone — only the DNSKEY RRset (signed by KSK) needs updating.
- Separation of concerns: Frequent ZSK rotation limits exposure from key compromise. KSK rotation is rare and more operationally complex.
Why DNSSEC Adoption Is Slow
Operational Complexity
- Key generation, rotation, and rollover procedures
- Signature expiry — if RRSIG records expire and aren't re-signed, the zone becomes BOGUS (validation fails, treated as SERVFAIL)
- Zone signing increases response sizes (4-6x typical), increasing bandwidth and causing UDP truncation more often
- DS record management at the registrar adds a dependency
Risk of Breaking Resolution
- A misconfigured DNSSEC zone is worse than unsigned — it returns SERVFAIL instead of answers
- DNSSEC failures are hard to diagnose for end users and even for many operators
- Some firewalls/middleboxes strip DNSSEC records or block large UDP responses
- Many zones are signed but few resolvers actually validate (chicken-and-egg problem)
Warning
DNSSEC outage example: If you enable DNSSEC and then forget to re-sign the zone before RRSIG expiry (e.g., your automated signing pipeline breaks), validating resolvers will return SERVFAIL for your entire domain. This is a complete outage for all validating clients. Some high-profile domains (including NASA, Comcast, and various government sites) have experienced DNSSEC-related outages.
Tip
Managed DNSSEC: If you use Route 53, Cloudflare, or Google Cloud DNS, they handle zone signing and key rotation automatically. You only need to add the DS record at your registrar (and some registrars, like Cloudflare Registrar, automate even that). This dramatically reduces operational risk.
Tools Deep Dive: dig
dig (Domain Information Groper) is the standard CLI tool for DNS queries. It's part of the BIND utilities package (bind-utils on RHEL/CentOS, dnsutils on Debian/Ubuntu).
Basic Queries
# Basic A record lookup
$ dig example.com
# Query a specific record type
$ dig example.com MX
$ dig example.com AAAA
$ dig example.com TXT
$ dig example.com NS
$ dig example.com SOA
$ dig example.com CAA
# Short output (just the answer)
$ dig example.com +short
93.184.216.34
# Query a specific nameserver
$ dig example.com @8.8.8.8
$ dig example.com @ns1.example.com
# Query ALL record types
$ dig example.com ANY
# Note: many servers now refuse ANY queries (RFC 8482) for DDoS mitigation
Reverse DNS Lookup
# Reverse lookup (IP -> hostname)
$ dig -x 93.184.216.34
;; ANSWER SECTION:
34.216.184.93.in-addr.arpa. 86400 IN PTR example.com.
# Short form
$ dig -x 93.184.216.34 +short
example.com.
dig +trace — Follow the Delegation Chain
The most powerful diagnostic: +trace makes dig perform iterative resolution from the root, showing every delegation step.
$ dig +trace api.example.com
; <<>> DiG 9.18.18 <<>> +trace api.example.com
;; global options: +cmd
. 518400 IN NS a.root-servers.net.
. 518400 IN NS b.root-servers.net.
. 518400 IN NS c.root-servers.net.
;; ... (13 root servers)
;; Received 239 bytes from 127.0.0.53#53 in 0 ms
com. 172800 IN NS a.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
;; ... (referral from root to .com TLD)
;; Received 836 bytes from 198.41.0.4#53(a.root-servers.net) in 24 ms
example.com. 172800 IN NS ns1.example.com.
example.com. 172800 IN NS ns2.example.com.
;; ... (referral from .com TLD to authoritative)
;; Received 460 bytes from 192.5.6.30#53(a.gtld-servers.net) in 32 ms
api.example.com. 300 IN A 93.184.216.34
;; Received 62 bytes from 198.51.100.1#53(ns1.example.com) in 15 ms
Tip
When to use +trace: When you need to verify that delegation is correct, diagnose where in the chain a resolution fails, confirm authoritative answers (bypassing caching), or debug DNSSEC chain-of-trust issues.
dig +dnssec — Request DNSSEC Records
$ dig example.com +dnssec
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; ad flag = Authenticated Data (DNSSEC validation passed)
;; ANSWER SECTION:
example.com. 86400 IN A 93.184.216.34
example.com. 86400 IN RRSIG A 13 2 86400 (
20240415120000 20240401120000 12345 example.com.
aB3cD4eF5gH6iJ7kL8mN9oP0qR... )
DNSSEC Flags in dig Output
| Flag | Meaning |
|---|---|
ad (Authenticated Data) |
The recursive resolver validated the DNSSEC chain successfully. The answer is authentic. |
cd (Checking Disabled) |
The client asked the resolver to skip DNSSEC validation (used for debugging). |
do (DNSSEC OK) |
EDNS0 flag indicating the client wants DNSSEC records in the response. |
Useful dig Combinations
# Check NS delegation at the TLD level
$ dig example.com NS @a.gtld-servers.net +norecurse
# Get all records at a name (with caveats)
$ dig example.com ANY +noall +answer
# Show only answer section, clean output
$ dig example.com +noall +answer
example.com. 86400 IN A 93.184.216.34
# Check if DNSSEC is configured (look for DS at parent)
$ dig example.com DS @a.gtld-servers.net +short
12345 13 2 aB3cD4eF5gH6iJ7kL8mN9oP0qR...
# Measure query time
$ dig example.com | grep "Query time"
;; Query time: 12 msec
# CNAME chain following
$ dig www.example.com +trace +nodnssec
# Check zone transfer (if allowed)
$ dig example.com AXFR @ns1.example.com
# Usually refused: Transfer failed.
Other Useful DNS Tools
| Tool | Package | Use Case |
|---|---|---|
dig |
bind-utils / dnsutils |
Full-featured DNS query tool (the standard) |
nslookup |
bind-utils / dnsutils |
Simpler DNS queries. Works on Windows. Less output than dig. |
host |
bind-utils / dnsutils |
Quick lookups. host example.com gives clean output. |
kdig |
knot-dnsutils |
Like dig, but supports DoT and DoH natively. kdig +tls @1.1.1.1 |
drill |
ldns-utils |
DNSSEC-focused dig alternative. Good chain validation output. |
dog |
Rust (cargo install dog) |
Modern, colorized DNS client. DoT and DoH support. |
resolvectl |
systemd | Query systemd-resolved cache, stats, and configuration. |
Quick Diagnostic Workflow
dig example.com +short— Does it resolve at all? What IP do I get?dig example.com @8.8.8.8— Is this a local resolver issue or global?dig example.com @ns1.example.com— Is the authoritative server returning the right answer?dig +trace example.com— Walk the delegation chain. Where does it break?dig example.com +dnssec— Is DNSSEC causing validation failures (SERVFAIL)?dig example.com +cd— Checking Disabled: if this works but step 5 doesn't, DNSSEC is broken.