Several A or AAAA records at one name provide an address set. A DNS server may rotate the order it returns, but the receiving resolver and application decide how to use that set. Ordinary address records contain no request-load measurement or application-health result. Their presence alone does not promise equal traffic, automatic failover, or even a particular first address across implementations.
When it breaks
DNS answer distribution and application request distribution have different denominators. One recursive resolver can serve many clients from a cached answer, and one application can issue thousands of requests after a single lookup. Connection pooling makes that separation even stronger. Even perfectly alternating first addresses in authoritative replies can coexist with highly unequal request counts at the actual servers.
A failed peer remains in an ordinary static RRset until DNS data changes or another system changes it. The DNS server does not learn application health simply by returning the address. Some clients try an alternate address after connection failure; others exhaust their deadline or expose the failure immediately. Retry behavior is a property to verify for the specific client and operation.
Describing the mechanism as a guaranteed coin flip is also inaccurate. Ordering can be configured, randomized, retained by a cache, or changed by address-selection policy. Clients may select the first address, race attempts, or reuse a prior choice. An observed distribution therefore needs the DNS configuration, resolver behavior, connection reuse, and workload alongside the server-side request counts.
Check it quickly
Save this as check-dns-addresses.py and run python3 check-dns-addresses.py with Docker. The pinned environment uses BIND 9.18.49 with rrset-order explicitly set to cyclic. Two real HTTP listeners bind separate loopback addresses on the same temporary port inside an isolated container. Their response bodies identify the listener that actually served each request:
from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid
recipe='FROM python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84\nRUN apt-get update && apt-get install -y --no-install-recommends bind9=1:9.18.49-1~deb12u2 bind9-dnsutils=1:9.18.49-1~deb12u2 dnsmasq-base=2.90-4~deb12u2 python3-dnspython=2.3.0-1 && rm -rf /var/lib/apt/lists/*\nCMD ["/usr/bin/python3"]\n'
program=r"""
from contextlib import ExitStack,contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import json,re,signal,subprocess,time
import dns.exception,dns.message,dns.query,dns.rcode,dns.rdatatype
def run(*args,check=True):
result=subprocess.run(args,capture_output=True,text=True,timeout=15)
if check:assert result.returncode==0,(args,result.returncode,result.stdout,result.stderr)
return result
def query(name,kind='A',port=1053):
return dns.query.udp(dns.message.make_query(name,kind),'127.0.0.1',port=port,timeout=1)
def values(response,kind='A'):
wanted=dns.rdatatype.from_text(kind)
return [item.to_text() for rrset in response.answer if rrset.rdtype==wanted for item in rrset]
def show(response):
def section(records):return [{'name':str(rr.name),'type':dns.rdatatype.to_text(rr.rdtype),'ttl':rr.ttl,'data':[item.to_text() for item in rr]} for rr in records]
return {'rcode':dns.rcode.to_text(response.rcode()),'answer':section(response.answer),'authority':section(response.authority)}
def wait(predicate,timeout=15):
deadline=time.monotonic()+timeout
while time.monotonic()<deadline:
try:
if predicate():return
except (OSError,dns.exception.Timeout):pass
time.sleep(.05)
raise AssertionError('DNS fixture condition did not become true')
def stop(process):
if process.poll() is None:
process.terminate()
try:process.wait(timeout=5)
except subprocess.TimeoutExpired:process.kill();process.wait(timeout=5)
@contextmanager
def lab(records,soa_ttl=8,minimum=4,extra=''):
with TemporaryDirectory(prefix='xbcatm-dns-') as directory,ExitStack() as stack:
root=Path(directory);zone=root/'zone';conf=root/'named.conf';log=root/'named.log';serial=1
def save(body):
zone.write_text(f'$ORIGIN lab.test.\n$TTL 30\n@ {soa_ttl} IN SOA ns.lab.test. hostmaster.lab.test. ({serial} 60 60 3600 {minimum})\n@ IN NS ns.lab.test.\nns IN A 127.0.0.1\n'+body+'\n')
run('named-checkzone','lab.test',str(zone))
save(records)
conf.write_text(f'''options {{ directory "{root}"; listen-on port 1053 {{127.0.0.1;}}; listen-on-v6 {{none;}};
recursion no; dnssec-validation no; notify no; querylog yes; max-cache-size 8m; pid-file "{root}/named.pid"; session-keyfile none;
{extra} }};
controls {{}};
zone "lab.test" {{ type primary; file "{zone}"; }};
''')
run('named-checkconf',str(conf))
output=stack.enter_context(log.open('w'))
authority=subprocess.Popen(['named','-g','-c',str(conf),'-n','1','-U','1'],stdout=output,stderr=subprocess.STDOUT)
stack.callback(stop,authority)
def ready():
assert authority.poll() is None,log.read_text()
response=query('lab.test.','SOA');return response.rcode()==0 and response.answer[0][0].serial==serial
wait(ready)
def update(body):
nonlocal serial
serial+=1;save(body);authority.send_signal(signal.SIGHUP);wait(ready)
def cache(port=1054,upstream=1053,size=150,options=()):
logfile=root/('cache-'+str(port)+'.log');stream=stack.enter_context(logfile.open('w'))
command=['dnsmasq','--no-daemon','--conf-file=/dev/null','--no-resolv','--no-hosts','--bind-interfaces',
'--listen-address=127.0.0.1','--port='+str(port),'--server=127.0.0.1#'+str(upstream),
'--cache-size='+str(size),'--log-queries','--log-facility=-',*options]
process=subprocess.Popen(command,stdout=stream,stderr=subprocess.STDOUT);stack.callback(stop,process)
def cache_ready():
assert process.poll() is None,logfile.read_text()
return query('lab.test.','SOA',port).rcode()==0
wait(cache_ready);return process,logfile
def queries(name,kind='A'):
return sum(bool(re.search(r'query: '+re.escape(name.rstrip('.'))+r' IN '+kind+r'\b',line)) for line in log.read_text().splitlines())
yield root,update,cache,queries
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
import http.client,threading
class Handler(BaseHTTPRequestHandler):
def log_message(self,*args):pass
def do_GET(self):
body=self.server.server_address[0].encode();self.server.hits+=1
self.send_response(200);self.send_header('Content-Length',str(len(body)));self.end_headers();self.wfile.write(body)
def request(address,port):
connection=http.client.HTTPConnection(address,port,timeout=.5)
try:
connection.request('GET','/');response=connection.getresponse();assert response.status==200;return response.read().decode()
finally:connection.close()
with lab('pool 30 IN A 127.0.0.2\npool 30 IN A 127.0.0.3',extra='rrset-order { order cyclic; };') as (_,update,cache,queries):
servers=[];threads=[]
try:
for address in ['127.0.0.2','127.0.0.3']:
server=ThreadingHTTPServer((address,0 if not servers else servers[0].server_port),Handler);server.hits=0
thread=threading.Thread(target=server.serve_forever,daemon=True);thread.start();servers.append(server);threads.append(thread)
port=servers[0].server_port;orders=[values(query('pool.lab.test.')) for _ in range(6)]
assert all(set(order)=={'127.0.0.2','127.0.0.3'} for order in orders)
assert {tuple(order) for order in orders}=={('127.0.0.2','127.0.0.3'),('127.0.0.3','127.0.0.2')}
saved=values(query('pool.lab.test.'));selected=saved[0]
hits=[request(selected,port) for _ in range(12)];assert hits==[selected]*12
distribution={s.server_address[0]:s.hits for s in servers};assert sorted(distribution.values())==[0,12]
victim=next(s for s in servers if s.server_address[0]==selected);victim.shutdown();victim.server_close()
still_published=values(query('pool.lab.test.'));assert selected in still_published
try:request(selected,port);raise AssertionError('stopped HTTP server accepted a connection')
except OSError as error:failure=type(error).__name__
alternate=next(a for a in saved if a!=selected);assert request(alternate,port)==alternate
print(json.dumps({'cyclic_answer_orders':orders,'one_lookup_twelve_requests':distribution,'after_peer_stopped':still_published,
'selected_address_error':failure,'explicit_alternate_succeeded':alternate},sort_keys=True))
finally:
for server in servers:server.shutdown();server.server_close()
for thread in threads:thread.join(timeout=2)
"""
name='xbcatm-dns-'+uuid.uuid4().hex[:10]
def docker(*args,**kwargs):return subprocess.run(['docker',*args],capture_output=True,text=True,timeout=300,**kwargs)
try:
with TemporaryDirectory(prefix=name+'-') as directory:
root=Path(directory);(root/'Dockerfile').write_text(recipe)
built=docker('build','-t',name,str(root));assert built.returncode==0,built.stderr
result=docker('run','--rm','-i','--name',name,'--network','none','--memory','384m','--pids-limit','128',name,
'/usr/bin/python3','-',input=program)
assert result.returncode==0,result.stderr
print(json.dumps(json.loads(result.stdout),sort_keys=True))
finally:
docker('rm','-f',name);docker('image','rm',name)
Six direct queries for pool.lab.test return the same two addresses, 127.0.0.2 and 127.0.0.3, in alternating order in the recorded run. Every answer includes the complete set. The assertion checks both membership and observed order changes. This establishes the configured behavior of this authoritative server; it does not claim that all DNS servers or recursive caches must preserve that order.
The example then performs one additional lookup, saves the returned list, and uses its first address for twelve HTTP requests. All twelve reach that selected listener, producing a 12:0 distribution. Each request establishes its own HTTP connection, so this imbalance does not require connection pooling. Retaining one DNS selection across multiple connections is enough to separate answer rotation from request balancing.
Next, the selected listener is stopped and its socket closed. A fresh authoritative DNS query still includes its address because the zone has not changed. A connection to that peer fails with ConnectionRefusedError in the recorded environment. The program explicitly selects the other saved address and receives a successful response, demonstrating an available alternate without attributing that choice to DNS itself.
Choose the balancing and failure contract explicitly
If simple address distribution is sufficient, keep all advertised peers capable of serving the hostname and the expected workload. Monitor each peer directly as well as the aggregate name. A successful request through one address cannot establish the health of every other advertised address, especially when the client silently tries alternatives and hides the first failure from the caller.
If a DNS provider removes unhealthy addresses, account for the separate detection delay and previously cached answers. Health-driven DNS changes improve future discovery but do not revoke old answers or move established connections. Keep a defined client retry policy and migration overlap where required. Lowering TTL changes the caching tradeoff; it does not turn DNS into an application-level scheduler.
When per-request routing, capacity weights, or bounded failover behavior is required, evaluate an appropriate load balancer and its actual health and retry settings. That choice brings its own operational responsibilities. The useful comparison is between observable contracts, not between a supposedly random DNS mechanism and a supposedly infallible proxy. Measure which peer served requests and what users experienced when one peer stopped.
Finally, remember that same-name, same-type address records form an RRset and should share one TTL. Assigning different lifetimes to individual members is not a sound way to express weights. Use a supported policy with defined semantics, and validate distribution under realistic resolver sharing and connection reuse before making promises about how evenly traffic will spread.
Sources
- RFC 2181 DNS clarifications, complete RRsets and equal TTLs within a set, checked 2026-09-12.
- BIND 9.18.49 reference, rrset-order and configured cyclic answer order, checked 2026-09-12.
- RFC 1034 domain names, resolver behavior and cached resource records, checked 2026-09-12.
