A hostname lookup starts with the program making it. On Linux, an application using glibc name-service interfaces can consult the services listed for hosts in /etc/nsswitch.conf. A program with its own DNS client may take a different path. There is no universal sequence in which every application checks a cache, then a hosts file, then the internet.

When it breaks

A hosts-file entry can override a correct DNS answer when files precedes dns. Editing the authoritative zone does not remove that override. Conversely, a direct DNS query can report a new address while an application still uses the local entry. Both observations can be accurate because they answer different questions about where the address came from.

NSS actions matter as much as order. A clause such as [NOTFOUND=return] after files stops lookup when the file service finds no entry. DNS can contain a valid record and still never be consulted. Treating that failure as an upstream outage leads to unnecessary DNS changes while leaving the actual lookup policy untouched.

Address-family filtering creates another misleading disagreement. With AI_ADDRCONFIG, glibc considers configured non-loopback addresses when deciding which families to return. A container containing only loopback interfaces can therefore behave differently from a normal host. Even two tools using libc need not request identical families or flags, so matching their command names is insufficient evidence of equivalent lookups.

Check it quickly

Save this as check-resolution.py and run python3 check-resolution.py with Docker available. It builds the pinned Debian-based environment with BIND 9.18.49, dnsmasq 2.90, and glibc 2.36. The disposable container has only loopback networking. Hosts, NSS, and resolver changes happen inside that container; the host’s configuration is not used as the experiment’s writable state:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid
recipe='FROM python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84\nRUN apt-get update && apt-get install -y --no-install-recommends bind9=1:9.18.49-1~deb12u2 bind9-dnsutils=1:9.18.49-1~deb12u2 dnsmasq-base=2.90-4~deb12u2 python3-dnspython=2.3.0-1 && rm -rf /var/lib/apt/lists/*\nCMD ["/usr/bin/python3"]\n'
program=r"""
from contextlib import ExitStack,contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import json,re,signal,subprocess,time
import dns.exception,dns.message,dns.query,dns.rcode,dns.rdatatype

def run(*args,check=True):
    result=subprocess.run(args,capture_output=True,text=True,timeout=15)
    if check:assert result.returncode==0,(args,result.returncode,result.stdout,result.stderr)
    return result
def query(name,kind='A',port=1053):
    return dns.query.udp(dns.message.make_query(name,kind),'127.0.0.1',port=port,timeout=1)
def values(response,kind='A'):
    wanted=dns.rdatatype.from_text(kind)
    return [item.to_text() for rrset in response.answer if rrset.rdtype==wanted for item in rrset]
def show(response):
    def section(records):return [{'name':str(rr.name),'type':dns.rdatatype.to_text(rr.rdtype),'ttl':rr.ttl,'data':[item.to_text() for item in rr]} for rr in records]
    return {'rcode':dns.rcode.to_text(response.rcode()),'answer':section(response.answer),'authority':section(response.authority)}
def wait(predicate,timeout=15):
    deadline=time.monotonic()+timeout
    while time.monotonic()<deadline:
        try:
            if predicate():return
        except (OSError,dns.exception.Timeout):pass
        time.sleep(.05)
    raise AssertionError('DNS fixture condition did not become true')
def stop(process):
    if process.poll() is None:
        process.terminate()
        try:process.wait(timeout=5)
        except subprocess.TimeoutExpired:process.kill();process.wait(timeout=5)

@contextmanager
def lab(records,soa_ttl=8,minimum=4,extra=''):
    with TemporaryDirectory(prefix='xbcatm-dns-') as directory,ExitStack() as stack:
        root=Path(directory);zone=root/'zone';conf=root/'named.conf';log=root/'named.log';serial=1
        def save(body):
            zone.write_text(f'$ORIGIN lab.test.\n$TTL 30\n@ {soa_ttl} IN SOA ns.lab.test. hostmaster.lab.test. ({serial} 60 60 3600 {minimum})\n@ IN NS ns.lab.test.\nns IN A 127.0.0.1\n'+body+'\n')
            run('named-checkzone','lab.test',str(zone))
        save(records)
        conf.write_text(f'''options {{ directory "{root}"; listen-on port 1053 {{127.0.0.1;}}; listen-on-v6 {{none;}};
recursion no; dnssec-validation no; notify no; querylog yes; max-cache-size 8m; pid-file "{root}/named.pid"; session-keyfile none;
{extra} }};
controls {{}};
zone "lab.test" {{ type primary; file "{zone}"; }};
''')
        run('named-checkconf',str(conf))
        output=stack.enter_context(log.open('w'))
        authority=subprocess.Popen(['named','-g','-c',str(conf),'-n','1','-U','1'],stdout=output,stderr=subprocess.STDOUT)
        stack.callback(stop,authority)
        def ready():
            assert authority.poll() is None,log.read_text()
            response=query('lab.test.','SOA');return response.rcode()==0 and response.answer[0][0].serial==serial
        wait(ready)
        def update(body):
            nonlocal serial
            serial+=1;save(body);authority.send_signal(signal.SIGHUP);wait(ready)
        def cache(port=1054,upstream=1053,size=150,options=()):
            logfile=root/('cache-'+str(port)+'.log');stream=stack.enter_context(logfile.open('w'))
            command=['dnsmasq','--no-daemon','--conf-file=/dev/null','--no-resolv','--no-hosts','--bind-interfaces',
                '--listen-address=127.0.0.1','--port='+str(port),'--server=127.0.0.1#'+str(upstream),
                '--cache-size='+str(size),'--log-queries','--log-facility=-',*options]
            process=subprocess.Popen(command,stdout=stream,stderr=subprocess.STDOUT);stack.callback(stop,process)
            def cache_ready():
                assert process.poll() is None,logfile.read_text()
                return query('lab.test.','SOA',port).rcode()==0
            wait(cache_ready);return process,logfile
        def queries(name,kind='A'):
            return sum(bool(re.search(r'query: '+re.escape(name.rstrip('.'))+r' IN '+kind+r'\b',line)) for line in log.read_text().splitlines())
        yield root,update,cache,queries
with lab('choice 30 IN A 127.0.0.2\ndns-only 30 IN A 127.0.0.3') as (_,update,cache,queries):
    cache(port=53)
    hosts=Path('/etc/hosts');hosts.write_text(hosts.read_text()+'\n127.0.0.9 choice.lab.test\n')
    Path('/etc/resolv.conf').write_text('nameserver 127.0.0.1\noptions timeout:1 attempts:1\n')
    nss=Path('/etc/nsswitch.conf');original=nss.read_text()
    def order(value):nss.write_text(re.sub(r'^hosts:.*$', 'hosts: '+value,original,flags=re.M))
    def application(name):
        result=run('/usr/bin/python3','-c',
            'import json,socket,sys;print(json.dumps(sorted({v[4][0] for v in socket.getaddrinfo(sys.argv[1],80,socket.AF_INET,socket.SOCK_STREAM)})))',name,check=False)
        return result.returncode,json.loads(result.stdout) if result.returncode==0 else None
    results={};order('files dns')
    assert application('choice.lab.test')==(0,['127.0.0.9'])
    filtered=run('getent','ahostsv4','choice.lab.test',check=False)
    assert filtered.returncode!=0
    libc=run('getent','-A','ahostsv4','choice.lab.test').stdout
    assert {line.split()[0] for line in libc.splitlines()}=={'127.0.0.9'}
    before=queries('choice.lab.test')
    direct=run('dig','@127.0.0.1','-p','1053','choice.lab.test.','A','+short').stdout.strip()
    wait(lambda: queries('choice.lab.test')==before+1)
    assert direct=='127.0.0.2';results['files_first']={'application':['127.0.0.9'],'getent_without_addrconfig':['127.0.0.9'],'dig':direct}
    results['loopback_only_getent_default_exit']=filtered.returncode
    order('dns files');assert application('choice.lab.test')==(0,['127.0.0.2']);results['dns_first']=['127.0.0.2']
    order('files [NOTFOUND=return] dns')
    before=queries('dns-only.lab.test');code,address=application('dns-only.lab.test')
    assert code!=0 and address is None and queries('dns-only.lab.test')==before
    assert values(query('dns-only.lab.test.'))==['127.0.0.3']
    wait(lambda: queries('dns-only.lab.test')==before+1)
    results['stop_after_files_miss']={'application_failed':True,'DNS_would_answer':['127.0.0.3']}
    print(json.dumps(results,sort_keys=True))
"""
name='xbcatm-dns-'+uuid.uuid4().hex[:10]
def docker(*args,**kwargs):return subprocess.run(['docker',*args],capture_output=True,text=True,timeout=300,**kwargs)
try:
    with TemporaryDirectory(prefix=name+'-') as directory:
        root=Path(directory);(root/'Dockerfile').write_text(recipe)
        built=docker('build','-t',name,str(root));assert built.returncode==0,built.stderr
        result=docker('run','--rm','-i','--name',name,'--network','none','--memory','384m','--pids-limit','128',name,
            '/usr/bin/python3','-',input=program)
        assert result.returncode==0,result.stderr
        print(json.dumps(json.loads(result.stdout),sort_keys=True))
finally:
    docker('rm','-f',name);docker('image','rm',name)

The authoritative server publishes choice.lab.test as 127.0.0.2. The container’s hosts file maps the same name to 127.0.0.9. With hosts: files dns, a fresh Python process calling getaddrinfo for IPv4 returns the hosts-file value. A direct dig query to the authoritative server returns the DNS value. Query-log counting confirms that the direct query actually reached that server.

In this loopback-only environment, default getent ahostsv4 exits with status 2. Adding its -A option disables address-configuration filtering and returns 127.0.0.9. The comparison records this failure rather than silently treating an empty tool result as a missing DNS record. The Python call explicitly supplies a family and uses no AI_ADDRCONFIG flag, explaining why it already succeeds.

Changing NSS to dns files makes the application return 127.0.0.2. Changing it again to files [NOTFOUND=return] dns makes lookup of dns-only.lab.test fail. The authoritative query counter remains unchanged during that application lookup, while a subsequent direct DNS query returns 127.0.0.3 and increments the counter. This distinguishes an early local stop from a server that answered negatively.

Inspect the path that the failing process uses

Start with the application’s resolver API, address family, flags, and runtime caching policy. Then inspect the process’s filesystem and network namespace. A container can have different hosts and resolver files from its host; a browser may use its own encrypted DNS service. The successful host-side command is useful context, but it does not establish the failing process’s effective configuration.

Within an NSS path, read the complete hosts line, including bracketed actions and available modules. Within a DNS path, identify the actual destination resolver, search domains, and query type. Use fully qualified names when isolating search-list effects. Query the authority and the recursive resolver separately, retaining response status, answer records, and remaining TTL instead of reducing every response to a single address.

Finally, separate name lookup from connection establishment. An address returned by either source may be unreachable, and an existing connection may keep using an older peer without resolving again. Correct the observed cause: update an unintended hosts entry, repair NSS policy, inspect the relevant cache, or investigate transport. Flushing an unrelated cache cannot change a hosts-file rule or the behavior of a different resolver implementation.

Further reading

Sources