The basic problem with getaddrinfo() is that it isn’t a DNS API, it’s a name service API, and the name service switch might look up names using /etc/hosts, mDNS, NIS, NETBIOS, who knows what else. Oh and probably DNS too. So the results you get from getaddrinfo() are very much a lowest common denominator.
Applications that depend on DNS specifics should probably pull in a modern stub resolver library, and avoid the ancient BIND-derived res_* functions. There are a couple of problems with this idea:
It might not be possible to get the recursive server IP addresses out of the libc resolver. Implementations have diverged since the BIND 4 days, especially wrt IPv6 support. Some resolver libraries parse /etc/resolv.conf themselves and try to do so in a compatible manner. Check support for fun things like scoped link-local IPv6 addresses.
The system might have a DNS cache that you might accidentally bypass. If that cache provides recursive DNS service on a loopback address it might be shit. You might be better off using a proprietary API instead. (Which is basically what the article concluded.)
The second problem is that it’s synchronous. Looking in hosts is fast and so that’s fine. NETBIOS is probably on the local network, so might be fast. mDNS is link-local and mostly served from a local cache. DNS may miss in a local cache and require a chain of synchronous fetches from DNS servers somewhere in the world, which may take hundreds of ms.
If you want to do one name lookup, that’s fine. If you want to do more, that’s a problem because the only way to overlap them is to have a separate thread for each.
Edit: this is a special case of the problem that a lot of transparent RPC systems encounter. It’s really hard to write software with performance that you can reason about when the latency of a call may vary by 3-4 orders of magnitude.
Applications that depend on DNS specifics should probably pull in a modern stub resolver library
I agree but I don’t think there is one, at least in C land your options are:
dns.c which is a suspiciously large amount of code for a not very complicated protocol (10k LOC), has a difficult API, and hasn’t been updated in 10 years
c-ares is a ridiculous amount of code (50k LOC)
rip out musl’s resolver, it’s not a lot of code but it’s split across 400 files and it all looks like this
It might not be possible to get the recursive server IP addresses out of the libc resolver.
The system might have a DNS cache that you might accidentally bypass.
i wouldn’t mind systems (the OS) just providing a (good!) recursive resolver on the loopback ip and listing it in /etc/resolv.conf. in other words: the dns protocol is the api.
then everyone can use their own stub resolver library with an api of their choice. the OS can do “smart things” in its recursive resolver (like redirect requests for internal domains to some other internal name server; macos has some config options for this in /etc that i have enjoyed but haven’t seen elsewhere; the system resolver could also do doh, etc). these stub resolver libraries may also look at /etc/hosts, but the system loopback resolver could even serve records based on that file too (and hopefully from a new file in /etc in zone syntax). going through a loopback resolver prevents the issue mentioned in the article with caching responses.
btw, is the /etc/resolv.conf syntax still updated at times? so stub resolvers could make use of modern features (i’m thinking of do[thq]).
it’s a name service API, and the name service switch might look up names using /etc/hosts, mDNS, NIS, NETBIOS, who knows what else
i haven’t used getaddrinfo much, certainly not recently. nowadays i (pretend to) live in a Go world, using the pure go resolver (https://pkg.go.dev/net#hdr-Name_Resolution), without going through the system libc resolver. works well for me. i’ve never expected name resolution to return names from mdns/nis/netbios, and i’ve never knowingly used nsswitch (perhaps i’m missing something great?). unfortunately, the current standard library go resolver api has the same issues as mentioned in the article with not providing access to the ttl, or the dnssec status, or the cnames followed (and their dnssec statusses).
systemd-resolved also exposes a varlink API, which isn’t too bad to work with (though I think systemd-resolved is pretty much the only application out there using varlink).
Apple’s documentation really really sucks! I couldn’t find any examples of this function being used on official Apple sites
Thats because the <dns_sd.h> API is quite low-level, and nearly any Mac/iOS code would use the friendlier wrappers in the Foundation (NSNetService) or Network frameworks.
DNSServiceQueryRecord Is kind of an outlier in dns_sd — most of that API is for mDNS and DNS-SD (aka Bonjour). I wasn’t aware this function could be used as a general purpose DNS server query.
What isn’t clear? I wrote that it uses regular DNS and the web page I linked to says in its first sentence “DNS Service Discovery is a way of using standard DNS programming interfaces, servers, and packet formats to browse the network for services.” This isn’t a hypothetical proposal, it has been deployed for many years.
well, to me this very explicitly means “this is the same API & protocol than normal DNS but won’t allow you to resolve google.com but rather select services relevant to your organization”.
Hybrid Unicast/Multicast DNS-Based Service Discovery (draft-cheshire-mdnsext-hybrid) describes a way to provide wide-area service discovery for devices that only advertise their services using link-local Multicast DNS.
Hybrid DNS-SD is a way to reduce the traffic associated with mDNS in large multicast networks. It uses wide-area DNS-SD but the existence of hybrid DNS-SD doesn’t imply that wide-area DNS-SD is restricted to local services.
Wide-area DNS-SD uses regular DNS but adds a layer on top to support browsing for services. If you are using a DNS-SD API then you will only be able to discover services that have service discovery records in the DNS, which probably doesn’t include google.com.
But if you are using a lower-level DNS API like DNSServiceQueryRecord() that is described as “Query for an arbitrary DNS record” then you can use it to resolve google.com.
The basic problem with getaddrinfo() is that it isn’t a DNS API, it’s a name service API, and the name service switch might look up names using /etc/hosts, mDNS, NIS, NETBIOS, who knows what else. Oh and probably DNS too. So the results you get from getaddrinfo() are very much a lowest common denominator.
Applications that depend on DNS specifics should probably pull in a modern stub resolver library, and avoid the ancient BIND-derived
res_*functions. There are a couple of problems with this idea:It might not be possible to get the recursive server IP addresses out of the libc resolver. Implementations have diverged since the BIND 4 days, especially wrt IPv6 support. Some resolver libraries parse /etc/resolv.conf themselves and try to do so in a compatible manner. Check support for fun things like scoped link-local IPv6 addresses.
The system might have a DNS cache that you might accidentally bypass. If that cache provides recursive DNS service on a loopback address it might be shit. You might be better off using a proprietary API instead. (Which is basically what the article concluded.)
Yeah, it’s a mess.
The second problem is that it’s synchronous. Looking in hosts is fast and so that’s fine. NETBIOS is probably on the local network, so might be fast. mDNS is link-local and mostly served from a local cache. DNS may miss in a local cache and require a chain of synchronous fetches from DNS servers somewhere in the world, which may take hundreds of ms.
If you want to do one name lookup, that’s fine. If you want to do more, that’s a problem because the only way to overlap them is to have a separate thread for each.
Edit: this is a special case of the problem that a lot of transparent RPC systems encounter. It’s really hard to write software with performance that you can reason about when the latency of a call may vary by 3-4 orders of magnitude.
I agree but I don’t think there is one, at least in C land your options are:
I was thinking of something like adns, ldns, getdns
There’s a lot of fiddly parsing in a DNS implementation because of the many different RR types, so it’s hard to make it both small and easy to use.
Also, that musl code appears to have completely incorrect domain name parsing. musl’s stub resolver is very bad.
libunbound is just shy of 3KLOC:
i wouldn’t mind systems (the OS) just providing a (good!) recursive resolver on the loopback ip and listing it in /etc/resolv.conf. in other words: the dns protocol is the api. then everyone can use their own stub resolver library with an api of their choice. the OS can do “smart things” in its recursive resolver (like redirect requests for internal domains to some other internal name server; macos has some config options for this in /etc that i have enjoyed but haven’t seen elsewhere; the system resolver could also do doh, etc). these stub resolver libraries may also look at /etc/hosts, but the system loopback resolver could even serve records based on that file too (and hopefully from a new file in /etc in zone syntax). going through a loopback resolver prevents the issue mentioned in the article with caching responses.
btw, is the /etc/resolv.conf syntax still updated at times? so stub resolvers could make use of modern features (i’m thinking of do[thq]).
i haven’t used getaddrinfo much, certainly not recently. nowadays i (pretend to) live in a Go world, using the pure go resolver (https://pkg.go.dev/net#hdr-Name_Resolution), without going through the system libc resolver. works well for me. i’ve never expected name resolution to return names from mdns/nis/netbios, and i’ve never knowingly used nsswitch (perhaps i’m missing something great?). unfortunately, the current standard library go resolver api has the same issues as mentioned in the article with not providing access to the ttl, or the dnssec status, or the cnames followed (and their dnssec statusses).
This is how most local (usually caching) resolvers work. unbound, systemd-resolved, etc…
I agree that it would be good to make it a “standard” such that it is always expected to be configured.
Another option on Linux may also be to query systemd-resolved via its native API.
systemd-resolved also exposes a varlink API, which isn’t too bad to work with (though I think systemd-resolved is pretty much the only application out there using varlink).
Thats because the <dns_sd.h> API is quite low-level, and nearly any Mac/iOS code would use the friendlier wrappers in the Foundation (NSNetService) or Network frameworks.
DNSServiceQueryRecord Is kind of an outlier in dns_sd — most of that API is for mDNS and DNS-SD (aka Bonjour). I wasn’t aware this function could be used as a general purpose DNS server query.
.. isn’t this function only for DNS-SD ? (e.g. Bonjour / mDNS / local DNS resolving, not anything internet-related).
There is wide-area DNS-SD which uses the regular DNS instead of mDNS. http://www.dns-sd.org/
Hm, it’s not clear to me if wide-area DNSSD would be regulars DNS or a completely different service
What isn’t clear? I wrote that it uses regular DNS and the web page I linked to says in its first sentence “DNS Service Discovery is a way of using standard DNS programming interfaces, servers, and packet formats to browse the network for services.” This isn’t a hypothetical proposal, it has been deployed for many years.
well, to me this very explicitly means “this is the same API & protocol than normal DNS but won’t allow you to resolve google.com but rather select services relevant to your organization”.
Hybrid DNS-SD is a way to reduce the traffic associated with mDNS in large multicast networks. It uses wide-area DNS-SD but the existence of hybrid DNS-SD doesn’t imply that wide-area DNS-SD is restricted to local services.
Wide-area DNS-SD uses regular DNS but adds a layer on top to support browsing for services. If you are using a DNS-SD API then you will only be able to discover services that have service discovery records in the DNS, which probably doesn’t include google.com.
But if you are using a lower-level DNS API like DNSServiceQueryRecord() that is described as “Query for an arbitrary DNS record” then you can use it to resolve google.com.