A lock with many keys: Spoofing DNSSEC-signed domains in 8.8.8.8

DNSSEC background

The Domain Name System (DNS) was not designed with security in mind. That was demonstrated by the noteworthy Kaminsky attack in 2008. Back then, Dan Kaminsky demonstrated that an off-path attacker could insert malicious responses into the cache of a recursive resolver. Such an attack can have a severe impact. For example, every client of a recursive resolver could be directed to a malicious domain name. In response to Kaminsky's finding, the deployment of DNS Security Extensions (DNSSEC) gained momentum. With DNSSEC, domain name operators can cryptographically sign the contents of their DNS zones. A recursive resolver can in turn validate the cryptographic signatures, enabling it to detect cache poisoning attacks and protect users against them. Today, many DNS operators sign their domain names, and an increasing number of recursive resolvers validate DNSSEC records, thereby protecting millions of users against cache poisoning attacks. The technology is widely used in .nl, see for instance https://stats.sidnlabs.nl/en/dnssec.html#validated%20queries. Google's Public DNS service, better known by its IPv4 address 8.8.8.8, also validates DNSSEC records. Currently, 8.8.8.8 is responsible for more than 15% of all DNS queries received by the .nl name servers.

The vulnerability

We found an attack vector with which an adversary could have theoretically tricked Google's resolvers into accepting poisoned DNS responses regarding DNSSEC-signed domain names. If an attacker had succeeded in poisoning Google's resolver cache with unsigned DNS responses, they could have then poisoned signed domain names as well. The actual flaw lay in the way Google Public DNS handled unknown DNSSEC keys. Basically, offering any cryptographic key would have convinced Google Public DNS to accept a fake response.

Validation under normal circumstances

Under normal circumstances, before validating a signature, a recursive resolver must be sure that it was created with a valid key. Otherwise, the signature must be considered 'bogus', meaning that the response cannot be trusted. The resolver should then return a SERVFAIL error code to the querying client. The way that a resolver should validate DNSSEC records is illustrated below using our own domain name sidnlabs.nl and the DNSSEC debugging tool DNSViz (see Figure 1). In the example, the records for sidnlabs.nl are signed with a Zone Signing Key (ZSK) with key ID 20853, which in turn is signed with the Key Signing Key (KSK) with ID 52720. The latter key is linked to the .nl zone through the DS record, building a chain of trust. After receiving the AAAA record for sidnlabs.nl and the corresponding signatures, a validating resolver fetches the keyset containing the ZSK and KSK.

Figure 1: The DNSSEC-signed domain name sidnlabs.nl, as seen in DNSViz. The resolver should validate the keyset it receives, i.e. check whether it is correct. If it is, then the resolver proceeds to validate the AAAA record. It uses the key ID specified in the signature on the AAAA record to identify the corresponding key. If it finds the key is valid, the resolver uses the key to validate the signature of the AAAA record and returns the record to the client.

What went wrong at Google Public DNS?

Figure 2: Misconfigured domain name servfail.nl, as seen in DNSViz. The resource records are signed with a non-existent Zone Signing Key.

Google Public DNS was vulnerable in the last stage of the validation process. We discovered the vulnerability more or less by accident. We maintain the domain name servfail.nl, which is deliberately misconfigured for DNSSEC for testing and debugging purposes. DNSSEC can be misconfigured in many ways. Originally, servfail.nl had expired signatures. However, for reasons outside the scope of this article, we decided to change the configuration, so that servfail.nl's AAAA record is now signed with an unknown key. Figure 2 shows the current configuration of servfail.nl, where its records are signed with the key with ID 45918. However, if you look closely, the keyset for servfail.nl consists of only the KSK with ID 15438 and the ZSK with ID 45916. As DNSViz highlights in the figure, the misconfiguration results in servfail.nl's other records being regarded as bogus (as indicated by the red frames), meaning that they cannot be validated. A validating resolver looking up the AAAA record for servfail.nl won't be able to find and validate the ZSK with ID 45918 and should therefore return the error code SERVFAIL to its clients. That is what resolvers with correctly implemented DNSSEC validation do: If we lookup the AAAA record with a validating resolver such as Quad9 it returns the error code SERVFAIL, as shown in Figure 3.

Figure 3: DNS query regarding the AAAA record for servfail.nl, sent using the Quad9 validating resolver.

Unfortunately, that is not how the Google Public DNS resolvers behaved. After querying the AAAA record for servfail.nl using 8.8.8.8, we received the actual AAAA record (see Figure 4) instead of the error code.

Figure 4: DNS query regarding the AAAA record for servfail.nl, sent using the Google Public DNS validating resolvers.

Note also that 8.8.8.8 did not return the AD flag set, indicating that the response had not been authenticated using DNSSEC.

Consequently, not only would Google's resolvers have failed to notice any misconfiguration where a record was signed with an unknown key, but the resolvers would have been vulnerable to downgrade attacks. An attacker, able to insert spoofed DNS record into Google's Public DNS cache, could also have even spoofed DNSSEC-signed records, simply by pretending that the signatures on the spoofed records were created using keys that don't exist.

Potential impact

The potential impact of spoofing DNSSEC-signed records would have been more severe than spoofing unsigned records. A response relating to an unsigned record cannot, be definition, be validated and should therefore always be treated as untrustworthy. By contrast, if a validating resolver receives a record for a signed domain name, it assumes that the answer is correct and that the information provided is trustworthy. The vulnerability we discovered undermined that trustworthiness.

The practical impact of the vulnerability is harder to assess. Since the Kaminsky attack, several other ways of increasing the chance of an attacker poisoning a resolver's cache have been published [1, 2]. Nevertheless, it is not clear to us how easy it was to insert spoofed responses into the Google Public DNS caches. Also, for ethical reasons, we did not actively attempt to exploit the vulnerability. Google Public DNS uses an elaborate caching architecture, likely making cache poisoning harder than with small-scale recursive resolvers.

Fix and checks beyond Google

We notified Google about the bug on 12 January this year, and they deployed fixes to all their instances by 23 February. Google's resolvers now also return SERVFAIL for servfail.nl. Google themselves are confident that their various protections make it extremely unlikely that a cache poisoning attack would succeed. We also wanted to understand whether other resolver implementations besides Google Public DNS might have been vulnerable to the attack. We accordingly reached out to the developers of the DNS software packages BIND, Unbound, Knot Resolver and PowerDNS Recursor. They checked their software and found that none of their resolvers were vulnerable to the attack method described. Finally, we carried out a large-scale DNS measurement, using 10,000 probes in the RIPE Atlas measurement network. We wanted to see whether we could find resolvers that returned the AAAA record of servfail.nl even though they were validating DNSSEC. Like the software developers, we could not find clear evidence of resolvers that could be vulnerable to our attack, other than the Google Public DNS resolvers.

General takeaways

While DNSSEC is crucial for the security of the DNS, it also remains the DNS's most complicated extension. This incident is still further evidence of that. Another outcome was that other developers of recursive resolvers ran into a few non-critical corner cases after we had modified servfail.nl, which were unrelated to the vulnerability at Google. As with other security protocols, we recommend that developers and operators rely on existing, well tested implementations of DNSSEC such as the ldns libray or one of the open-source recursive resolvers rather than 'home-grown' implementations.

Acknowledgements

For reporting this bug, we received $5,000 from Google's bug bounty programme. We will donate the bounty to charity. We thank Google for their open communication and for reviewing this article.

Timeline of events