"Can we do without IPv4 yet?" News of our annual IPv6-only experiment

What is the IPv6-only experiment?

With hindsight, it's easy to see ways that the development of IPv6 could have been handled better. However, the engineers who thought of and developed 'IPng', as it was originally known, in the early 1990s simply didn't know everything we know now. As a result, things didn't go quite as everyone hoped. The transition from old to new proved particularly problematic. It became a long, drawn-out process, which has only recently really gathered speed. So, although migration is now progressing quickly, it's safe to assume that IPv4 is going to be around for a while yet, alongside IPv6. The reason being that IPv4 is deeply embedded in all manner of software and network infrastructures, not to mention the heads of developers and engineers. Nevertheless, the ultimate goal is for IPv4 and IPv6 to be disentangled, and for the internet to be usable without IPv4.

We periodically run an IPv6-only experiment with the aim of investigating how we're progressing towards that goal. Where do we currently stand and what problems do we encounter, despite using software that claims to be 'IPv6-ready', if we completely disable IPv4? It has to be acknowledged that our first IPv6-only experiments didn't take long. Technically speaking, it more or less worked, but any attempt to do anything practical didn't get far. It wasn't much fun trying one thing after another and seeing them fail. The software often didn't work and, if it did, something else was bound to go wrong. For the most part, only sites operated by the 'usual suspects' actually worked. Over the years, however, things have improved. We've now reached a point where the experiments contrast sharply with the disappointments of the past. Our experiments are conducted from the user's perspective. We don't check out every one of the 2.1 million or more .nl domain names that claim one way or another to be reachable using IPv6. The approach is more of a random sampling exercise, designed to give us a sense of the quality of the internet experience if we disable IPv4, and to flag up the areas where there's scope for improvement.

Research method

Our experiment has always been done the same way. However, over the years, we've gradually raised the bar. So, whereas we began by simply configuring an ethernet interface without an IPv4 address, possibly assisted by a NAT64/DNS64 set-up so that we could at least get somewhere, we now take things a little further. Because, with the exception of a few issues, everything now works, providing that we have a NAT64/DNS64 bridge to the IPv4 world. The novelty of that wore off a good while ago. Our focus nowadays is therefore mainly on seeing what happens if you really do without IPv4 altogether. Not because we imagine that doing without IPv4 would be a sensible move for the average internet user, but purely to satisfy our curiosity. The experiments are done using a desktop system that doesn't support IPv4 at all. For that, we're indebted to the good people behind the FreeBSD project, whose open-source operating system enables the 'kernel' to be compiled without the IPv4 stack. As far as we're aware, it's the only OS with that option. With FreeBSD installed and configured, we fire up the desktop. In the interests of thoroughness, we then make a few more modifications. For example, we delete the 127.0.0.1 statement from /etc/hosts, even though that's unlikely to be problematic. We also change the default 0.freebsd.pool.ntp.org setting in ntp.conf to 2.freebsd.pool.ntp.org, because sadly it's the only node that supports IPv6. Other similarly minor changes are made, so that we really are completely without IPv4. 'Hidden' IPv4 dependencies can take some finding. For example, avahi-daemon.conf has a line that reads 'use-ipv4=yes', which we changed to 'no'. Such things make very little difference when there is no IPv4 stack, but finding them becomes something of a treasure hunt. And 'well done' to the developers for even including that option. Because we can, we also configure the e-mail, so that we can receive mail from the system. Configuring mail for IPv6 is easy enough with the big mail service provider we use. Otherwise, our machine is running a conventional 13.0 release of FreeBSD, including Gnome3 and/or the XFce4 graphical desktop. On the OS, we instal various pieces of standard software, such as the Firefox and Chrome web browsers.

Figure 1: An 'ifconfig' of our test system.

One new feature of our latest experiment was that we used a DNS resolver that supports only IPv6 communication with the outside world. And, although that provided us with a few surprises, it was amazing that we could permit ourselves that indulgence without immediately becoming mired in functional issues. We took that as further confirmation that IPv6 adoption is definitely heading in the right direction. I should add that the resolver configuration was achieved using the 'local_unbound' application supplied with FreeBSD. All very quick and easy. However, we did have to tell the application to connect to ::1 instead of the default 127.0.0.1. The latter arrangement shouldn't be confused with something else we have sometimes done. See, for example, https://42.dnslabs.nl/ (if it works). That page will appear only if both your connection and the resolvers you're using support IPv6. That's not something you would necessarily consider, but can be a source of trouble.

Findings

First of all, we found that the set-up was technically functional and mature. We spotted nothing serious in the logs. That was actually what we were expecting on the basis of our experiments with NAT64/DNS64 (my personal workstation runs entirely on that set-up) and, for example, earlier Apple announcements. However, the next question was of course whether our technically functional set-up could actually be used to do anything practical on the internet. Most important still: where can things still be improved? Google, YouTube, Netflix, Facebook and Instagram (which are the internet for some users) all worked fine, of course. I say 'of course' because it's generally well-known that those services support IPv6. Even https://web.whatsapp.com was fine. Many Dutch government websites also worked perfectly without the exchange of a single IPv4 packet, including defensie.nl, politie.nl, rivm.nl, and coronadashboard.rijksoverheid.nl. Indeed, so many sites now run smoothly with IPv6 that it's impractical to list them. A good few don't actually support IPv4 at all.

Given the 'use-or-explain' requirements, support for IPv6 by public service providers is only to be expected. However, by no means all IPv6-enabled sites are government sites: the protocol is being used successfully by all manner of site operators. Our incentive scheme [1, 2] and the services of Cloudflare and others may have helped or are helping in that regard. It's clear, and very pleasing, that a lot of things are going well. However, it's perhaps more interesting to consider the things that aren't going well. Let's start with the website of the Dutch national police, politie.nl. At first sight, everything seems to be fine. But an issue lurks below the waterline. Unfortunately, w.usabilla.com doesn't support IPv6. A site user is very unlikely to be troubled by that, but I doubt whether it's to the webmaster's liking. We encountered similar 'underwater' situations elsewhere, including trackers and webmaster tools that don't work with IPv6 (as well as many that do). In most cases, there's no practical consequence for the visitor (in some cases there were actually pluses, such as no ads), but sometimes that's a nuisance. For example, revu.nl has contracted out its prominent cookie wall to privacymanager.io, who don't seem to support IPv6. So, although revu.nl seemed to be IPv6-enabled, it wasn't possible to get past the cookie wall without IPv4:

Figure 3: An IPv4-only cookie wall on revu.nl stopped us accessing the website.

Next, we tried LinkedIn, where we encountered a problem we hadn't seen before. On the face of it, the site worked fine with IPv6. However, because we took a strict approach that involved IPv6 DNS resolving, our particular set-up hit a snag. The issue being that, for us, 'www.linkedin.com' is a CNAME for 'www-linkedin-com.l-0005.l-msedge.net', Microsoft's CDN, and both authoritative name servers for the 'l-msedge.net' domain are IPv4-only. We have no idea why, because the arrangement serves no purpose that we can think of. We had a similar problem with Bing: it works fine with IPv6, but the name servers for 'trafficmanager.net' don't support IPv6. Similarly, openstreetmap.org works with Fastly's CDN, whose name servers can't be reached using IPv6. Such DNS issues are a real shame, because the sites themselves are fully IPv6-enabled.

What else did we discover?

A number of websites, including Dutch broadcaster NOS's site, were accessible but didn't look the way they should. On close inspection, the explanation usually turned out to be that some content elements wouldn't load without IPv4. In the case of NOS, for example, components such as 'cdn.nos.nl' turned out not to be IPv6-enabled:

Figure 4: The NOS website was missing a lot of content when accessed with an IPv6-only set-up.

Common problems included missing CSS style sheets and non-functional video content. For example, we had no difficulty browsing the site of the Dutch daily newspaper AD, but none of the site's video content would play. It was disappointing to come across sites that only half work with IPv6. The technical team at the City of Nijmegen were equally disappointed to hear about the issues affecting their site, which they proceeded to fix inside twenty-four hours!

When we tried cross-referencing, we found that some sites don't show the same content to IPv6 visitors as to IPv4 visitors. One example being a standard Apache page viewed with IPv6.

Geo-location issues were encountered several times.

Ironically, we found just such an issue on the otherwise excellent site https://ipv6-test.com/. Their speed test utility tries to find the nearest test server, but in our experiment that failed because the APIs supporting the 'db-ip.com' service are IPv4-only. So we couldn't get any further.

One final finding worth mentioning is that many users type domain names into their browsers without the 'www', as in 'example.nl'. At present, that often yields a redirect to 'www.example.nl', and often from 'http' to 'https'. Unfortunately, the redirects for some sites didn't work for us. So, while the sites themselves were fine, we did have to type in the whole URL, including the 'www'. Otherwise, the sites didn't load. That was the case with ‘tno.nl’, ‘buienradar.nl’ and ‘knmi.nl’., for example. Such issues should be technically straightforward to resolve, which begs the question why they haven't already been fixed. Perhaps they've simply been overlooked.

Conclusions

We investigated the quality of websites and services that claim to support IPv6, by trying them without the 'aid' of IPv4. Our central finding is that the situation is now much better than what we encountered in our previous experiments. What's more, the residual problems appear to be readily resolvable. They don't look like complex technical issues, but simple oversights -- things that have gone unnoticed on account of most IPv6 connections being dual-stack, with IPv6 and IPv4 running side by side.

Overall, this was the most successful IPv6-only test we've ever done. It's very encouraging to see first hand how mature and functional IPv6 now is, and how quickly content is becoming accessible to IPv6 users. Many sites and services [1, 2] besides those mentioned in this blog can now be accessed without using IPv4.

Of course, a few minor issues remain, which one only really notices when using an IPv6-only set-up. Where possible, we report such issues to the developers or companies in question. It must be acknowledged that there are some more fundamental problems out there too. It's disappointing, for example, that even after all this time major networks such as Twitter and GitHub (to name but two) are still only partially IPv6-enabled. Evidence that our experiment was more than an agreeable way to occupy a Friday afternoon can be found in, for example, official US and Chinese government publications. Having previously announced its intention to work towards an 'IPv4-free' internet, the American government recently began making good that commitment. Maybe the website chosen isn't the most exciting, but it serves as a very clear signal of intent:

Figure 7: Evidence of the US government's commitment to an IPv4-free internet: https://clintonwhitehouse2.archives.gov/.

It's pleasing to note that more and more users in the Netherlands are still able to visit sites that go IPv4-free. That's because providers such as Ziggo and KPN have been rolling out IPv6 to increasing numbers of their customers. We advise anyone involved in this field, including web developers, to run IPv6-only tests on their products before putting them into production. Such tests are easy to set up. Finally, we hope that everyone who reads this blog is, like us, inspired by the knowledge that, albeit after a very long wait, the migration to IPv6 is going well and gathering pace. And we're already looking forward to our next IPv6-only experiment and what it reveals. So, if you've got any ideas or advice, do let us know!