Assessing the security of Internet paths

Cloud-based email services are becoming more dominant

An article published in March 2024 by NOS, one of the mainstream news outlets in the Netherlands, reports that 63 percent of the approximately 3,800 largest companies in the Netherlands rely on Microsoft or Google for cloud-based email services. Critical Infrastructures (CIs) are no exception and many CIs such as ING, KPN (a big telecom operator in the Netherlands) and Schiphol Airport (the main international airport in the Netherlands) rely on cloud services (e.g., email) for their daily operations. SIDN and the University of Twente also use Microsoft's cloud-based email services.

The problem: limited insight into the security of paths towards clouds

At the same time, CIs typically have limited insight into the security status of the paths that their traffic might follow across the Internet to reach the email infrastructure of their cloud provider. For example, a CI might not know their traffic passes through Autonomous Systems (ASes) that do not implement Route Origin Validation (ROV). As a result, the CI’s traffic is vulnerable to prefix hijacks, which can render the cloud operator unavailable to the CI or breach the confidentiality and integrity of the CI’s data. Even if a single AS on a path does not deploy ROV, it can remain vulnerable to BGP hijacks, known as “collateral damage”.

We think of routing hijacks as a supply chain risk for CIs. The risk is real because BGP prefix hijacks are common incidents on the Internet and are well-known to have been used for malicious purposes, for instance, to attack payment systems, steal cryptocurrency, disrupt traffic and create DDoS attacks.

Our question, therefore, is what it takes to enable CIs to get more insight into the security status of each AS on a path towards their cloud provider or other destination.

Our approach: calculate the ROV score of valid Internet paths

We combine BGP routing data collected from public route collectors (RouteViews and RIPE RIS) with ROV scores to assess path security. Using Microsoft as our example cloud-based email service, we go through the following four steps to make such assessments:

1. Path collecting

For a CI’s AS, we look for all its BGP announcements while for Microsoft’s AS we look for BGP announcements from their email service, which has prefix 52.96.0.0/12, originated by Microsoft’s AS (AS8075). In this way, we have two types of paths as seen from the route collectors: from the CI’s AS to the route collectors and from Microsoft’s AS to the route collectors.

2. Path stitching

We create an undirected graph from the two types of paths, in which the nodes are AS Numbers (ASNs) and tuples of ASes on the AS path form the edges. The common node between the two paths is the AS that dumps routing data to the route collectors, which is the stitching point for us to join the two paths.

3. Path sanitizing

We check if a stitched path is valid, meaning the prefix of a source AS (CI) can reach the destination AS (cloud-based email service) and vice versa. We accomplish this through Gao Rexford’s model for route export and valley-free conditions. We first obtain the relationship between ASes on stitched paths using CAIDA AS rank: Customer to Provider (C2P), Provider to Customer (P2C) and Peer to Peer P2P). Next, we select stitched paths that meet the route export condition: routes learned from customers are exported to any providers or peers, routes learned from providers are exported only to customers and routes learned from peers are exported only to customers. Finally, we filter out stitched paths that meet the valley-free condition: at most one P2P link exists in the path, the P2C link must not be followed by a C2P or P2P link and the P2P link must not be followed by a C2P link.

4. Security scoring

We use the ROV scores of ASes on valid paths from the ROVista API to assess the security of an Internet path. ROVista determines the score based on the number of RPKI-invalid prefixes reachable by an AS. The ROV score varies from 0% to 100%, where 0% means no filtering of RPKI-invalid prefixes.

Case study in the Netherlands

We use our method of calculating the ROV score of paths for 4 CIs in the Netherlands (ING bank, water supplier Vitens, Eneco energy and ABN-AMRO Bank), with their destination as the Microsoft email service. Figure 1 shows the number of valid paths for each CI and their security statuses (in terms of ROV scores).

As seen from Figure 1a, AS15625 (ING) has the highest number of paths, which might be because it is a multi-homed AS with four upstream providers, and more providers means more ways that an AS gets BGP routes. In total, there are 40 ROV-unprotected paths (0% ROV in Figure 1a) and 20 fully ROV-protected paths (100% ROV) for path lengths of 1, 2 and 3 AS hops.

For the case of AS56517 (Figure 1b), there are a total of 10 ROV-unprotected paths (0% ROV) and 3 fully ROV-protected paths (100%), again summed up for path lengths of 1, 2 and 3 AS hops. This shows that there is quite a significant number of paths that are prone to BGP prefix hijacking, which is a risk for the CI’s email traffic as it might take these less ROV-secure paths toward their mail server at Microsoft.

Figure 1. Number of paths for different numbers of AS hops between the four CIs and the Microsoft mail service.

Figure 1 also illustrates that the number of additional ROV-protected paths increases significantly if only a few upstreams implement ROV. For example, if AS40985’s upstream were to implement ROV, then that would result in all its 14 valid paths towards Microsoft (1, 2, or 3 AS hops) becoming fully ROV-protected instead of 0, as shown in Figure 1c. Similarly, if AS15916’s upstream provider implements 100% ROV rather than around 62%, it will give ABN-AMRO Bank 9 fully ROV-protected paths out of 20 valid paths. If these upstreams do not implement ROV, then there are no 100% ROV-protected paths from AS40985 and AS15916 toward Microsoft’s email infrastructure.

ASes’ coalition to forward CI’s traffic

Table 1 shows the number of unique ASes on valid paths to and from the 4 CIs that have a 100% ROV score. As an example, there are a total of 85 different valid paths between AS15625 and Microsoft’s AS, with 15 unique ASes on these paths having an ROV score of 100%. If these 15 ASes were to form a coalition (e.g., a Trust Zone), they would have more fully ROV-protected paths to forward traffic among each other. However, the actual number of such 100% ROV-protected paths will depend on traffic forwarding agreements that the members of the coalition set up.

In the future, we envision that ASes could also offer such concepts as a value-added service to the customers who need more insight in the (ROV-based) security of ASes that handle their traffic (e.g., through visualizations) and control over such paths.

Table 1. CIs and the unique ASes that are on their valid paths to Microsoft’s mail service and that have a 100% ROV score.

Summary and future work

Although we used our method to calculate the ROV scores of Internet paths from CIs to Microsoft’s mail service in the Netherlands, it can be used to assess the ROV security of paths from any source AS to any destination AS with or without a particular destination IP prefix. Our case study shows that there are multiple paths that are 100% ROV-protected and multiple others with less or even without ROV protection, which might introduce a supply chain security risk for CI operators. However, our analysis also reveals that implementing ROV fully by upstream providers of CIs will increase the number of fully ROV-protected paths toward the Microsoft mail service by 72.5% on average.

Our future work includes calculating a path’s security status based on security metrics other than ROV (e.g., the availability of DDoS protection), which AS operators could consider when making inter-domain routing decisions. We publicly provide our analysis code on GitHub.

Acknowledgements

This work was conducted as part of the CATRIN project, which received funding from the Dutch Research Council (NWO).

About author Shyam Krishna Khadka

Shyam Krishna Khadka is a PhD student at the University of Twente, with interests in internet routing security and internet measurements. He has over a decade of working experience in software development and network technologies across various companies.