Cisco

De Kleetlaan 64 Diegem 1831 Belgium evyncke@cisco.com

Université de Liège

Liege Belgium raphael.leas@student.uliege.be

Université de Liège

Institut Montefiore B28 Allee de la Decouverte 10 Liege 4000 Belgium justin.iurman@uliege.be

Operations and Management IPv6 Operations Internet-Draft In 2016, RFC7872 has measured the drop of packets with IPv6 extension headers. This document presents a slightly different methodology with more recent results. It is still work in progress. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the IPv6 Operations Working Group mailing list (), which is archived at . Source for this draft and an issue tracker can be found at .

Introduction In 2016, has measured the drop of packets with IPv6 extension headers on their transit over the global Internet. This document presents a slightly different methodology with more recent results. Since then, has provided some recommendations for filtering transit traffic, so there may be some changes in providers policies. Also, raises awareness about the operational and security implications of IPv6 extension headers and present reasons why some networks would drop them intentionally. It is still work in progress, but the authors wanted to present some results at IETF-113 (March 2022). The code is open source and is available at .

Methodology In a first phase, the measurement is done between collaborating IPv6 nodes, a.k.a. vantage points, spread over the Internet and multiple Autonomous Systems (ASs). As seen in , the source/destination/transit ASs include some "tier-1" providers per , so, they are probably representative of the global Internet core. Relying on collaborating nodes has some benefits:

• propagation can be measured even in the absence of any ICMP message or reply generated by the destination;
• traffic timing can be measured accurately to answer whether extension headers are slower than plain IPv6 packets;
• traffic can be captured into .pcap file at the source and at the destination for later analysis.

Future phases will send probes to non-collaborating nodes with a much reduced probing speed. The destination will include top-n websites, popular CDN, as well as random prefix from the IPv6 global routing table. A revision of this IETF draft will describe those experiments.

Measurements

Vantage Points Several servers were used worldwide. lists all the vantage points together with their AS number and country. All vantage AS

ASN	AS Name	Country code	Location
267	NETHER-NET	US	Southfield, MI
3764	AFRINIC-Ops	ZA	Johanesburg
4134	Chine Telecom	CN	Beijing
7195	Edge Uno	AR	Buenos Aires
12414	NL-SOLCON SOLCON	NL	Amsterdam
14061	Digital Ocean	CA	Toronto, ON
14061	Digital Ocean	USA	New York City, NY
14601	Digital Ocean	DE	Francfort
14601	Digital Ocean	IN	Bangalore
14601	Digital Ocean	SG	Singapore
14601	Digital Ocean	UK	London
16276	OVH	AU	Sydney
16276	OVH	PL	Warsaw
20473	The Constant Company (Vultr)	MX	Mexico
20473	The Constant Company (Vultr)	SP	Madrid
20473	The Constant Company (Vultr)	JP	Tokyo
37684	Angani	KE	Nairobi
37708	AfriNIC Corporate Network	MU	Ebene
44684	Mythic Beasts	UK	Cambridge
60011	MYTHIC-BEASTS-USA	US	Fremont, CA
198644	GO6	SI	Ljubljana

Tested Autonomous Systems During first phase (traffic among fully-meshed collaborative nodes), show the ASs for which our probes have collected data. All AS (source/destination/transit)

AS Number	AS Description	Comment
174	COGENT-174, US	Tier-1
267	NETHER-NET, US
1299	TWELVE99 Arelion, fka Telia Carrier, SE	Tier-1
2497	IIJ Internet Initiative Japan Inc., JP
2828	XO-AS15, US	Regional Tier
2914	NTT-LTD-2914, US	Tier-1
3257	GTT-BACKBONE GTT, US	Tier-1
3320	DTAG Internet service provider operations, DE	Tier-1
3356	LEVEL3, US	Tier-1
3491	BTN-ASN, US	Tier-1
4134	CHINANET-BACKBONE No.31,Jin-rong Street, CN	Regional Tier
4637	ASN-TELSTRA-GLOBAL Telstra Global, HK	Regional Tier
4755	TATACOMM-AS TATA Communications formerly VSNL is Leading ISP, IN
4788	TMNET-AS-AP TM Net, Internet Service Provider, MY
5511	OPENTRANSIT, FR	Tier-1
5603	SIOL-NET Telekom Slovenije d.d., SI
6453	AS6453, US	Tier-1
6461	ZAYO-6461	Tier-1
6762	SEABONE-NET TELECOM ITALIA SPARKLE S.p.A., IT	Tier-1
6895	ESPANIX Neutral Interconect Exchange for Spain, ES
6939	HURRICANE, US	Regional Tier
7195	EDGEUNO SAS, CO
8447	A1TELEKOM-AT A1 Telekom Austria AG, AT
9498	BBIL-AP BHARTI Airtel Ltd., IN
12129	123NET, US
12414	NL-SOLCON SOLCON, NL
14061	DIGITALOCEAN-ASN, US
14103	ACDNET-ASN1, US
16276	OVH, FR
20473	AS-CHOOPA, US
21283	A1SI-AS A1 Slovenija, SI
23889	MauritiusTelecom, MU
33764	AFRINIC-ZA-JNB-AS, MU
34779	T-2-AS AS set propagated by T-2 d.o.o., SI
37100	SEACOM-AS, MU
37271	Workonline
37684	ANGANI-AS, KE
37708	AFRINIC-MAIN, MU
44684	MYTHIC Mythic Beasts Ltd, GB
58461	CT-HANGZHOU-IDC No.288,Fu-chun Road, CN
60011	MYTHIC-BEASTS-USA, GB
198644	GO6, SI
211722	Nullroute
328578	KEMNET-TECHNOLOGIES-AS, KE

The table attributes some tier qualification to some ASs based on the Wikipedia page , but there is no common way to decide who is a tier-1. Based on some CAIDA research, all the above (except GO6, which is a stub network) are transit providers. While this document lists some operators, the intent is not to build a wall of fame or a wall of shame but more to get an idea about which kind of providers drop packets with extension headers and how widespread the drop policy is enforced and where, i.e., in the access provider or in the core of the Internet.

Drop attribution to AS Comparing the traceroutes with and without extension headers allows the attribution of a packet drop to one AS. But, this is not an easy task as inter-AS links often use IPv6 address of only one AS (if not using link-local per ). This document uses the following algorithm to attribute the drop to one AS for packet sourced in one AS and then having a path traversing AS#foo just before AS#bar:

• if the packet drop happens at the first router (i.e., hop limit == 1 does not trigger an ICMP hop-limit exceeded), then the drop is assumed to this AS as it is probably an ingress filter on the first router (i.e., the hosting provider in most of the cases - except if collocated with an IXP).
• if the packet drop happens in AS#foo after one or more hop(s) in AS#bar, then the drop is assumed to be in AS#foo ingress filter on a router with an interface address in AS#foo
• if the packet drop happens in AS#bar after one or more hop(s) in AS#bar before going to AS#foo, then the drop is assumed to be in AS#foo ingress filter on a router with an interface address in AS#bar

In several cases, the above algorithm was not possible (e.g., some intermediate routers do not generate an ICMP unreachable hop limit exceeded even in the absence of any extension headers), then the drop is not attributed. Please also note that the goal of this document is not to 'point fingers to operators' but more to evaluate the potential impact. I.e., a tier-1 provider dropping packets with extension headers has a much bigger impact on the Internet traffic than an access provider. Future revision of this document will use the work of . Readers are urged not to rely on the AS attribution in this document version.

Tested Extension Headers In the first phase among collaborating vantage points, packets always contained either a UDP payload or a TCP payload, the latter is sent with only the SYN flag set and with data as permitted by section 3.4 of (2nd paragraph). A usual traceroute is done with only the UDP/TCP payload without any extension header with varying hop-limit in order to learn the traversed routers and ASs. Then, several UDP/TCP probes are sent with a set of extension headers:

• hop-by-hop options header containing:
  • one PadN option for a length of 8 octets
  • one unknown option with the "discard" bits for a length of 8 octets
  • one unknown option (two sets: with "discard" and "skip" bits) for a length of 256 and 512 octets
• destination options header containing:
  • one PadN option for a length of 8 octets
  • one unknown option with the "discard" bits for a length of 8 octets
  • one unknown option (two sets: with "discard" and "skip" bits) for a length of 16, 32, 64, 128, 256, and 512 octets
  • one unknown option (with "skip" bits) for a length of 24, 40, 48, and 56 octets
• routing header with routing types from 0 to 6 inclusive
• fragment header of varying frame length 512, 1280, and 1500 octets:
  • atomic fragment (i.e., M-flag = 0 and offset = 0)
  • non-atomic first fragment (i.e., M-flag = 1 and offset = 0)
• encapsulation security payload (ESP) header with dummy SPI followed by UDP/TCP header and a 38 octets payload
• authentication header (AH) with dummy SPI followed by UDP/TCP header and a 38 octets payload

In the above, length is the length of the extension header itself except for the fragmentation header where the length is the IP packet length (i.e., including the IPv6, and TCP/UDP headers + payload). Also, an unknown option means an option with an unassigned code in the IANA registry . For hop-by-hop and destination options headers, the choice was made to use one unknown option instead of multiple consecutive PadN options in order to avoid packets from being discarded on the destination. Indeed, the Linux kernel does not accept consecutive Pad1 or PadN options if their total size exceeds 7 octets. Not only multiple PadN options violate section 2.1.9.5 of , but it is also considered as suspicious (see section 5.3 of ). Nevertheless, for comparative purposes, multiple PadN options were used for experiments of length 256 octets. In that very specific case, drops on the destination are not considered as drops. In addition to the above extension headers, other probes were sent with next header field of IPv6 header set to:

• 59, which is "no next header", especially whether extra octets after the no next header as section 4.7 requires that "those octets must be ignored and passed on unchanged if the packet is forwarded"
• 143, which is Ethernet payload (see section 10.1 of )

Results This section presents the current results out of phase 1 (collaborating vantage points) testing. Probe packets were sent between all pairs of vantage points with a hop-limit from 1 to the number of hops between the two vantage points and for all the extension headers described in .

Routing Header lists all routing header types and the percentage of experiments that were successful, i.e., packets with routing header reaching their destination, both for UDP and TCP: Per Routing Header Types Transmission

Routing Header Type	UDP	TCP
0	74.3%	71.2%
1	88.3%	81.4%
2	97.4%	90.4%
3	97.6%	91.3%
4	78.8%	72.6%
5	97.4%	90.9%
6	97.4%	90.0%

and respectively list ASs that drop packets with the routing header type 0 (the original source routing header, which is now deprecated) and packets with the routing header type 1 (NIMROD , which is now deprecated). ASs dropping Routing Header Type 0

AS Number	AS description
1299	TWELVE99 Arelion, fka Telia Carrier, SE
5511	OPENTRANSIT, FR
6895	ESPANIX Neutral Interconect Exchange for Spain, ES
6939	HURRICANE, US
7195	EDGEUNO (DE-CIX Frankfurt)
9498	BBIL-AP BHARTI Airtel Ltd. (Equinix Hong Kong)
14061	DIGITALOCEAN (DE-CIX Frankfurt)
16276	OVH
37684	ANGANI-AS, KE
58461	CT-HANGZHOU-IDC No.288,Fu-chun Road, CN
328578	KEMNET-TECHNOLOGIES-AS, KE

ASs dropping Routing Header Type 1

AS Number	AS description
1299	TWELVE99 Arelion, fka Telia Carrier, SE
4134	CHINANET-BACKBONE No.31,Jin-rong Street, CN
5511	OPENTRANSIT, FR
37684	ANGANI-AS, KE
58461	CT-HANGZHOU-IDC No.288,Fu-chun Road, CN
328578	KEMNET-TECHNOLOGIES-AS, KE

Regarding the routing type 0, it is possibly due to a strict implementation of but it is expected that no packet with such routing type would be transmitted anymore. So, this is not surprising. The same reasoning could be applied to the routing type 1. lists ASs that drop packets with the routing header type 4 (Segment Routing Header ). ASs dropping Routing Header Type 4

AS Number	AS description
1299	TWELVE99 Arelion, fka Telia Carrier, SE
4637	ASN-TELSTRA-GLOBAL Telstra Global, HK
5511	OPENTRANSIT, FR
6939	HURRICANE, US
7195	EDGEUNO SAS, CO
14061	DIGITALOCEAN-ASN, US
16276	OVH, FR
37100	SEACOM-AS, MU
58461	CT-HANGZHOU-IDC No.288,Fu-chun Road, CN

This drop of SRH was to be expected as SRv6 is specified to run only in a limited domain. Other routing header types (2 == mobile IPv6 , 3 == RPL , and even 5 == CRH-16 and 6 == CRH-32) can be transmitted over the global Internet without being dropped (assuming that the 2.5% of dropped packets are within the measurement error). At least, this is true for UDP. We still need to investigate the differences for TCP equivalent transmissions.

Hop-by-Hop Options Header lists all experiments (types and lengths) along with their success percentages, i.e., packets with a hop-by-hop header reaching their destination, both for UDP and TCP: Hop-by-hop Header Transmission

Option Type	Length (bytes)	UDP	TCP
Skip	8	8.6%	9.1%
Discard	8	0.0%	0.0%
Skip	256	2.4%	2.4%
Skip w/ PadN	256	0.5%	0.6%
Discard	256	0.0%	0.0%
Skip	512	1.4%	1.5%
Discard	512	0.0%	0.0%

It appears that hop-by-hop options headers cannot reliably traverse the global Internet; only small headers with 'skipable' options have some chances. If the unknown hop-by-hop option has the 'discard' bits, it is dropped per specification, although we observed in some cases that such packets were not necessarily dropped directly by the very first hop. Globally, there are no notable differences between UDP and TCP.

Destination Options Header lists all lengths that have been tested along with their success percentages, i.e., packets with a destination header reaching their destination, both for UDP and TCP: Destination Header Transmission

Length (bytes)	UDP	TCP
8	97.8%	94.3%
16	97.7%	90.5%
24	97.6%	89.8%
32	93.5%	86.2%
40	93.9%	86.2%
48	93.7%	86.1%
56	93.8%	52.7%
64	45.9%	37.8%
128	10.9%	10.9%
256	4.3%	4.3%
256 w/ PadN	4.3%	4.3%
512	3.1%	3.1%

The measurement revealed no difference with the discard bits, which tends to show that routers do not look inside the destination header, as expected. The size of the destination options header has a major impact on the drop probability. It appears that destination headers larger than 24 octets already cause drops. It may be because the 40 octets of the IPv6 header + the 24 octets of the extension header (total 64 octets) is still in the limits of some router hardware lookup mechanisms while the next measured size (extension header size of 32 octets for a total of 72 octets) is beyond the hardware limit and some ASs have a policy to drop packets where the TCP/UDP ports are unknown. A major drop also occurs once the size reaches 64 bytes for UDP while, surprisingly, it happens at 56 bytes for TCP. In either case, the chances of surviving are approximately halved. We still need to investigate the differences for TCP equivalent transmissions.

Fragmentation Header The propagation of two kinds of fragmentation headers was analysed: atomic fragment (offset == 0 and M-flag == 0) and plain first fragment (offset == 0 and M-flag == 1). The displays the propagation differences. IPv6 Fragments Transmission

M-flag	UDP	TCP
0 (atomic)	55.8%	49.8%
1	89.2%	87.6%

The size of the overall IPv6 packets (512, 1280, and 1500 octets) has no major impact on the propagation.

No extension headers drop at all lists ASs that do not drop transit traffic with extension headers and therefore follow the recommendations of : ASs not dropping packets with Extension Headers

AS Number	AS Description
267	NETHER-NET, US
2497	IIJ Internet Initiative Japan Inc., JP
14103	ACDNET-ASN1, US
21283	A1SI-AS A1 Slovenija, SI
33764	AFRINIC-ZA-JNB-AS, MU
37271	Workonline
37708	AFRINIC-MAIN, MU
60011	MYTHIC-BEASTS-USA, GB
198644	GO6, SI

Other Next Headers Measurements also include two protocol numbers that are mainly new use of IPv6 as well as AH and ESP. indicates the percentage of packets reaching the destination. Transmission of Special IP Protocols

Next Header	Transmission
NoNextHeader (59)	98.2%
Ethernet (143)	98.3%
Authentication (AH)	98.1%
ESP	98.3%

The above indicates that those IP protocols can be transmitted over the global Internet without being dropped (assuming that the 2% of dropped packets are within the measurement error). Globally, there are no notable differences between UDP and TCP, for cases where it applies.

Summary of the collaborating parties measurements While the analysis has areas of improvement (geographical distribution and impact on latency), it appears that:

• AH, ESP, and non-atomic fragmentation headers (to some extent) can traverse the Internet;
• only routing headers types 0, 1 and 4 experiment problems over the Internet, other types have no problems;
• hop-by-hop options headers do not traverse the Internet, whatever the size;
• destination options headers are not reliable enough when it exceeds 24 octets.

Of course, the next phase of measurement with non-collaborating parties will probably give another view.

Security Considerations While active probing of the Internet may be considered as an attack, this measurement was done among collaborating parties and using the probe attribution technique described in to allow external parties to identify the source of the probes if required.

IANA Considerations This document has no IANA actions.

References Normative References n.d. This document specifies version 6 of the Internet Protocol (IPv6). It obsoletes RFC 2460. Informative References n.d. n.d. Université de Liège n.d. University College London University College London CAIDA, San Diego Supercomputer Center, University Of California San Diego CAIDA, San Diego Supercomputer Center, University Of California San Diego This document presents real-world data regarding the extent to which packets with IPv6 Extension Headers (EHs) are dropped in the Internet (as originally measured in August 2014 and later in June 2015, with similar results) and where in the network such dropping occurs. The aforementioned results serve as a problem statement that is expected to trigger operational advice on the filtering of IPv6 packets carrying IPv6 EHs so that the situation improves over time. This document also explains how the results were obtained, such that the corresponding measurements can be reproduced by other members of the community and repeated over time to observe changes in the handling of packets with IPv6 EHs. EdgeUno Huawei Technologies This document analyzes the security implications of IPv6 Extension Headers and associated IPv6 options. Additionally, it discusses the operational and interoperability implications of discarding packets based on the IPv6 Extension Headers and IPv6 options they contain. Finally, it provides advice on the filtering of such IPv6 packets at transit routers for traffic not directed to them, for those cases where such filtering is deemed as necessary. This document summarizes the operational implications of IPv6 extension headers specified in the IPv6 protocol specification (RFC 8200) and attempts to analyze reasons why packets with IPv6 extension headers are often dropped in the public Internet. Sandelman Software Works Inc This document describes the format used by the libpcap library to record captured packets to a file. Programs using the libpcap library to read and write those files, and thus reading and writing files in that format, include tcpdump. In an IPv6 network, it is possible to use only link-local addresses on infrastructure links between routers. This document discusses the advantages and disadvantages of this approach to facilitate the decision process for a given network. The transition from a pure IPv4 network to a network where IPv4 and IPv6 coexist brings a number of extra security considerations that need to be taken into account when deploying IPv6 and operating the dual-protocol network and the associated transition mechanisms. This document attempts to give an overview of the various issues grouped into three categories: o issues due to the IPv6 protocol itself, o issues due to transition mechanisms, and o issues due to IPv6 deployment. This memo provides information for the Internet community. This document defines requirements for IPv6 nodes. It is expected that IPv6 will be deployed in a wide range of devices and situations. Specifying the requirements for IPv6 nodes allows IPv6 to function well and interoperate in a large number of situations and deployments. This document obsoletes RFC 6434, and in turn RFC 4294. The Segment Routing over IPv6 (SRv6) Network Programming framework enables a network operator or an application to specify a packet processing program by encoding a sequence of instructions in the IPv6 packet header. Each instruction is implemented on one or several nodes in the network and identified by an SRv6 Segment Identifier in the packet. This document defines the SRv6 Network Programming concept and specifies the base set of SRv6 behaviors that enables the creation of interoperable overlays with underlay optimization. This document presents the requirements that the Nimrod routing and addressing architecture has upon the internetwork layer protocol. To be most useful to Nimrod, any protocol selected as the IPng should satisfy these requirements. This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. The functionality provided by IPv6's Type 0 Routing Header can be exploited in order to achieve traffic amplification over a remote path for the purposes of generating denial-of-service traffic. This document updates the IPv6 specification to deprecate the use of IPv6 Type 0 Routing Headers, in light of this security concern. [STANDARDS-TRACK] Segment Routing can be applied to the IPv6 data plane using a new type of Routing Extension Header called the Segment Routing Header (SRH). This document describes the SRH and how it is used by nodes that are Segment Routing (SR) capable. This document specifies Mobile IPv6, a protocol that allows nodes to remain reachable while moving around in the IPv6 Internet. Each mobile node is always identified by its home address, regardless of its current point of attachment to the Internet. While situated away from its home, a mobile node is also associated with a care-of address, which provides information about the mobile node's current location. IPv6 packets addressed to a mobile node's home address are transparently routed to its care-of address. The protocol enables IPv6 nodes to cache the binding of a mobile node's home address with its care-of address, and to then send any packets destined for the mobile node directly to it at this care-of address. To support this operation, Mobile IPv6 defines a new IPv6 protocol and a new destination option. All IPv6 nodes, whether mobile or stationary, can communicate with mobile nodes. This document obsoletes RFC 3775. [STANDARDS-TRACK] In Low-Power and Lossy Networks (LLNs), memory constraints on routers may limit them to maintaining, at most, a few routes. In some configurations, it is necessary to use these memory-constrained routers to deliver datagrams to nodes within the LLN. The Routing Protocol for Low-Power and Lossy Networks (RPL) can be used in some deployments to store most, if not all, routes on one (e.g., the Directed Acyclic Graph (DAG) root) or a few routers and forward the IPv6 datagram using a source routing technique to avoid large routing tables on memory-constrained routers. This document specifies a new IPv6 Routing header type for delivering datagrams within a RPL routing domain. [STANDARDS-TRACK] Juniper Networks NTT Communications Corporation Liquid Telecom Liquid Telecom Verizon This document defines two new Routing header types. Collectively, they are called the Compact Routing Headers (CRH). Individually, they are called CRH-16 and CRH-32. Cisco Université de Liège Université de Liège Active measurements at Internet-scale can target either collaborating parties or non-collaborating ones. This is similar scan and could be perceived as aggressive. This document proposes a couple of simple techniques allowing any party or organization to understand what this unsolicited packet is, what is its purpose, and more importantly who to contact.

Acknowledgments The authors want to thank AfriNIC, Angani, China Telecom, Jared Mauch, Sander Steffann, XiPeng Xiao, and Jan Zorz for allowing the free use of their labs. Other thanks to Ben Campbell and Fernando Gont who indicated a nice IPv6 hosting provider in Africa and South America. Special thanks as well to Professor Benoit Donnet for his support and advices. This document would not have existed without his support.