Boeing Research & Technology

P.O. Box 3707 Seattle WA 98124 USA fltemplin@acm.org

SiPanda

Santa Clara CA USA tom@sipanda.io

I-D Internet-Draft The Internet Protocol, version 4 (IPv4) header includes a 16-bit Identification field in all packets, but this length is too small to ensure reassembly integrity even at moderate data rates in modern networks. Even for Internet Protocol, version 6 (IPv6), the 32-bit Identification field included when a Fragment Header is present may be smaller than desired for some applications. This specification addresses these limitations by defining an IPv6 Extended Fragment Header (EFH) that includes a 64-bit Identification with efficient fragmentation and reassembly procedures.

The Internet Protocol, version 4 (IPv4) header includes a 16-bit Identification in all packets , but this length is too small to ensure reassembly integrity even at moderate data rates in modern networks . For Internet Protocol, version 6 (IPv6), the Identification field is only present in packets that include a Fragment Header , but even its standard length of 32 bits may be too small for some applications such as those that regard the Identification value as a sequence number. This specification therefore defines a means to extend the IPv6 Identification length to 64 bits through the introduction of a new IPv6 Extended Fragment Header (EFH). The IPv6 EFH employs a fragmentation/reassembly algorithm based on an ordinal fragment index combined with the non-final fragment payload size instead of a 13-bit integer encoding an 8-octet offset. In this arrangement, both fragmentation and reassembly are greatly simplified allowing for efficient implementations. The IPv6 EFH may be useful for networks that engage fragmentation and reassembly at extreme data rates, or for cases when advanced packet Identification uniqueness assurance is critical. (The placement of the IPv6 EFH in a Destination Options header may also make the option less prone to network filtering.) This specification further defines a messaging service for adaptive realtime response to loss and congestion related to fragmentation/reassembly. Together, these extensions support robust fragmentation and reassembly services as well as packet Identification uniqueness for IPv6. The IPv6 EFH 64-bit Identification is in some aspects analogous to the IPsec AH and ESP Extended Sequence Number (ESN). In environments where transmitting the full 64-bit Identification may be wasteful, nodes can apply header compression techniques to transmit only the 32 least significant bits while retaining the full 64-bit value in cache memory.

The terms "Maximum Transmission Unit (MTU)", "Effective MTU to Receive (EMTU_R)", "Effective MTU to Send (EMTU_S)" and "Maximum Segment Lifetime (MSL)" are used exactly the same as for standard Internetworking terminology . The term "Maximum Reassembly Unit (MRU)" is equivalent to EMTU_R, and the term "maximum datagram lifetime (MDL)" (defined in ) is equivalent to MSL. The term "Packet Too Big (PTB)" refers to an ICMPv6 "Packet Too Big" message. The term "flow" refers to a sequence of packets sent from a particular source to a particular unicast, anycast or multicast destination that a node desires to label as a flow per . The term "Extended Fragment Header (EFH)" refers to a new IPv6 Destination Option as specified in this document. The EFH is included as the first option in the final Per-Fragment header, while the remainder of the packet that follows is known as the "fragment payload". The Automatic Extended Route Optimization (AERO) and Overlay Multilink Network Interface (OMNI) services rely on the IPv6 EFH for secure adaptation layer encapsulation and fragmentation. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Upper layer protocols often achieve greater performance by configuring segment sizes that exceed the path Maximum Transmission Unit (MTU). When the segment size exceeds the path MTU, lower layer IP fragmentation is a natural consequence. However, the 4-octet (32-bit) Identification field of the Fragment Header may be too small to ensure reassembly integrity at sufficiently high data rates, especially when the source resets the starting sequence number often to maintain an unpredictable profile . This specification therefore proposes to fortify the IPv6 Identification by extending its length. Performance increases for upper layer protocols that use larger segment sizes was historically observed for NFS over UDP, and can still be readily observed today for TCP and the Delay Tolerant Network (DTN) Licklider Transmission Protocol (LTP) as reported in . The test setup included a pair of modern high-performance servers with 100Gbps Ethernet cards connected via a point-to-point link and running a modern public domain linux release. TCP performance using the public domain 'iperf3' tool was proven to increase when larger segment sizes are used even if they exceed the path MTU and invoke a service known as Generic Segment/Receive Offload (GSO/GRO). LTP performance with segment sizes that exceed the path MTU was similarly proven using the HDTN and ION-DTN LTP implementations which engage IP fragmentation and reassembly. A significant effort was made to incorporate Generic Segment and Receive Offload (GSO/GRO) in ION-DTN, but those services contributed minimal performance improvements whereas the performance improvements with IP fragmentation and reassembly were dramatic. In addition to accommodating higher data rates in the presence of fragmentation and reassembly, extending the IPv6 Identification can enable other important services. For example, an extended Identification can enable a duplicate packet detection service where the network remembers recent Identification values for a flow to aid detection of potential duplicates. When an encapsulation source includes an IPv6 EFH, the extended Identification can also serve as a sequence number that allows each encapsulation destination to exclude any packets with values outside of the current sequence number window for a flow as potential spurious transmissions. A robust IP fragmentation and reassembly service can provide a useful tool for performance maximization in the Internet when an extended Identification is available. This document therefore presents a means to extend the IPv6 Identification to better support these services through the introduction of an IPv6 Extended Fragment Header (EFH).

For a standard 4-octet IPv6 Identification, the source can simply include a standard IPv6 Fragment Header as specified in with the Fragment Offset field and M flag set either to values appropriate for a fragmented packet or the value 0 for an unfragmented packet. The source then includes a 4-octet Identification value for the packet. For an extended Identification, advanced fragmentation and reassembly procedures and/or for paths that drop packets including the standard IPv6 Fragment Header, this specification permits the source to instead include an IPv6 EFH. The source includes the IPv6 EFH as the first option in a Destination Options Header positioned as the final IPv6 Per-Fragment Header. The remainder of the packet beginning with the Extension and Upper Layer Headers for the first fragment or protocol data for non-first fragments is known as the fragment payload. The Destination Options header that includes the EFH option therefore appears in each fragment in the same position where the standard Fragment Header would otherwise appear while the Fragment Header itself is omitted - see Sections 4.1 and 4.5 of . The IPv6 EFH Destination Option is formatted as shown in :

The IPv6 EFH Destination Option is therefore identified as an Option Type with the low-order 5 bits set to TBD (see: IANA Considerations) and with the third-highest-order bit (i.e., "chg") set to 1. The highest-order 2 bits (i.e., "act") are set to '01' or '1X' so that destinations that do not recognize the option will drop the packet or fragment and (possibly) also return an ICMPv6 Parameter Problem message. The Identification field is 8 octets (64 bits) in length and a Destination Options header that includes the option may appear either in an unfragmented IPv6 packet or in one for which IPv6 fragmentation is applied.

All aspects of IPv6 fragmentation using the EFH are conducted exactly as for standard IPv6 fragmentation per Section 4.5 of except as specified below. When the source performs fragmentation using the IPv6 EFH, it SHOULD produce the smallest number of fragments possible within current path MTU constraints and MUST produce no more than 64 fragments per packet. The payload of each non-final fragment following the Destination Options header MUST NOT be smaller than 1024 octets, allowing for up to 256 octets of Per-Fragment headers plus any lower-layer IP encapsulations within the 1280 octet IPv6 minimum path MTU. The payload of each non-final fragment MUST be equal in length, while the final fragment payload MAY be smaller and MUST NOT be larger. For each of the F fragments produced during fragmentation, the source writes an ordinal index number beginning with 0 in the "Index" field for the first fragment and increasing by 1 for each successive non-first fragment while setting the "M" flag accordingly. Specifically, the source sets (Index, M) to (0, 1) for the first fragment, (1, 1) for the second, (2, 1) for the third, etc., up to and including ((F-1), 0) for the final fragment. The source also sets (Sub-Index, N) to (0, 0) in all fragments. For each fragment produced during fragmentation, the source includes a Destination Options header as the final Per-Fragment header with the IPv6 EFH as the first option followed by any other options. The source then caches the Destination Options header Next Header value in the NH-Actual field and (for each non-first fragment) resets the Next Header field to "No Next Header". Intermediate systems that forward non-first fragments prepared in this way will therefore ignore the fragment payload that follows (by virtue of the "No Next Header" setting) unless they are configured to more deeply inspect the data content. After performing source fragmentation but before releasing the fragments, the source may become aware that the path and/or path MTU has changed. If so, the source can perform further fragmentation for each original fragment by producing "S" minimum-length sub-fragments under the same procedures above, with non-final sub-fragment payloads equal in length and no smaller than 1024 octets. The source MUST write the same (NH-Actual, index, M) values into each sub-fragment while setting (sub-index, N) to (0, 1), (1, 1), (2, 1), etc. in each successive non-final sub-fragment and setting ((S-1), 0) in the final sub-fragment. The source MUST NOT perform further fragmentation on sub-fragments that already include (sub-index, N) values other than (0, 0). Note: when the source performs fragmentation in conjunction with IPv6 encapsulation over small MTU paths, the size of the encapsulation headers may require the source to use a slightly larger non-final fragment payload size (e.g., 1025 octets, 1026 octets, etc.) to accommodate (near-)maximum sized original IP packets within the 64 fragment limit.

When an intermediate system detects a path change and/or a path MTU reduction while forwarding an original fragment it SHOULD perform further fragmentation if it can easily locate the IPv6 EFH supplied by the source; otherwise, it drops the fragment and returns a PTB message. Intermediate system procedures for performing further fragmentation are identical to those specified for the source (see: ). The intermediate system acts only as an agent for the source and MUST NOT insert an EFH itself.

All aspects of IPv6 reassembly using the EFH are conducted exactly as for standard IPv6 reassembly per Section 4.5 of except as specified below. When the destination receives original EFH fragments with the same (Source, Destination, Identification)-tuple, it reassembles by concatenating the payloads of consecutive fragments in ascending ordinal Index numbers, i.e., ordinal 0, followed by 1, followed by 2, etc. until all fragments are concatenated. In the process, the destination discards any non-final fragments with payload lengths less than 1024 octets or with payload lengths that differ from the others. When the destination receives EHF sub-fragments that set (Sub-Index, N) to a value other than (0, 0), it first reassembles the original fragment by concatenating the payloads of each sub-fragment the same as described above. When the original fragment has been reconstructed, the destination submits it for fragment-level reassembly as described above. The destination should also return an indication (subject to rate limiting) to recommend that the source re-probe the path MTU.

Destinations that do not recognize the IPv6 EFH option drop the packet. If the Option Type "act" code is '1X', destinations also return a Code 2 ICMPv6 Parameter Problem message . (ICMPv6 messages may be lost on the return path and/or manufactured by an adversary, however, and therefore provide only an advisory indication.) The source can then test whether destinations recognize the IPv6 EFH option by occasionally sending "probe" packets that include the option. The source has assurance that a destination recognizes the option if it receives acknowledgments; otherwise, it may receive Code 2 ICMPv6 Parameter Problem messages as hints that a destination does not recognize the option. The source should re-probe destinations occasionally in case routing redirects a flow to a different anycast destination or in case a multicast group membership changes (see: ). The source can also include the IPv6 Minimum Path MTU Discovery Hop-by-Hop option in packets/fragments sent to unicast, multicast or anycast destinations per . If the source receives acknowledgements that include an MTU/Fragmentation Report Option (see: ), it should regard the reported MTU as the largest potential packet/fragment size for this destination under current path MTU constraints noting that the actual size may be smaller still for some paths.

When a router attempts to forward an IPv6 packet (or fragment) that exceeds the next hop link MTU but for which fragmentation is forbidden, it returns a standard PTB message to the source and discards the packet. This always results in wasted transmissions by the source and all routers on the path toward the one with the restricting link. Moreover, the messages are subject to spoofing and loss in the network . The source should therefore probe the path MTU per to determine the maximum fragment size that can traverse the path. The source can then perform source fragmentation using the IPv6 EFH option with a maximum fragment size limited to the path MTU. If an intermediate system detects a path change and can easily locate the IPv6 EFH included by the source, it can forward smaller sub-fragments that will not incur further MTU restrictions.

End systems that recognize the IPv6 EFH also recognize an MTU/Fragmentation Report Option that uses option type TBD the same as for the EFH itself except with the act code set to '00' and formatted as shown below:

The destination end system includes the MTU/Fragmentation Report Option in a return packet to the source when reassembly congestion and/or fragment loss occurs. (Destinations that receive IPv6 packets with both the IPv6 EFH Destination Option and the IPv6 Minimum Path MTU Discovery Hop-by-Hop Option can also return MTU/Fragmentation Reports in this way.) Any packet that is part of an ongoing flow can be used to carry the option, especially if it includes identifying parameters and/or authentication signatures. The destination sets MTU to the minimum of the incoming link MTU and the MTU value recoded in the IPv6 Minimum Path MTU HBH option (if present) to determine the maximum fragment size for the path. The destination then sets MRU to the most significant 15 bits of the recommended maximum reassembly size under current congestion conditions, sets the P flag to 1 if the source should re-probe the path and sets Identification to the value for the current reassembly in all cases. If all fragments of the packet (or the unfragmented packet itself) have arrived the destination sets Opt Data Len to 12 and omits the Bitmap field. If some fragments are missing, the destination instead sets Length to 20 and includes a 64-bit Bitmap field with Bitmap(i) (for i=0 to 63) set to 1 for each ordinal fragment index it has received for this reassembly and set to 0 for all others. When the source receives authentic IP packets with the MTU/Fragmentation Report Option, it should decrease the size of its future packet transmissions to this destination if the value in the MRU field includes a smaller recommended maximum reassembly size under current congestion conditions. If the P flag is set, the source should also arrange to include an IPv6 Minimum Path MTU Hop-by-Hop option in future packets to re-probe the path. If the option further includes a Bitmap with some bits set to 0, the source can retransmit any missing ordinal fragments if it still has them in its cache. When the source ceases to receive MTU/Fragmentation Reports, it can begin to increase the size of its future packet transmissions to this destination. Note: the MTU/Fragmentation Report option may appear in the same Destination Options header that includes an IPv6 EFH option. The IPv6 EFH must appear as the first option in the header and the two options are differentiated by their 8-bit Option Type values. Note: the above source packet size adaptation based on destination reassembly feedback parallels the Additive Increase, Multiplicative Decrease (AIMD) congestion control strategy employed by TCP and other reliable transports.

In addition to unicast flows, similar considerations apply for flows in which the destination is a multicast group or an anycast address. Unless the source and all candidate destinations are members of a limited domain network for which all nodes recognize the IPv6 EFH, some destinations may recognize the option while others drop packets containing the option and may return a Code 2 ICMPv6 Parameter Problem message . When a source sends packets/fragments with IPv6 EFH options to a multicast group, the packets/fragments may be replicated in the network such that a single transmission may reach N destinations over as many as N different paths. Some destinations may then return IPv6 packets with MTU/Fragmentation Reports if they experience congestion and/or loss; they may also return Code 2 ICMPv6 Parameter Problem messages if they do not recognize the IPv6 EFH option. While the source receives authentic PTB messages or authentic IP packets with MTU/Fragmentation Reports, it should reduce the sizes of the packets/fragments it sends to the multicast group even if only one or a few paths or destinations are currently experiencing congestion. This means that transmissions to a multicast group will converge to the performance characteristics of the lowest common denominator group member destinations and/or paths. While the source receives ICMPv6 Parameter Problem messages and/or otherwise detects that some multicast group members do not recognize the IPv6 Extended Fragment Header option, it must determine whether the benefits for group members that recognize the option outweigh the drawbacks of service denial for those that do not. When a source sends packets/fragments with IPv6 Extended Fragment Headers option to an anycast address, routing may direct initial fragments of the same packet to a first destination that configures the address while directing the remaining fragments to other destinations that configure the address. These wayward fragments will simply result in incomplete reassemblies at each such anycast destination which will soon purge the fragments from the reassembly buffer. The source will eventually retransmit, and all resulting fragments should eventually reach a single reassembly target.

All normative aspects of standard IPv6 fragmentation and reassembly as specified in apply also to the IPv6 EFH except where this document specifies differences. All nodes that process an IPv6 Destination Options Header with IPv6 EFH option observe the extension header limits specified in . Destinations that accept flows using IPv6 EFH options MUST configure an EMTU_R of 65535 octets or larger. Sources MUST include at most one IPv6 Standard or Extended Fragment Header in each IPv6 packet/fragment, and destinations MUST silently drop packets/fragments with multiples. If the source includes an IPv6 EFH, it MUST appear as the first option in a Destination Options header that MUST appear as the final Per-Fragment header before the fragment payload. Sources that include an IPv6 EFH option MUST perform fragmentation such that at most 64 fragments are produced and all non-final fragments include equal-length fragment payloads no smaller than 1024 octets while the final fragment MAY be smaller and MUST NOT be larger. Sources that include the IPv6 EFH option in packet transmissions MUST also recognize the MTU/Fragmentation Report option in received packets as specified in . Sources SHOULD maintain a cache of recently-sent fragments in case the destination requests retransmission. The destination is required to send an "ICMP Time Exceeded - Fragment Reassembly Time Exceeded" message if insufficient fragments are received to complete reassembly within 60 seconds (see: Section 4.5 of ), but that time may be longer than practical for the source to retain fragments in a retransmission cache. The source SHOULD therefore maintain the cache for only a small delay beyond the round-trip time to the destination, and the destination SHOULD send Fragmentation Reports as early as practically possible upon experiencing fragment loss.

During the earliest days of internetworking, researchers attributed the warning label "harmful" to IP fragmentation based on empirical observations in the ARPANET, DARPA Internet and other internetworks of the day . This inspired an engineering discipline known as "Path MTU Discovery" within an emerging community of interest known as the Internet Engineering Task Force (IETF). In more recent times, the IETF published "IP Fragmentation Considered Fragile" to characterize the current state of fragmentation in the modern Internet. The IPv6 Extended Fragment Header now introduces a more robust solution based on an improved IPv6 fragmentation and reassembly service that addresses these historical concerns.

In progress.

The IANA is requested to assign a new IPv6 Destination Option type in the "Destination Options and Hop-by-Hop Options" table of the 'ipv6-parameters' registry (registration procedures IESG Approval, IETF Review or Standards Action). The option type should appear in 4 consecutive table entries. The first entry sets "act" to '00, "chg" to '0', "rest" to TBD, "Description" to "IPv6 MTU/Fragmentation Report" and "Reference" to this document [RFCXXXX]. The next three entries set "act" to 'XX', "chg" to '1', "rest" to TBD, "Description" to "IPv6 Extended Fragment Header" and "Reference" to this document [RFCXXXX] (i.e., with "act" set to '01' for the first entry, '10' for the second and '11' for the final entry). Each table entry also sets "Hex Value" to the 2-digit hexadecimal value corresponding to the 8-bit concatenation of act/chg/rest.

All aspects of IP security apply equally to this document, which does not introduce any new vulnerabilities. Moreover, when employed correctly the mechanisms in this document robustly address known IP reassembly integrity concerns and also provide an advanced degree of packet Identification uniqueness assurance. All security aspects of , including the algorithms for selecting fragment identification values, apply also to the IPv6 EFH. In particular, the source should reset its starting Identification value frequently (e.g., per the algorithms found in ) to maintain an unpredictable profile. All normative security guidance on IPv6 fragmentation found in (e.g., processing of tiny first fragments, overlapping fragments, etc.) applies also to the fragments generated under the IPv6 EFH. IPsec AH and ESP define an Extended Sequence Number (ESN) that is analogous to the 64-bit Identification specified for the IPv6 EFH option. Nodes that employ the IPv6 EFH can use the Identification value as a sequence number to improve security in the same fashion as for IPsec AH/ESP ESNs.

This work was inspired by continued DTN performance studies. Amanda Baber, Bob Hinden, Christian Huitema, Mark Smith and Eric Vyncke offered useful insights that helped improve the document. Honoring life, liberty and the pursuit of happiness.

<< RFC Editor - remove prior to publication >> Differences from earlier versions: First draft publication.