Reliable and Available Wireless Framework

Introduction Wireless networks operate on a shared medium where uncontrolled interference, including the self-induced multipath fading, cause random transmission losses and add new dimensions to the statistical effects that affect reachability and packet delivery. To defeat those additional causes of transmission delay and loss, Reliable and Available Wireless (RAW) leverages deterministic layer-2 capabilities with diversity in the spatial, time, code, technology, and frequency domains. The challenge is to provide enough diversity and redundancy to ensure the timely packet delivery while preserving energy and optimizing the use of the shared spectrum. The "RAW Architecture" document presents the RAW problem and architectural concepts such as path and Tracks to provide IPv6 flows with Service Level Objectives (SLO) in terms of packet delivery ratio (PDR), maximum contiguous losses or latency boundaries over wireless access and meshes. RAW distinguishes the longer time scale at which routes are computed from the the shorter forwarding time scale where per-packet decisions are made. RAW operates within the Network Plane at the forwarding time scale on one DetNet flow over a complex path called a Track. The Track is preestablished and installed by means outside of the scope of RAW; it may be strict or loose depending on whether each or just a subset of the hops are observed and controlled by RAW. The RAW Architecture is structured as an OODA Loop (Observe, Orient, Decide, Act). It involves:

Network Plane measurement protocols for Operations, Administration and Maintenance (OAM) to Observe some or all hops along a Track as well as the end-to-end packet delivery
Controller plane elements to reports the links statistics to a Path computation Element (PCE) in a centralized controller that computes and installs the Tracks and provides meta data to Orient the routing decision
A Runtime distributed Path Selection Engine (PSE) that Decides which subTrack to use for the next packet(s) that are routed along the Track
Packet (hybrid) ARQ, Replication, Elimination and Ordering Dataplane actions that operate at the DetNet Service Layer to increase the reliability of the end-to-end transmission. The RAW architecture also covers in-situ signaling when the decision is Acted by a node that down the Track from the PSE.

The RAW Framework combines IETF specification to enable RAW Service Level Agreements (SLA) over a selected technologies . The framework implements the OODA loop to optimizes the use of redundancy while minimizing the use of constrained resources such as spectrum and battery.

Terminology This document uses the terminology defined in the "RAW Architecture".

Use Cases and Requirements Served In order to focus on real-worlds issues and assert the feasibility of the proposed capabilities, RAW focuses on selected technologies that can be scheduled at the lower layers: IEEE Std. 802.15.4 timeslotted channel hopping (TSCH), 3GPP 5G ultra-reliable low latency communications (URLLC), IEEE 802.11ax/be where 802.11be is extreme high throughput (EHT), and L-band Digital Aeronautical Communications System (LDACS). See for more. "Deterministic Networking Use Cases" presents a number of wireless use cases including Wireless, such as application to Industrial Applications, Pro-Audio, and SmartGrid Automation. adds a number of use cases that demonstrate the need for RAW capabilities for new applications such as Pro-Gaming and drones. The use cases can be abstracted in two families, Loose Protection, e.g., protecting the first hop in Radio Access Protection and Strict Protection, e.g., providing End-to-End Protection in a wireless mesh.

Radio Access Protection To maintain the required SLA at all times, a wireless Host may use more than one Radio Access Network (RAN) in parallel.

Radio Access Protection ... .. RAN 1 ----- ... .. ... / . .. .... +--------+ / . .... +-----------+ |Wireless|- . ..... | Service | | Device |-***-- RAN 2 -- . Internet ....---| / | |(STA/UE)|- .. ..... |Application| +--------+ $$$ . ....... +-----------+ \ ... ... ..... RAN n -------- ... ..... *** = flapping at this time $$$ expensive The RANs may be heterogeneous, e.g., 3GPP 5G and Wi-Fi for high-speed communication, in which case a Layer-3 abstraction becomes useful to select which of the RANs are used at a particular point of time, and the amount of traffic that is distributed over each RAN. The idea is that the rest of the path to the destination(s) is protected separately (e.g., uses non-congruent paths, leverages DetNet / TSN, etc...) and is a lot more reliable, e.g., wired. In that case, RAW observes the reliability of the end-to-end operation through each of the RANs but only observes and controls the wireless operation the first hop. A variation of that use case has a pair of wireless Hosts connected over a wired core / backbone network. In that case, RAW observes and controls the Ingress and Egress RANs, while neglecting the hops in the core. The resulting loose Track may be instantiated, e.g., using tunneling or loose source routing between the RANs.

End-to-End Protection in a Wireless Mesh In radio technologies that support mesh networking (e.g., Wi-Fi and TSCH), a Track is a complex path with distributed PAREO capabilities. In that case, RAW operates through the multipath and makes decisions either at the Ingress or at every hop (more in ).

End-to-End Protection A-------B-------C-----D / \ / / \ Ingress ----M-------N--zzzzz--- Egress \ \ / / P--zzz--Q-------------R zzz = flapping now The Protection may be imposed by the source based on end-to-end OAM, or performed hop-by-hop, in which case the OAM must enables the intermediate Nodes to estimate the quality of the rest of the feasible paths in the remainder of the Track to the destination.

Related Work at The IETF RAW intersects with protocols or practices in development at the IETF as follows:

The Dynamic Link Exchange Protocol (DLEP) from can be leveraged at each hop to derive generic radio metrics (e.g., based on LQI, RSSI, queueing delays and ETX) on individual hops, and obtain link characteristics such as speed in a timely manner.
provides an OAM framework with that applies within the DetNet dataplane described in , which is typically based on MPLS or IPv6 pseudowires.
Bidirectional Forwarding Detection and its variants (bidirectional and remote BFD) detects faults in the path between an Ingress and an Egress forwarding engines. The art of BFD expects a serial path and needs one session per path, which makes it suited to observe a Segment it is unaware of the complexity of a track , and expects bidirectionality. to protect a path. BFD asynchronous mode considers delivery as success whereas with DetNet and RAW, the bounded latency can be as important as the delivery itself, and delivering too late is actually a failure. Note that the BFD Demand mode with unsolicited notifications may be more suitable then the Asynchronous BFD mode. The use of the Demand mode in MPLS is analyzed in and similar considerations could apply to IP as well.
and define in-band signaling that influences the routing when decided at the head-end on the path. There's already one RAW-related draft at BIER more may follow. RAW will need new in-band signaling when the decision is distributed, e.g., required chances of reliable delivery to destination within latency. This signaling enables relays to tune retries and replication to meet the required SLA.
defines protocol-independent metrics and parameters (measurement attributes) for describing links and paths that are required for routing and signaling in technology-specific networks. RAW would be a source of requirements for CCAMP to define metrics that are significant to the focus radios.
develops and maintains standard metrics that can be applied to the quality, performance, and reliability of Internet data delivery services and applications running over transport layer protocols (e.g. TCP, UDP) over IP.

Scope and Prerequisites A prerequisite to the RAW operation is that an end-to-end routing function computes a complex sub-topology along which forwarding can happen between a source and one or more destinations. The concept of Track is specified in the 6TiSCH Architecture to represent that complex sub-topology. Tracks provide a high degree of redundancy and diversity and enable the DetNet PREOF, network coding, and possibly RAW specific techniques such as PAREO, leveraging frequency diversity, time diversity, and possibly other forms of diversity as well. How the routing operation (e.g., PCE) in the Controller Plane computes the Track is out of scope for RAW. The scope of the RAW operation is one Track, and the goal of the RAW operation is to optimize the use of the Track at the forwarding timescale to maintain the expected SLA while optimizing the usage of constrained resources such as energy and spectrum. Another prerequisite is that an IP link can be established over the radio with some guarantees in terms of service reliability, e.g., it can be relied upon to transmit a packet within a bounded latency and provides a guaranteed BER/PDR outside rare but existing transient outage windows that can last from split seconds to minutes. The radio layer can be programmed with abstract parameters, and can return an abstract view of the state of the Link to help the Network Layer forwarding decision (think DLEP from MANET). How the radio interface manages its lower layers is out of control and out of scope for RAW. In the same fashion, the non-RAW portion along a loose Track is by definition out of control and out of scope for RAW. Whether it is a single hop or a mesh is also unknown and out of scope.

Wireless Tracks The "6TiSCH Architecture" introduces the concept of Track. RAW extends the concept to any wireless mesh technology, including, e.g., Wi-Fi. A simple Track is composed of a direct sequence of reserved hops to ensure the transmission of a single packet from a source Node to a destination Node across a multihop path. A Complex Track provides multiple N-ECMP forwarding solutions. The Complex Track enables to support multi-path redundant forwarding by employing PRE functions and the ingress and within the Track. For example, a Complex Track may branch off and rejoin over non-congruent segments. In the context of RAW, some links or segments in the Track may be reversible, meaning that they can be used in either direction. In that case, an indication in the packet signals the direction of the reversible links or segments that the packet traverses and thus places a constraint that prevents loops from occurring. An individual packet follows a destination-oriented directed acyclic graph (DODAG) towards a destination Node inside the Complex Track.

Flow Identification vs. Path Identification Section 4.7 of the DetNet Architecture ties the app-flow identification which is an application-layer concept with the network path identification that depends on the networking technology by "exporting of flow identification", e.g., to a MPLS label. With RAW, this exporting operation is injective but not bijective. e.g., a flow is fully placed within one RAW Track, but not all packets along that Track are necessarily part of the same flow. For instance, out-of-band OAM packets must circulate in the exact same fashion as the flows that they observe. It results that the network layer identification of an application layer flow (typically ther 5- or 6- tuple) must be separate from the path identification that is used to forward a packet. Section 3.4 of the DetNet data-plane framework indicates that for a DetNet IP Data Plane, a flow is identified by an IPv6 6-tuple. With RAW, that 6-tuple is not what indicates the Track, in other words, the flow ID is not the Track ID. For instance, the 6TiSCH Architecture uses a combination of the address of the Egress End System and an instance identifier in a Hop-by-hop option to indicate a Track. This way, if a packet "escapes" the Track, it will reach the Track Egress point through normal routing and be treated at the service layer through, say, elimination and reordering. The RAW service includes forwarding over a subset of the Links that form the Track (a subTrack). Packets from the same or a different flow that are routed through the same Track will not necessarily traverse the same Links. The PSE selects a subTrack for a packet based on the links that are preferred and those that should be avoided at this time. Each packet is forwarded within the subTrack that provides the best adequation with the SLA of the flow and the energy and bandwidth constraints of the network.

Flow Injection With 6TiSCH, packets are tagged with the same (source address, instance ID) will experience the same RAW service regardless of the IPv6 6-tuple that indicates the flow. The forwarding does not depend on whether the packets transport application flows or OAM. In the generic case, the Track or the subTrack can be signaled in the packet through other means, e.g., encoded in the suffix of the destination address as a Segment Routing Service Instruction , or leveraging Bit Index Explicit Replication Traffic Engineering .

Source-Routed vs. Distributed Forwarding Decision Within a large routed topology, the route-over mesh operation builds a particular complex Track with one source and one or more destinations; within the Track, packets may follow different paths and may be subject to RAW forwarding operations that include replication, elimination, retries, overhearing and reordering. The RAW forwarding decisions include the selection of points of replication and elimination, how many retries can take place, and a limit of validity for the packet beyond which the packet should be destroyed rather than forwarded uselessly further down the Track. The decision to apply the RAW techniques must be done quickly, and depends on a very recent and precise knowledge of the forwarding conditions within the complex Track. There is a need for an observation method to provide the RAW Data Plane with the specific knowledge of the state of the Track for the type of flow of interest (e.g., for a QoS level of interest). To observe the whole Track in quasi real time, RAW considers existing tools such as L2-triggers, DLEP, BFD and leverages in-band and out-of-band OAM to capture and report that information to the PSE. One possible way of making the RAW forwarding decisions within a Track is to position a unique PSE at the Ingress and express its decision in-band in the packet, which requires the explicit signaling of the subTrack within the Track. In that case, the RAW forwarding operation along the Track is encoded by the source, e.g., by indicating the subTrack in the Segment Routing (SRv6) Service Instruction, or by leveraging BIER-TE such as done with . The alternate way is to operate the PSE in each forwarding Node, which makes the RAW forwarding decisions for a packet on its own, based on its knowledge of the expectation (timeliness and reliability) for that packet and a recent observation of the rest of the way across the possible paths based on OAM. Information about the desired service should be placed in the packet and matched with the forwarding Node's capabilities and policies. In either case, a per-track/subTrack state is installed in all the intermediate Nodes to recognize the packets that are following a Track and determine the forwarding operation to be applied.

Encapsulation and Decapsulation In the generic case where the Track Ingress Node is not the source of the Packet, the Ingress Node needs to encapsulate IP-in-IP to ensure that the Destination IP Address is that of the Egress Node and that the necessary Headers (Routing Header, Segment Routing Header and/or Hop-By-Hop Header) can be added to the packet to signal the Track or the subTrack, conforming that discourages the insertion of a Header on the fly. In the specific case where the Ingress Node is the source of the packet, the encapsulation can be avoided, provided that the source adds the necessary headers and that the destination is set to the Egress Node. Forwarding to a final destination beyond the Egress Node is possible, e.g., with a Segment Routing Header that signals the rest of the way. In that case a Hop-by-Hop Header is not recommended since its validity is within the Track only.

Operations Administration and Maintenance

DetNet OAM provides an OAM framework with that applies within the DetNet dataplane described in ,which is typically based on MPLS or IPv6 pseudowires. How the framework applies to IPv6 is detailed in . Within that framework, OAM messages follow the same forward path as the data packets and gather information about their individual treatment at each hop. When the destination receives an OAM message, it gets a view on the full path or at least of a segment of the path from the source of the flow. In-situ OAM (IOAM) adds telemetry information about the experience of one packet within the packet itself , with the caveats that the measurement and the consecutive update of the packet interfere with the operation being observed, e.g., may increase the latency of the packet for which it is measured and into which it is stamped. Note: IOAM and analogous on-path telemetry methods are capable of facilitating collection of useful telemetry information that characterizes the state of a system as experienced by the packet. But because of statistical character of a packet network, these methods may not be used to monitor the continuity of a path (Track) or proper connectivity of the Track (no leaking packets across Tracks). This effect can be alleviated by measuring on the fly but reporting later, e.g., by exporting the data as a separate management packet . proposes an hybrid two-steps method (HTS) where a trigger message starts the measurement and a follow up along the Track packet gathers the measured data. "Error Performance Measurement" uses Fault Management (FM) and Performance Management (PM) OAM mechanisms to determine availability/unavailability of a path according to predefined SLA.

RAW Extensions Classical OAM typically measures information at the transmitter, e.g., residence time in the node or transmit queue size. With RAW, there is a need to combine information at the sender (number of retries) with that at the receiver (LQI, RSSI). This doubles the operating cost of an IAOM processing that would gather the experience of a single packet. The RAW PSE may be centralized at the Track Ingress, or distributed long the Track. Either way, the PSE needs instant information about the rest of the way to the destination over the possible next-hop adjacencies along the Track in order to decide how to perform simple forwarding, load balancing, and/or replication, as well as determining how much latency credit is available for ARQ. To provide that information timely, it makes sense that the OAM packets that gather instantaneous values from the radio senders and receivers at each hop flow on the reverse path and inform the PSE at the source and/or the PAREO relays about the state of the rest of the way. This is achieved using Reverse OAM packets that flow along the Reversed Track, West to East. Because the quality of transmission over a wireless medium varies continuously, it is important that RAW OAM captures the state of the medium across an adjacency over multiple transmission and over a recent period of time, whether the transmitted packets belong to this flow or another. Some of the measured information relates to the medium itself. In other words, the captured information does not only relate to the experience of one packet as is the case for IOAM, but also to the medium itself. This makes an approach like HTS more suitable as it can trigger the capture of multiple measurements over a short period of time. On the other hand, the PSE needs a continuous measurement stream where a single trigger is followed by a periodic follow up capture. In other words, the best suited OAM method to enable the PSE make accurate PAREO forwarding decisions is a periodic variation of the two-steps method flowing along the reverse Track, as an upstream OAM technique. provides more information on the RAW OAM problem and solution approaches.

Observed Metrics The Dynamic Link Exchange Protocol (DLEP) from can be leveraged at each hop to derive generic radio metrics (e.g., based on LQI, RSSI, queueing delays and ETX) on individual hops. Those lower-layer metrics are aggregated along a multihop segment into abstract layer 3 information that reflect the instant reliability and latency of the observed path.

Security Considerations RAW uses all forms of diversity including radio technology and physical path to increase the reliability and availability in the face of unpredictable conditions. While this is not done specifically to defeat an attacker, the amount of diversity used in RAW makes an attack harder to achieve.

Forced Access RAW will typically select the cheapest collection of links that matches the requested SLA, for instance, leverage free WI-Fi vs. paid 3GPP access. By defeating the cheap connectivity (e.g., PHY-layer interference) the attacker can force an End System to use the paid access and increase the cost of the transmission for the user.

IANA Considerations This document has no IANA actions.

Acknowledgments The editor wishes to thank:

Xavi Vilajosana:: Wireless Networks Research Lab, Universitat Oberta de Catalunya
Remous-Aris Koutsiamanis:: IMT Atlantique
Nicolas Montavont:: IMT Atlantique
Georgios Z. Papadopoulos:: IMT Atlantique
Rex Buddenberg:: Individual contributor
Greg Mirsky:: ZTE

for their contributions to the text and ideas exposed in this document.