]> Huawei Technologies

China c.l@huawei.com

China Mobile

China duzongpeng@chinamobile.com

Orange

France mohamed.boucadair@orange.com

Telefonica

Spain luismiguel.contrerasmurillo@telefonica.com

Juniper Networks, Inc.

United States of America jdrake@juniper.net

ZTE

China huang.guangping@zte.com.cn

Verizon Inc.

United States of America hayabusagsm@gmail.com

Routing area cats User Experience Collaborative Networking Service optimization This document describes a framework for Computing-Aware Traffic Steering (CATS). Particularly, the document identifies a set of CATS components, describes their interactions, and exemplifies the workflow of the control and data planes.

Introduction Edge computing architectures have been expanding from single edge nodes to multiple, sometimes collaborative, edge nodes to address various issues (e.g., long response times or suboptimal service and network resource usage). The underlying networking infrastructures that include edge computing resources usually provide relatively static service dispatching (that is, the selection of the sevice instances that will be invoked for a request). In such infrastructures, service-specific traffic is often directed to the closest edge resource from a routing perspective without considering the actual network state (e.g., traffic congestion conditions). As described in , traffic steering that takes into account computing resource metrics would benefit several services, including latency-sensitive service like immersive services that rely upon the use of augmented reality or virtual reality (AR/VR) techniques. This document provides an architectural framework that aims at facilitating the making of compute- and network-aware traffic steering decisions in networking environments where edge computing resources are deployed. The Computing-Aware Traffic Steering (CATS) framework assumes that there may be multiple service instances running on different edge nodes, globally providing one given service. A single edge node may have limited computing resources available at a given time, whereas the various edge nodes may experience different resource availability issues over time. A single edge node may also host multiple instances of a service or just one service instance. The CATS framework is an ingress-based overlay framework for the selection of the suitable service instance(s) from a set of instance candidates. The exact characterization of 'suitable' will be determined by a combination of networking and computing metrics. To that aim, the CATS framework assumes that edge nodes collaborate with each other under a single administrative domain to achieve a global objective of dispatching service demands (and thereby optimizing their processing by the most relevant edge computing resources) over the various and available edge computing resources, by taking into account both service instance status and network state (e.g., reachability considerations, path cost, and traffic congestion conditions). Also, this document describes a workflow of the main CATS procedures that are executed in both the control and data planes.

Terminology This document makes use of the following terms:

Client:

An endpoint that is connected to a service provider network.

Computing-Aware Traffic Steering (CATS):

A traffic engineering approach that takes into account the dynamic nature of computing resources and network state to optimize service-specific traffic forwarding towards a given service instance. Various relevant metrics may be used to enforce such computing-aware traffic steering policies.

CATS Service ID (CS-ID):

An identifier representing a service, which the clients use to access it. See .

CATS Binding ID (CB-ID):

An identifier of a single service instance or location of a given service instance (CS-ID). See .

Service:

An offering provided by a service provider and which is delivered using one or more service functions .

Service instance:

A run-time environment (e.g., a server or a process on a server) that makes a service instance available (i.e., up and running). One service can be accessed through multiple instances running at the same or different locations.

Service demand:

The demand for a service identified by a CATS Service ID (CS-ID).

Service request:

The request for a specific service instance.

CATS-Router:

A network device (usually located at the edge of the network) that makes forwarding decisions based on CATS information to steer traffic specific to a service demand towards a corresponding yet selected service instance. The selection of a service instance relies upon a multi-metric CATS-based path computation. A CATS router may behave as Ingress or Egress CATS-Router.

Ingress CATS-Router:

A node that serves as a service access point for CATS clients. It steers service-specific traffic along a CATS-computed path that leads to an Egress CATS-Router that connects to the most suitable edge site that hots the service instance selected to satisfy the initial service demand.

Egress CATS-Router:

A node that is located at the end of a CATS-computed path and which connects to a CATS-serviced site.

CATS Service Metric Agent (C-SMA):

An agent that is responsible for collecting service capabilities and status, and for reporting them to a CATS Path Selector (C-PS). See .

CATS Network Metric Agent (C-NMA):

A functional entity that is responsible for collecting network capabilities and status, and for reporting them to a C-PS. See .

CATS Path Selector (C-PS):

A computation logic that calculates and selects paths towards service locations and instances and which accommodates the requirements of service demands. Such a path computation engine takes into account the service and network status information. See .

CATS Traffic Classifier (C-TC):

A functional entity that is responsible for determining which packets belong to a traffic flow for a particular service demand. It is also responsible for forwarding such packets along the C-PS computed path that leads to the relevant service instance. See .

Framework and Components

Assumptions CATS assumes that there are multiple service instances running on different edge nodes, and which provide a given service that is represented by the same service identifier (see ).

CATS Identifiers CATS introduces the following identifiers:

CATS Service ID (CS-ID):

An identifier representing a service, which the clients use to access it. Such an ID identifies all the instances of a given service, rgardless of their location. The CS-ID is independent of which service instance serves the service demand. Service demands are spread over the service instances that can accommodate them, considering the location of the initiator of the service demand and the availability (in terms of resource/traffic load, for example) of the service instances resource-wise among other considerations like traffic congestion conditions.

CATS Binding ID (CB-ID):

An identifier of a single service instance or location of a given service instance (CS-ID).

CATS Components The network nodes make forwarding decisions for a given service demand that has been received from a client according to both service instances and network status information. The main CATS functional elements and their interactions are shown in .

CATS Functional Components

Edge Sites and Services Instances Edge sites (or edges for short) are the premises that provide access to edge computing resources. As mentioned in , a compute service (e.g., for face recognition purposes or a game server) is uniquely identified by a CATS Service IDentifier (CS-ID). Service instances can be instantiated and accessed through different edge sites so that a single service can be represented and accessed by several instances that run in different regions of the network. shows two edge nodes ("CATS-Router 1" and "CATS-Router 3") that provide access to service instances. These nodes behave as Egress CATS-Routers ().

• Note: "Egress" is used here in reference to the direction of the service request placement. The directionality is called to explicitly identify the exit node of the CATS infrastructure.

CATS Service Metric Agent (C-SMA) The CATS Service Metric Agent (C-SMA) is a functional component that gathers information about edge sites and server resources, as well as the status of the different service instances. The C-SMAs are located adjacent to the service instances and can be hosted by the Egress CATS-Routers () or located next to them. shows one C-SMA embedded in "CATS-Router 3", and another C-SMA that is adjacent to "CATS-Router 1".

The CATS Network Metric Agent (C-NMA) The CATS Network Metric Agent (C-NMA) is a functional component that gathers information about the state of the network. The C-NMAs may be implemented as standalone components or may be hosted by other components, such as CATS-Routers or CATS Path Selectors (C-PS) (). shows a single, standalone C-NMA within the underlay network. There may be one or more C-NMAs for an underlay network.

CATS Path Selector (C-PS) The C-SMAs and C-NMAs share the collected information with CATS Path Selectors (C-PSes) that use such information to select the Egress CATS-Routers (and potentially the service instances) where to forward traffic for a given service demand. C-PSes also determine the best paths (possibly using tunnels) to forward traffic, according to various criteria that include network state and traffic congestion conditions. The collected information is encoded into one or more metrics that feed the C-PS path computation logic. Such an information also includes CS-ID and possibly CB-ID identifiers. There may be one or more C-PSes used to compute CATS paths. They can be integrated into CATS-Routers (e.g., "CATS-Router 2" in ) or they may be standalone components that communicate with CATS-Routers (e.g., "CATS-Router 4" in ).

CATS Traffic Classifier (C-TC) CATS Traffic Classifier (C-TC) is a functional component that is responsible for associating incoming packets with existing service demands. CATS classifiers also ensure that packets that are bound to a specific service instance are all forwarded along the same path that leads to the same service instance, as instructed by a C-PS. CATS classifiers are typically hosted in CATS routers that are located at the edge of the network.

Overlay CATS-Routers The Egress CATS-Routers are the endpoints that behave as an overlay egress for service requests that are forwatded over a CATS infrastructure. An edge location that hosts service instances may be connected to one or more Egress CATS routers (that is, multi-homing is of course a design option). If a C-PS has selected a specific service instance and the C-TC has marked the traffic with the CB-ID, the Egress CATS-Router then forwards traffic to the relevant service instance. In some cases, the choice of the service instance may be left open to the Egress CATS-Router (i.e., traffic is marked only with the CS-ID). In such cases, the Egress CATS-Router selects a service instance using its knowledge of service and network capabilities as well as the current load as observed by the CATS router, among other considerations. Absent explicit policy, an Egress CATS-Router must make sure to forward all packets that pertain to a given service demand towards the same service instance. Note that, depending on the design considerations and service requirements, per-service instance computing-related metrics or aggregated per-site computing related metrics (and a combination thereof) can be used by a C-PS. Using aggregated per-site computing related metrics appears as a privileged option scalability-wise, but relies on Egress CATS-Routers that connect to various service instances to select the proper service instance.

Underlay Infrastructure The "underlay infrastructure" in indicates an IP/MPLS network that is not necessarily CATS-aware. The CATS paths that are computed by a P-CS will be distributed among the overlay CATS-Routers (), and will not affect the underlay nodes. A CATS implementation may rely upon a control or management plane to distribute service metrics and network metrics - this document does not define a specific solution.

Deployment Considerations This document does not make any assumption about how the various CATS functional elements are implemented and deployed. Concretely, whether a CATS deployment follows a fully distributed design or relies upon a mix of centralized (e.g., a C-PS) and distributed CATS functions (e.g., CATS traffic classifiers) is deployment-specific and may reflect the savoir-faire of the (CATS) service provider. Centralized designs where the computing related metrics from the C-SMAs are collected by a (logically) centralized path computation logic (e.g., a Path Computation Element (PCE) ) that also collects network metrics may be adopted. In the latter case, the CATS computation logic may process incoming service requests to compute and select paths and, therefore, service instances. The outcomes of such a computation process may then be communicated to CATS traffic classifiers (C-TCs).

CATS Framework Workflow The following subsections provide an overview of how the CATS workflow operates assuming a distributed CATS design.

Provisioning of CATS Components TBC: --detail required provisioning at CAST elements (booptsrapping, credentials of peer CAST nodes, services, optimization metrics per service, etc.)--

Service Announcement A service is associated with a unique identifier called a CS-ID. A CS-ID may be a network identifier, such as an IP address. The mapping of CS-IDs to network identifiers may be learned through a name resolution service, such as DNS .

Metrics Distribution As described in , a C-SMA collects both service-related capabilities and metrics, and associates them with a CS-ID that identifies the service. The C-SMA may aggregate the metrics for multiple service instances, or maintain them separately or both. The C-SMA then advertises the CS-IDs along with the metrics to be received by all C-PSes in the network. The service metrics include computing-related metrics and potentially other service-specific metrics like the number of end-users who access the service instance at any given time, their location, etc. Computing metrics may change very frequently (see for a discussion). How frequently such information is distributed is to be determined as part of the specification of any communication protocol (including routing protocols) that may be used to distribute the information. Various options can be considered, such as (but not limited to) interval-based updates, threshold-triggered updates, or policy-based updates. Additionally, the C-NMA collects network-related capabilities and metrics. These may be collected and distributed by existing routing protocols, although extensions to such protocols may be required to carry additional information (e.g., link latency). The C-NMA distributes the network metrics to the C-PSes so that they can use the combination of service and network metrics to determine the best Egress CATS-Router to provide access to a service instance and invoke the compute function required by a service demand. Network metrics may also change over time. Dynamic routing protocols may take advantage of some information or capabilities to prevent the network from being flooded with state change information (e.g., Partial Route Computation (PRC) of OSPFv3 ). C-NMAs should also be configured or instructed like C-SMAs to determine when and how often updates should be notified to the C-PSes. shows an example of how CATS metrics can be distributed. There is a client attached to the netowrk via "CATS-Router 1". There are three instances of the service with CS-ID "1": two are located at "Edge Site 2" attached via "CATS-Router 2" and have CB-IDs "1" and "2"; the third service instance is located at "Edge Site 3" attached via "CATS-Router 3" and with CB-ID "3". There is also a second service with CS-ID "2" with only one service instance located at "Edge Site 2". In , the C-SMA collocated with "CATS-Router 2" distributes the service metrics for both service instances (i.e., (CS-ID 1, CB-ID 1) and (CS-ID 1, CB-ID 2)). Note that this information may be aggregated into a single advertisement, but in this case, the metrics for each service instance are indicated separately. Similarly, the C-SMA agent located at "Edge Site 2" advertises the service metrics for the two services hosted by "Edge Site 2". The service metric advertisements are processed by the C-PS hosted by "CATS-Router 1". The C-PS also processes network metric advertisements sent by the C-NMA. All metrics are used by the C-PS to compute and select the most relevant path that leads to the Egress CATS-Router according to the initial client's service demand, the service that is requested ("CS-ID 1" or "CS-ID 2"), the state of the service instances as reported by the metrics, and the state of the network.

Example CATS Metric Distribution Service CS-ID 1, instance CB-ID 2 :<----------------------: : : +-------+ : : |CS-ID 1| : : +--|CB-ID 1| : +-------------+ | +-------+ : | C-SMA |----| Edge Site 2 : +-------------+ | +-------+ : |CATS-Router 2| +--|CS-ID 1| : +-------------+ |CB-ID 2| +--------+ : | +-------+ | Client | : Network +----------------------+ +--------+ : metrics | +-------+ | | : :<---------| C-NMA | | | : : | +-------+ | +-------------------+ | | |CATS-Router 1|C-PS |------| | +-------------------+ | Underlay | : | Infrastructure | +-------+ : | | |CS-ID 1| : +----------------------+ +---|CB-ID 3| : | | +-------+ : +-------------+ +-------+ : |CATS-Router 3|--| C-SMA | Edge Site 3 : +-------------+ +-------+ : : | +-------+ : : +---|CS-ID 2| : : +-------+ :<-------------------------------: Service CS-ID 1, instance CB-ID 3 Service CS-ID 2, ]]>

The example in mainly describes a per-instance computing-related metric distribution. In the case of distributing aggregated per-site computing-related metrics, the per-instance CB-ID information will not be included in the advertisement. Instead, a per-site CB-ID may be used in case multiple sites are connected to the Egress CATS-Router to explicitly indicate the site the aggregated metrics come from. A CB-ID is not required if the edge site can support consistently service instance selection.

Service Demand Processing The C-PS computes paths that lead to Egress CATS-Routers according to the service and network metrics that have been advertised. The C-PS may be collocated with an Ingress CATS-Router (as shown in ) or logically centralized. This document does not specify any algorithm for path computation and selection purposes, but it is expected that a service demand or local policy may feed the C-PS computation logic with Objective Functions that provide some information about the path characteristics (e.g., in terms of maximum latency) and the selected service instance. In the example shown in , when the client sends a service demand to "CATS-Router 1", the router solicits the C-PS to select a service instance hosted by an edge site that can be accessed through a particular Egress CATS-Router. The C-PS also determines a path to that Egress CATS-Router. This information is provided to the Ingress CATS-Router ("CATS-Router 1") so that it can forward packets to their proper destination, as computed by the C-PS. A service transaction consists of one or more service packets sent by the client to an Ingress CATS-Router to which the client is connected to. The Ingress CATS-Router classifies incoming packets received from clients by soliciting the CATS classifier (C-TC). When a matching classification entry is found for the packets, the Ingress CATS-Router encapsulates and forwards them to the C-PS selected Egress CATS-Router. When these packets reach the Egress CATS-Router, the outer header of the possible overlay encapsulation is removed and inner packets are sent to the relevant service instance.

• Note that multi-homed clients may be connected to multiple CATS domains that may be operated by the same or distinct service providers. This version of the framework does not cover multihoming specifics.

Service Instance Affinity Instance affinity means that packets that belong to a flow associated with a service should always be sent to the same Egress CATS-Router which will forward them to the same service instance. Furthermore, packets of a given flow should be forwarded along the same path to avoid mis-ordering and to prevent the introduction of unpredictable latency variations. The affinity is determined at the time of newly formulated service demands. Note that different services may have different notions of what constitutes a 'flow' and may, thus, identify a flow differently. Typically, a flow is identified by the 5-tuple transport coordinates (source and destination addresses, source and destination port numbers, and protocol). However, for instance, an RTP video stream may use different port numbers for video and audio channels: in that case, affinity may be identified as a combination of the two 5-tuple flow identifiers so that both flows are addressed to the same service instance. Hence, when specifying a protocol to communicate information about service instance affinity, a certain level of flexibility for identifying flows should be supported. Or, from a more general perspective, there should be a flexible mechanism to specify and identify the set of packets that are subject to a service instance affinity. More importantly, the means for identifying a flow for the purpose of ensuring instance affinity should be application-independent to avoid the need for service-specific instance affinity methods. However, service instance affinity information may be configurable on a per-service basis. For each service, the information can include the flow/packets identification type and means, affinity timeout value, etc. This document does not define any mechanism for defining or enforcing service instance affinity.

Security Considerations The computing resource information changes over time very frequently, especially with the creation and termination of service instances. When such an information is carried in a routing protocol, too many updates may affect network stability. This issue could be exploited by an attacker (e.g., by spawning and deleting service instances very rapidly). CATS solutions must support guards against such misbehaviors. For example, these solutions should support aggregation techniques, dampening mechanisms, and threshold-triggered distribution updates. The information distributed by the C-SMA and C-NMA agents may be sensitive. Such information could indeed disclose intel about the network and the location of compute resources hosted in edge sites. This information may be used by an attacker to identify weak spots in an operator's network. Furthermore, such information may be modified by an attacker resulting in disrupted service delivery for the clients, up to and including misdirection of traffic to an attacker's service implementation. CATS solutions must support authentication and integrity-protection mechanisms between C-SMAs/C-NMAs and C-PSes, and between C-PSes and Ingress CATS-Routers. Also, C-SMA agents need to support a mechanism to authenticate the services for which they provide information to C-PS computation logics, among other CATS functions.

Privacy Considerations Means to prevent that on-path nodes in the underlay infrastructure to fingerprint and track clients (e.g., determine which client accesses which service) must be supported by CATS solutions. More generally, personal data must not be exposed to external parties by CATS beyond what is carried in the packet that was originally issued by the client. Since the service will, in some cases, need to know about applications, clients, and even user identity, it is likely that the C-PS computed path information will need to be encrypted if the client/service communication is not already encrypted. For more discussion about privacy, refer to and .

IANA Considerations This document makes no requests for IANA action.

Informative References China Mobile Huawei Technologies Orange Telefonica Huawei Technologies Huawei Technologies China Mobile Many service providers have been exploring distributed computing techniques to achieve better service response time and optimized energy consumption. Such techniques rely upon the distribution of computing services and capabilities over many locations in the network, such as its edge, the metro region, virtualized central office, and other locations. In such a distributed computing environment, providing services by utilizing computing resources hosted in various computing facilities (e.g., edges) is being considered, e.g., for computationally intensive and delay sensitive services. Ideally, services should be computationally balanced using service-specific metrics instead of simply dispatching the service requests in a static way or optimizing solely connectivity metrics. For example, systematically directing end user-originated service requests to the geographically closest edge or some small computing units may lead to an unbalanced usage of computing resources, which may then degrade both the user experience and the overall service performance. We have named this kind of network with dynamic sharing of edge compute resources "Computing-Aware Networking" (CAN). This document provides the problem statement and the typical scenarios of CAN, which is to show the necessity of considering more factors when steering the traffic to the appropriate service instance based on the basic edge computing deployment to provide the service equivalency. Old Dog Consulting This document describes the principles of traffic engineering (TE) in the Internet. The document is intended to promote better understanding of the issues surrounding traffic engineering in IP networks and the networks that support IP networking, and to provide a common basis for the development of traffic engineering capabilities for the Internet. The principles, architectures, and methodologies for performance evaluation and performance optimization of operational networks are also discussed. This work was first published as RFC 3272 in May 2002. This document obsoletes RFC 3272 by making a complete update to bring the text in line with best current practices for Internet traffic engineering and to include references to the latest relevant work in the IETF. This document describes an architecture for the specification, creation, and ongoing maintenance of Service Function Chains (SFCs) in a network. It includes architectural concepts, principles, and components used in the construction of composite services through deployment of SFCs, with a focus on those to be standardized in the IETF. This document does not propose solutions, protocols, or extensions to existing protocols. Constraint-based path computation is a fundamental building block for traffic engineering systems such as Multiprotocol Label Switching (MPLS) and Generalized Multiprotocol Label Switching (GMPLS) networks. Path computation in large, multi-domain, multi-region, or multi-layer networks is complex and may require special computational components and cooperation between the different network domains. This document specifies the architecture for a Path Computation Element (PCE)-based model to address this problem space. This document does not attempt to provide a detailed description of all the architectural components, but rather it describes a set of building blocks for the PCE architecture from which solutions may be constructed. This memo provides information for the Internet community. This RFC is the revised basic definition of The Domain Name System. It obsoletes RFC-882. This memo describes the domain style names and their used for host address look up and electronic mail forwarding. It discusses the clients and servers in the domain name system and the protocol used between them. This document describes the modifications to OSPF to support version 6 of the Internet Protocol (IPv6). The fundamental mechanisms of OSPF (flooding, Designated Router (DR) election, area support, Short Path First (SPF) calculations, etc.) remain unchanged. However, some changes have been necessary, either due to changes in protocol semantics between IPv4 and IPv6, or simply to handle the increased address size of IPv6. These modifications will necessitate incrementing the protocol version from version 2 to version 3. OSPF for IPv6 is also referred to as OSPF version 3 (OSPFv3). Changes between OSPF for IPv4, OSPF Version 2, and OSPF for IPv6 as described herein include the following. Addressing semantics have been removed from OSPF packets and the basic Link State Advertisements (LSAs). New LSAs have been created to carry IPv6 addresses and prefixes. OSPF now runs on a per-link basis rather than on a per-IP-subnet basis. Flooding scope for LSAs has been generalized. Authentication has been removed from the OSPF protocol and instead relies on IPv6's Authentication Header and Encapsulating Security Payload (ESP). Even with larger IPv6 addresses, most packets in OSPF for IPv6 are almost as compact as those in OSPF for IPv4. Most fields and packet- size limitations present in OSPF for IPv4 have been relaxed. In addition, option handling has been made more flexible. All of OSPF for IPv4's optional capabilities, including demand circuit support and Not-So-Stubby Areas (NSSAs), are also supported in OSPF for IPv6. [STANDARDS-TRACK] On December 8-9, 2010, the IAB co-hosted an Internet privacy workshop with the World Wide Web Consortium (W3C), the Internet Society (ISOC), and MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL). The workshop revealed some of the fundamental challenges in designing, deploying, and analyzing privacy-protective Internet protocols and systems. Although workshop participants and the community as a whole are still far from understanding how best to systematically address privacy within Internet standards development, workshop participants identified a number of potential next steps. For the IETF, these included the creation of a privacy directorate to review Internet-Drafts, further work on documenting privacy considerations for protocol developers, and a number of exploratory efforts concerning fingerprinting and anonymized routing. Potential action items for the W3C included investigating the formation of a privacy interest group and formulating guidance about fingerprinting, referrer headers, data minimization in APIs, usability, and general considerations for non-browser-based protocols. Note that this document is a report on the proceedings of the workshop. The views and positions documented in this report are those of the workshop participants and do not necessarily reflect the views of the IAB, W3C, ISOC, or MIT CSAIL. This document is not an Internet Standards Track specification; it is published for informational purposes. This document offers guidance for developing privacy considerations for inclusion in protocol specifications. It aims to make designers, implementers, and users of Internet protocols aware of privacy-related design choices. It suggests that whether any individual RFC warrants a specific privacy considerations section will depend on the document's content.

Acknowledgements The authors would like to thank Joel Halpern, John Scudder, Dino Farinacci, Adrian Farrel, Cullen Jennings, Linda Dunbar, Jeffrey Zhang, Peng Liu, Fang Gao, Aijun Wang, Cong Li, Xinxin Yi, Jari Arkko, Mingyu Wu, Haibo Wang, Xia Chen, Jianwei Mao, Guofeng Qian, Zhenbin Li, and Xinyue Zhang for their comments and suggestions.

Contributors China Mobile

yaohuijuan@chinamobile.com

Huawei Technologies

liyizhou@huawei.com

Huawei Technologies

dirk.trossen@huawei.com

Huawei Technologies

luigi.iannone@huawei.com

Huawei Technologies

shihang9@huawei.com

New H3C Technologies

linchangwang.04414@h3c.com

CICT

xswang@fiberhome.com

Ruijie Networks

wangxuewei1@ruijie.com.cn

Orange

christian.jacquenet@orange.com