LISP Traffic Engineeringlispers.netSan JoseCaliforniaUSA408-718-2001farinacci@gmail.comcisco Systems111 Wood Avenue SouthISELINNJUSAmikowal@cisco.comUSAparantap.lahiri@gmail.com
Intenet Area
Internet Engineering Task ForcetemplateThis document describes how LISP re-encapsulating tunnels can be used
for Traffic Engineering purposes. The mechanisms described in this
document require no LISP protocol changes but do introduce a new locator
(RLOC) encoding. The Traffic Engineering features provided by these
LISP mechanisms can span intra-domain, inter-domain, or combination of
both.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
"MAY", and "OPTIONAL" in this document are to be interpreted as
described in BCP 14 when, and only when, they appear in all
capitals, as shown here.This document describes extensions to the Locator/Identifier
Separation Protocol (LISP) for traffic engineering features.When LISP routers encapsulate packets to other LISP routers, the
path stretch is typically 1, meaning the packet travels on a direct path
from the encapsulating ITR to the decapsulating ETR at the destination
site. The direct path is determined by the underlying routing protocol
and metrics it uses to find the shortest path.This specification will examine how re-encapsulating tunnels
can be used so a packet can take an
administratively specified path, a congestion avoidance path, a
failure recovery path, or multiple load-shared paths, as it
travels from ITR to ETR. By introducing an Explicit Locator Path
(ELP) locator encoding , an ITR can
encapsulate a packet to a Re-Encapsulating Tunnel Router (RTR)
which decapsulates the packet, then encapsulates it to the next
locator in the ELP.Refer to for authoritative definitions for terms EID, RLOC, RTR,
and Recursive Tunneling. The other terms defined in this section add to the canonical
definition to reflect the design considerations in this specification.The ELP is an
explicit list of RLOCs for each RTR a packet must travel to
along its path toward a final destination ETR (or PETR). The
list is a strict ordering where each RLOC in the list is
visited. However, the path from one RTR to another is determined
by the underlying routing protocol and how the infrastructure
assigns metrics and policies for the path.An RTR is a
router that acts as an ETR (or PETR) by decapsulating packets
where the destination address in the "outer" IP header is one of
its own RLOCs. Then acts as an ITR (or PITR) by making a
decision where to encapsulate the packet based on the next
locator in the ELP towards the final destination ETR.Typically, a packet's path from source EID to destination EID travels
through the locator core via the encapsulating ITR directly to the
decapsulating ETR as the following diagram illustrates:Legend:Packet is originated by source EID 'seid'.Packet is consumed by destination EID 'deid'.Core routers in different ASes.The physical topological path between two
routers.A multi-hop LISP dynamic tunnel between LISP
routers.Typical Data Path from ITR to ETR A --> B --> C --> D ---)---> ETR deid |
| / || ( ) ^^ \ |
+--------+ || ( ) || +---------+
|| (----------------------------) ||
|| ||
===========================================
LISP Tunnel
]]>Let's introduce RTRs 'X' and 'Y' so that, for example, if it
is desirable to route around the path from B to C, one could
provide an ELP of (X,Y,etr):ELP tunnel path ITR ==> X, then X ==> Y, and then Y ==> ETR A --> B --> C --> D ---)---> ETR deid |
| / || ( / ^ ) ^^ \ |
| / || ( | \ ) || \ |
+-------+ || ( v | ) || +--------+
|| ( X ======> Y ) ||
|| ( ^^ || ) ||
|| (--------||---------||-------) ||
|| || || ||
================= =================
LISP Tunnel LISP Tunnel
]]>There are various reasons why the path from 'seid' to 'deid'
may want to avoid the path from B to C. To list a few:There may not be sufficient capacity provided by the networks
that connect B and C together.There may be a policy reason to avoid the ASes that make up the
path between B and C.There may be a failure on the path between B and C which makes
the path unreliable.There may be monitoring or traffic inspection resources close to RTRs
X and Y that do network accounting or measurement.There may be a chain of services performed at RTRs X and Y regardless
if the path from ITR to ETR is through B and C.The notation for a general formatted ELP is (x, y, etr) which represents
the list of RTRs a packet SHOULD travel through to reach the final tunnel
hop to the ETR.The procedure for using an ELP at each tunnel hop is as follows:The ITR will retrieve the ELP from the mapping database.The ITR will encapsulate the packet to RLOC 'x'.The RTR with RLOC 'x' will decapsulate the packet. It will use the
decapsulated packet's destination address as a lookup into the
mapping database to retrieve the ELP.RTR 'x' will encapsulate the packet to RTR with RLOC 'y'.The RTR with RLOC 'y' will decapsulate the packet. It will use the
decapsulated packet's destination address as a lookup into the
mapping database to retrieve the ELP.RTR 'y' will encapsulate the packet on the final tunnel hop to ETR
with RLOC 'etr'.The ETR will decapsulate the packet and deliver the packet to the
EID inside of its site.The specific encoding format for the ELP can be found in
. It is defined that an ELP will
appear as a single encoded locator in a locator-set. Say for
instance, we have a mapping entry for EID-prefix 10.0.0.0/8 that
is reachable via 4 locators. Two locators are being used as
active/active and the other two are used as active/active if the
first two go unreachable (as noted by the priority assignments
below). This is what the mapping entry would look like:If an ELP is going to be used to have a policy path to ETR-A and
possibly another policy path to ETR-B, the locator-set would be encoded
as follows:The mapping entry with ELP locators is registered to the
mapping database system just like any other mapping entry
would. The registration is typically performed by the ETR(s) that
are assigned and own the EID-prefix. That is, the destination
site makes the choice of the RTRs in the ELP. Alternatively, it may be
common practice for a provisioning system to program the mapping
database with ELPs.Another case where a locator-set can be used for flow-based load-sharing
across multiple paths to the same destination site:Using this mapping entry, an ITR would load split 75% of the EID flows
on the (x, y, ETR-A) ELP path and 25% of the EID flows on the (q, r, ETR-A)
ELP path. If any of the ELPs go down, then the other can take 100% of the
load.ELP re-optimization is a process of changing the RLOCs of an ELP
due to underlying network change conditions. Just like when
there is any locator change for a locator-set, the procedures from
the main LISP specification are followed.When a RLOC from an ELP is changed, Map-Notify messages
can be used
to inform the existing RTRs in the ELP so they can do a lookup to
obtain the latest version of the ELP. Map-Notify messages can also
be sent to new RTRs in an ELP so they can get the ELP in advance
to receiving packets that will use the ELP. This can minimize packet
loss during mapping database lookups in RTRs.In the previous examples, we showed how an ITR encapsulates
using an ELP of (x, y, etr). When a packet is encapsulated by
the ITR to RTR 'x', the RTR may want a policy path to RTR 'y'
and run another level of re-encapsulating tunnels for packets
destined to RTR 'y'. In this case, RTR 'x' does not encapsulate
packets to 'y' but rather performs a mapping database lookup on
the address 'y', requests the ELP for RTR 'y', and encapsulates
packets to the first-hop of the returned ELP. This can be done
when using a public or private mapping database. The decision to
use address 'y' as an encapsulation address versus a lookup
address is based on the L-bit setting for 'y' in the ELP
entry. The decision and policy of ELP encodings are local to the
entity which registers the EID-prefix associated with the
ELP.Another example of recursion is when the ITR uses the ELP (x,
y, etr) to first prepend a header with a destination RLOC of the
ETR and then prepend another header and encapsulate the packet
to RTR 'x'. When RTR 'x' decapsulates the packet, rather than
doing a mapping database lookup on RTR 'y' the last example
showed, instead RTR 'x' does a mapping database lookup on ETR
'etr'. In this scenario, RTR 'x' can choose an ELP from the
locator-set by considering the source RLOC address of the ITR
versus considering the source EID.This additional level of recursion also brings advantages for
the provider of RTR 'x' to store less state. Since RTR 'x' does
not need to look at the inner most header, it does not need to
store EID state. It only stores an entry for RTR 'y' which many
EID flows could share for scaling benefits. The locator-set for
entry 'y' could either be a list of typical locators, a list of
ELPs, or combination of both. Another advantage is that packet
load-splitting can be accomplished by examining the source of a
packet. If the source is an ITR versus the source being the
last-hop of an ELP the last-hop selected, different forwarding
paths can be used.Paths to an ETR may want to be selected based on different
classes of service. Packets from a set of sources that have
premium service can use ELP paths that are less congested
where normal sources use ELP paths that compete for less
resources or use longer paths for best effort service.Using source/destination lookups into the mapping database
can yield different ELPs. For example, a premium service
flow with (source=1.1.1.1, dest=10.1.1.1) can be described by
using the following mapping entry:And all other best-effort sources would use different mapping entry
described by:If the source/destination lookup is coupled with recursive
lookups, then an ITR can encapsulate to the ETR, prepending a
header that selects source address ITR-1 based on the premium
class of service source, or selects source address ITR-2 for
best-effort sources with normal class of service. The ITR then
does another lookup in the mapping database on the prepended header
using lookup key (source=ITR-1, dest=10.1.1.1) that returns the
following mapping entry:And all other sources would use different mapping entry with a lookup
key of (source=ITR-2, dest=10.1.1.1):This will scale the mapping system better by having fewer
source/destination combinations. Refer to the Source/Dest LCAF
type described in for encoding EIDs
in Map-Request and Map-Register messages.An ELP that is first used by an ITR must be inspected for
encoding loops. If any RLOC appears twice in the ELP, it MUST
not be used.Since it is expected that multiple mapping systems will be
used, there can be a loop across ELPs when registered in
different mapping systems. The TTL copying procedures for
re-encapsulating tunnels and recursive tunnels in
MUST be followed.An ELP can be used to deploy services at each reencapsulation
point in the network. One example is to implement a honey-pot
service when a destination EID is being DoS attacked. That is,
when a DoS attack is recognized when the encapsulation path is
between ITR and ETR, an ELP can be registered for a destination
EID to the mapping database system. The ELP can include an RTR so
the ITR can encapsulate packets to the RTR which will decapsulate
and deliver packets to a scrubber service device. The scrubber
could decide if the offending packets are dropped or allowed to be sent
to the destination EID. In which case, the scurbber delivers packets
back to the RTR which encapsulates to the ETR.Since an RTR knows the next tunnel hop to encapsulate to, it can monitor
the reachability of the next-hop RTR RLOC by doing RLOC-probing according
to the procedures in . When the RLOC is determined
unreachable by the RLOC-probing mechanisms, the RTR can use another
locator in the locator-set. That could be the final ETR, a RLOC of another
RTR, or an ELP where it must search for itself and use the next RLOC in
the ELP list to encapsulate to.RLOC-probing can also be used to measure delay on the path between
RTRs and when it is desirable switch to another lower delay ELP.Since an ELP-node knows the reachabiliy of the next ELP-node in
a ELP by using RLOC probing, the sum of reachability can determine
the reachability of the entire path. A head-end ITR/RTR/PITR can
determine the quality of a path and decide to select one path from
another based on the telemetry data gathered by RLOC-probing for
each encapsulation hop.ELP-probing mechanism details can be found in . defines procedures for how non-LISP sites
talk to LISP sites. The network elements defined in the Interworking
specification, the proxy ITR (PITR) and proxy ETR (PETR) (as well as
their multicast counterparts defined in ) can
participate in LISP-TE. That is, a PITR and a PETR can appear in an
ELP list and act as an RTR.Note when an RLOC appears in an ELP, it can be of any address-family.
There can be a mix of IPv4 and IPv6 locators present in the same ELP.
This can provide benefits where islands of one address-family or the
other are supported and connectivity across them is necessary.
For instance, an ELP can look like:(x4, a46, b64, y4, etr)Where an IPv4 ITR will encapsulate using an IPv4 RLOC 'x4' and 'x4'
could reach an IPv4 RLOC 'a46', but RTR 'a46' encapsulates to an IPv6
RLOC 'b64' when the network between them is IPv6-only. Then RTR 'b64'
encapsulates to IPv4 RLOC 'y4' if the network between them is
dual-stack. Note that RTRs can be used for NAT-traversal scenarios as well to reduce the
state in both an xTR that resides behind a NAT and the state the
NAT needs to maintain. In this case, the xTR only needs a default
map-cache entry pointing to the RTR for outbound traffic and all
remote ITRs can reach EIDs through the xTR behind a NAT via a
single RTR (or a small set RTRs for redundancy).
RTRs have some scaling features to reduce the number of
locator-set changes, the amount of state, and control packet overhead:When ITRs and PITRs are using a small set of RTRs for
encapsulating to "orders of magnitude" more EID-prefixes, the
probability of locator-set changes are limited to the RTR RLOC
changes versus the RLOC changes for the ETRs associated with the
EID-prefixes if the ITRs and PITRs were directly encapsulating
to the ETRs. This comes at an expense in packet stretch, but
depending on RTR placement, this expense can be mitigated.When RTRs are on-path between many pairwise EID flows, ITRs and PITRs
can store a small number of coarse EID-prefixes.RTRs can be used to help scale RLOC-probing. Instead of ITRs
RLOC-probing all ETRs for each destination site it has cached, the ITRs
can probe a smaller set of RTRs which in turn, probe the destination
sites.ELPs have application in multicast environments. Just like RTRs can
be used to provide connectivity across different address family islands,
RTRs can help concatenate a multicast region of the network to one that
does not support native multicast.Note there are various combinations of connectivity that can be
accomplished with the deployment of RTRs and ELPs:Providing multicast forwarding between IPv4-only-unicast regions
and IPv4-multicast regions.Providing multicast forwarding between IPv6-only-unicast regions
and IPv6-multicast regions.Providing multicast forwarding between IPv4-only-unicast regions
and IPv6-multicast regions.Providing multicast forwarding between IPv6-only-unicast regions
and IPv4-multicast regions.Providing multicast forwarding between IPv4-multicast regions
and IPv6-multicast regions.An ITR or PITR can do a (S-EID,G) lookup into the mapping database. What
can be returned is a typical locator-set that could be made up of
the various RLOC addresses:An entry for host 'S-EID' sending to application group 'G'The locator-set above can be used as a replication list. That is some
RLOCs listed can be unicast RLOCs and some can be delivery group RLOCs.
A unicast RLOC in this case is used to encapsulate a multicast packet
originated by a multicast source EID into a unicast packet for unicast
delivery on the underlying network. ETR-A could be an IPv4 unicast
RLOC address and ETR-B could be a IPv6 unicast RLOC address.A delivery group address is used when a multicast packet
originated by a multicast source EID is encapsulated in a
multicast packet for multicast delivery on the underlying network. Group
address 'g1' could be an IPv4 delivery group RLOC and group address
'g2' could be an IPv6 delivery group RLOC.Flexibility for these various types of connectivity
combinations can be achieved and provided by the mapping database
system. And the RTR placement allows the connectivity to occur
where the differences in network functionality is located.Extending this concept by allowing ELPs in locator-sets, one could
have this locator-set registered in the mapping database for (S-EID, G).
For example:Using ELPs for multicast flowsIn the above situation, an ITR would encapsulate a multicast
packet originated by a multicast source EID to the RTR with unicast
RLOC 'x'. Then RTR 'x' would decapsulate and unicast encapsulate
to RTR 'y' ('x' or 'y' could be either IPv4 or IPv6 unicast RLOCs),
which would decapsulate and unicast encapsulate to the final RLOC
'ETR-A'. The ETR 'ETR-A' would decapsulate and deliver the
multicast packet natively to all the receivers joined to application
group 'G' inside the LISP site.Let's look at the ITR using the ELP (a, g, b, ETR-B). Here the
encapsulation path would be the ITR unicast encapsulates to
unicast RLOC 'a'. RTR 'a' multicast encapsulates to delivery
group 'g'. The packet gets to all ETRs that have joined delivery
group 'g' so they can deliver the multicast packet to joined
receivers of application group 'G' in their sites. RTR 'b' is
also joined to delivery group 'g'. Since it is in the ELP, it will
be the only RTR that unicast encapsulates the multicast packet to
ETR 'ETR-B'. Lastly, 'ETR-B' decapsulates and delivers the multicast
packet to joined receivers to application group 'G' in its LISP site.As one can see there are all sorts of opportunities to provide multicast
connectivity across a network with non-congruent support for multicast
and different address-families. One can also see how using the mapping
database can allow flexible forms of delivery policy, rerouting, and
congestion control management in multicast environments.When an RTR receives a LISP encapsulated packet, it can look at
the outer source address to verify that RLOC is the one listed as
the previous hop in the ELP list. If the outer source RLOC address
appears before the RLOC which matches the outer destination RLOC
address, the decapsulating RTR (or ETR if last hop), MAY choose to
drop the packet.This document does not make any request to IANA.The authors would like to thank the following people for their
ideas and comments. They are Albert Cabellos, Khalid Raza, and Vina
Ermagan, Gregg Schudel, Yan Filyurin, Robert Raszuk, and Truman Boyes.Posted April 2024.Made changes to reflect comments from Luigi as we ready
document for standards track.Posted February 2024.Update references and document timer.Posted August 2023.Update references (to proposed standard documents) and document timer.Posted March 2023.Update references (to propsed standard documents) and document timer.Posted September 2022.Update document timer and references.Posted March 2022.Update document timer and references.Posted September 2021.Update document timer and references.Posted March 2021.Update document timer and references.Posted October 2020.Update document timer and references.Posted April 2020.Update document timer and references.Posted October 2019.Update document timer and references.Posted April 2019.Update document timer and references.Posted October 2018.Update document timer and references.Posted April 2018.Update document timer and references.Posted October 2017.Added section on ELP-probing that tells an ITR/RTR/PITR the
feasibility and reachability of an Explicit Lcoator Path.Posted April 2017.Changed draft-farinacci-lisp-te-12 to working group document.Many postings from January 2013 through February 2017.Update references and document timer.Posted July 2012.Add the Lookup bit to allow an ELP to be a list of encapsulation
and/or mapping database lookup addresses.Indicate that ELPs can be used for service chaining.Add text to indicate that Map-Notify messages can be sent to new
RTRs in a ELP so their map-caches can be pre-populated to avoid mapping
database lookup packet loss.Fixes to editorial comments from Gregg.Initial draft posted March 2012.