How to decapsule ERSPAN tunnel

Recently, I needed to send SPAN traffic to a network analyzer, but I had to resolve two problems:

1. The network analyzer was installed on a virtual machine.
2. I could not manage the virtual switch.

For these reasons, I decided to send ERSPAN (Encapsulated Remote Switched Port Analyze) traffic directly to the virtual machine but a problem still persisted. How can I remove ERSPAN header, if the software used is not able to decode the tunnel?

First of all, you need to find which ERSPAN version is used. To do it, open a wireshark / tcpdump on your virtual machine where the network analyzer software is installed and check the ERSPAN Version field as below:

There are two versions:

• Version 1 named as “type II” (this case)
• Version 2 named as “type III”

Below the ethernet packet of a ERSPAN type II session:

As you notice, a lot of overhead is added by GRE / ERSPAN protocol.

In that case, the ERSPAN configuration is defined on a Cisco Catalyst 9500; my goal is to mirror vlan 2020 IP traffic to the network analyzer (10.0.50.103).

ERSPAN Configuration

Ciscozine(config)#monitor session 10 type erspan-source
Ciscozine(config-mon-erspan-src)# source vlan 2020
Ciscozine(config-mon-erspan-src)# no shutdown 
Ciscozine(config-mon-erspan-src)# destination
Ciscozine(config-mon-erspan-src-dst)# erspan-id 10
Ciscozine(config-mon-erspan-src-dst)# mtu 1900
NOTE: Configured MTU of size 1900 will be part of ERSPAN payload and captured at dest port, rest will be discarded
Ciscozine(config-mon-erspan-src-dst)#  ip address 10.0.50.103
Ciscozine(config-mon-erspan-src-dst)#  origin ip address 10.0.50.1

Remember: Make sure you increase the MTU of the ERSPAN session and the IP path to accommodate the extra bytes of tunnel header.

Ciscozine#sh monitor session 10
 Session 10
 Type                     : ERSPAN Source Session
 Status                   : Admin Enabled
 Source VLANs             :
     Both                 : 2020
 Destination IP Address   : 10.0.50.103
 MTU                      : 1900
 Destination ERSPAN ID    : 10
 Origin IP Address        : 10.0.50.1
 Ciscozine#

Note: not all devices support MTU truncation, like in that case. If your devices is IOS-XE and you want to find that information use the command:

Ciscozine#show capability feature monitor erspan-source 
 ERSPAN Source Session:ERSPAN Source Session Supported: TRUE
 No of Rx ERSPAN source session: 8
 No of Tx ERSPAN source session: 8
 ERSPAN Header Type supported: II and III
 ACL filter Supported: TRUE
 SGT filter Supported: TRUE
 Fragmentation Supported: TRUE
 Truncation Supported: FALSE
 Sequence number Supported: FALSE
 QOS Supported: TRUE
 Ciscozine#

Linux configuration

In this POC, an Ubuntu 20.04 with two virtual NIC is used:

• ens160: the management server interface (192.168.100.28)
• ens192: the erspan destination (IP 10.0.50.103)

Increase ethernet MTU

ifconfig ens192 mtu 1900

Create a new virtual interface

ip link add dev cisco_erspan mtu 1900 type erspan seq key 10 local 10.0.50.103 remote 10.0.50.1 erspan_ver 1

Note:

• The new interface “cisco_erspan” decapsulates the GRE / ERSPAN tunnel
• The key must be equal to the “erspan-id” defined in the ERSPAN switch configuration . In that case the erspan-id is “10”, so the key must be “10”.
• The local IP is the ens192 address (the IP address of the virtual machine).
• The remote IP is the Catalyst 9500 address.
• The ERSPAN version is 1 (type II).

Enable the new virtual interface

ip link set cisco_erspan up

If you try to check cisco_erspan statistics, you notice that RX packets counters do not increase:

root@Ciscozine-Analyzer:/home/ciscozine#ifconfig cisco_erspan
cisco_erspan: flags=4163 mtu 1900
inet6 fe80::9cd6:77ff:fe47:c460 prefixlen 64 scopeid 0x20
ether 9e:d6:77:47:c4:60 txqueuelen 1000 (Ethernet)
RX packets 1 bytes 58 (58.0 B)
RX errors 19342 dropped 0 overruns 19342 frame 0
TX packets 21 bytes 2453 (2.4 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

to get more details:

root@Ciscozine-Analyzer:/home/ciscozine# ip -d -s -s link show dev cisco_erspan
10: cisco_erspan@NONE: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1900 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 9e:d6:77:47:c4:60 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 0 
    erspan remote 10.0.50.1 local 10.0.50.103 ttl inherit ikey 0.0.0.10 okey 0.0.0.10 iseq oseq erspan_index 0 erspan_ver 1 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
    RX: bytes  packets  errors  dropped overrun mcast   
    58         1        173262  0       0       0       
    RX errors: length   crc     frame   fifo    missed
               0        0       0       173262  0       
    TX: bytes  packets  errors  dropped carrier collsns 
    2954       27       0       0       0       0       
    TX errors: aborted  fifo   window heartbeat transns
               0        0       0       0       0       
root@Ciscozine-Analyzer:/home/ciscozine#

The fifo errors are caused by input sequence number mismatch. In fact, when ERSPAN tunnel receives mirrored packet, it checks whether the mirrored packets are in sequence (based on the sequence number in the erspan header). If it’s not, then kernel will drop it.

To avoid the dropping

ip link set dev cisco_erspan type erspan noiseq

The rx errors stop to increase, the RX packets counters begin to increase and the network analyzer starts to analyze IP packets!

References:

• http://vger.kernel.org/lpc_net2018_talks/erspan-linux-presentation.pdf
• https://tools.ietf.org/html/draft-foschiano-erspan-00