Packet Capture

50 minLabPacket tracingObservationVirtualOpen-source

Packet-tracing lab: capture the VXLAN traffic on the underlay and dissect one packet layer by layer — original tenant Ethernet frame, then the outer Ethernet, outer IP, UDP, and the VXLAN header with its VNI — proving on the wire what the control plane claimed.

Safety: Caution

May change state — take a snapshot first.

tcpdump is read-only, but running it privileged and writing files warrants care on shared labs — hence 'caution'.

Two-leaf VXLAN lab (Host1—Leaf1—underlay—Leaf2—Host2)

  Host1 (Linux)                                   Host2 (Linux)
  10.10.10.11/24                                  10.10.10.12/24
      │ eth1                                           eth1 │
      │                                                     │
  ┌───┴────────┐        virtual underlay          ┌────────┴───┐
  │  Leaf1     │  Ethernet0  ── p2p ──  Ethernet0  │  Leaf2     │
  │  SONiC-VS  │◀───────── routed IP fabric ──────▶│  SONiC-VS  │
  │  Lo0 =     │        (BGP, ECMP-capable)        │  Lo0 =     │
  │  10.0.0.1  │                                   │  10.0.0.2  │
  └────────────┘                                   └────────────┘
   access: EthernetX (VLAN 10 ↔ VNI 10010)   access: EthernetX (VLAN 10 ↔ VNI 10010)

Objective

• Capture VXLAN-encapsulated traffic on the underlay interface (UDP 4789).
• Identify each header: outer Ethernet, outer IP (VTEP src/dst), UDP, VXLAN + VNI, inner Ethernet.
• Confirm the VNI on the wire matches the configured L2VNI and the outer IPs match the loopbacks.

Starting state

• Host1 ↔ Host2 forwarding across the overlay; a shell on Leaf1 (or the underlay path) able to run tcpdump.

Prerequisites: Lab 4.4 complete (forwarding working, MAC traced).

Tasks

1
Start a capture on the underlay interface
Filter on the VXLAN UDP port so you only see overlay traffic, not control plane.
Linux
```
sudo tcpdump -ni Ethernet0 udp port 4789 -c 20 -w /tmp/vxlan.pcap
```
Expected:
- Capture file collects encapsulated packets while traffic flows.
What happened internally: Rung 9 of the ladder. Capturing on the underlay interface shows the encapsulated form — the outer headers the fabric sees.
2
Generate traffic during the capture
Linuxfrom Host1, while tcpdump runs
```
ping -c 10 -s 200 10.10.10.12
```
Expected:
- The capture fills with UDP/4789 packets between the two loopbacks.
What happened internally: Deterministic size/count makes the packets easy to find and dissect.
3
Dissect the layers
Read the packet from the outside in and match each field to configuration.
Linuxverbose, with link-layer
```
sudo tcpdump -nr /tmp/vxlan.pcap -vvv -e
```
Expected:
- Outer Ethernet (VTEP↔nexthop MAC); Outer IP src=10.0.0.1 dst=10.0.0.2 (the loopbacks); UDP dst 4789; VXLAN header with VNI 10010; inner Ethernet = Host1↔Host2 MACs; inner payload = the ICMP.
What happened internally: This is Lesson 4.1's encapsulation diagram made real: outer Eth / outer IP / UDP / VXLAN(VNI) / inner Eth / payload.
4
Verify the wire matches intent
SONiC CLIcompare VNI on wire vs configured
```
show vxlan vlanvnimap
```
SONiC CLIcompare outer IPs vs tunnel src/dst
```
show vxlan tunnel
```
Expected:
- VNI in the VXLAN header equals the configured L2VNI (10010); outer IPs equal the two Loopback0 addresses.
What happened internally: A VNI mismatch on the wire vs config, or wrong outer IPs, immediately localizes a Lab 4.6-style fault.

Troubleshooting branches

If: No UDP/4789 packets captured

Then: Confirm forwarding actually works (Lab 4.4), capture on the right underlay interface, and ECMP isn't hashing to a different link.

If: VNI on wire ≠ configured VNI

Then: VLAN-VNI map mismatch between leaves — a classic failure; re-check show vxlan vlanvnimap on both.

If: Inner frame looks wrong/truncated

Then: MTU/fragmentation on the underlay; revisit Lab 4.1 MTU validation.

Cleanup / rollback

Remove capture files (sudo rm /tmp/vxlan.pcap).
State can remain for Lab 4.6.

Lab evidence checklist

☐ A pcap containing UDP/4789 VXLAN packets.
☐ All headers identified: outer Eth, outer IP, UDP, VXLAN+VNI, inner Eth, payload.
☐ VNI on the wire matches configured L2VNI; outer IPs match the loopbacks.

Reflection

Why does capturing on the underlay show encapsulation while a capture on the access port shows the bare tenant frame?
If the VNI on the wire were 10011 instead of 10010, which object would you suspect and why?

Lab notes & observations