Module 2 · Control Plane, FRR & Route Lifecycle

Trace One Route Through the System

60 minLabCorrelationObservationVirtual

Take the prefix you learned in Lab 2.1 and document it at every layer: FRR (bgpd/zebra), the SONiC route command, APPL_DB, next-hop/neighbor state, ASIC_DB, and finally a data-plane test. This is a correlation lab — you are proving the same route exists (or does not) at each layer and filling in a tracing worksheet.

Safety: Safe

Non-disruptive. Safe to run in the virtual lab.

All reads. Use SCAN/targeted keys; avoid KEYS * on production databases as a matter of habit.

Reuses Lab 2.1

  sonic-a ── 10.0.12.0/31 ── peer
  trace peer prefix 10.2.2.0/24 on sonic-a

Objective

• Correlate one prefix across FRR, APPL_DB, neighbor state, and ASIC_DB.
• Practice targeted Redis reads (SCAN / specific keys), never KEYS *.
• Confirm forwarding with a data-plane test, not just table reads.
• Complete a route-tracing worksheet you can reuse in incidents.

Starting state

• eBGP Established; peer prefix 10.2.2.0/24 received on sonic-a.
• You are on sonic-a with host shell + vtysh access.

Prerequisites: Lab 2.1 established with a received prefix. · Lessons 2.2 and 2.3.

Tasks

1
Worksheet — layer-by-layer route trace (checklist)
Fill one row per layer for prefix 10.2.2.0/24. Record present/absent, the next-hop seen, and the exact command used:
[ ] bgpd Loc-RIB — best path present? next-hop?
[ ] zebra RIB — selected? next-hop + egress interface?
[ ] SONiC route view — matches zebra?
[ ] APPL_DB ROUTE_TABLE — entry present? nexthop/ifname?
[ ] NEIGH state — next-hop MAC resolved?
[ ] ASIC_DB — SAI_OBJECT_TYPE_ROUTE_ENTRY for the dest present?
[ ] Data plane — ping/traffic to a host in the prefix succeeds?
Expected:
- A completed worksheet with the first absent/unresolved layer (if any) circled.
What happened internally: The worksheet is the deliverable. The first layer where the route is missing localizes any fault to the hop just above it.
2
Layer 1–2: FRR (bgpd then zebra)
FRR / vtysh
```
vtysh -c 'show ip bgp 10.2.2.0/24'
```
FRR / vtysh
```
vtysh -c 'show ip route 10.2.2.0/24'
```
Expected:
- Best path in bgpd; selected route in zebra with a next-hop and egress interface.
What happened internally: This confirms best-path and RIB selection — the route has cleared FRR and should be exported over FPM.
3
Layer 3: SONiC route view
SONiC CLI
```
show ip route 10.2.2.0/24
```
Expected:
- The SONiC route command agrees with zebra.
What happened internally: show ip route in SONiC surfaces FRR's view; agreement here rules out a CLI-vs-daemon mismatch.
4
Layer 4: APPL_DB ROUTE_TABLE
SONiC DB
```
sonic-db-cli APPL_DB HGETALL 'ROUTE_TABLE:10.2.2.0/24'
```
Expected:
- An entry with nexthop and ifname fields matching the FRR next-hop.
What happened internally: Presence here proves fpmsyncd carried the route from zebra into APPL_DB — the seam between FRR and SONiC orchestration.
5
Layer 5: next-hop / neighbor resolution
SONiC DBnext-hop neighbor
```
sonic-db-cli APPL_DB HGETALL 'NEIGH_TABLE:Ethernet0:10.0.12.2'
```
Linux
```
ip neigh show 10.0.12.2
```
Expected:
- A resolved neighbor (MAC present), the dependency the route object needs.
What happened internally: Unresolved neighbor here is exactly the 'held pending' condition from Lesson 2.3 — a valid route that cannot be programmed.
6
Layer 6: ASIC_DB SAI route object
SONiC DBIllustrative outputSCAN, then HGETALL the entry whose dest is 10.2.2.0/24
```
sonic-db-cli ASIC_DB EVAL "return redis.call('SCAN',0,'MATCH','ASIC_STATE:SAI_OBJECT_TYPE_ROUTE_ENTRY:*','COUNT',200)" 0
```
Expected:
- A route-entry object whose destination matches the prefix (simulated on VS).
What happened internally: This is the hardware-intent layer. On real silicon syncd programs it into the ASIC; on VS it is simulated, so treat presence as a plumbing check only.
7
Layer 7: data-plane test
Linuxa live host/loopback inside the prefix
```
ping -c 3 10.2.2.1
```
Expected:
- Replies if forwarding works end to end (subject to VS simulation limits).
What happened internally: Table presence is necessary but not sufficient — the data-plane test is the only layer that proves actual forwarding.

Troubleshooting branches

If: Present in zebra, absent in APPL_DB

Then: Suspect fpmsyncd/FPM export; check the bgp container health and fpmsyncd logs.

If: In APPL_DB, no ASIC_DB object

Then: Check neighbor resolution and orchagent; on real hardware also check CRM resources (Lesson 2.4).

If: All layers present, ping fails

Then: On VS suspect simulation limits; on hardware check the reverse path and the peer's route back to your source.

Cleanup / rollback

None — read-only. Keep the completed worksheet as lab evidence.

Lab evidence checklist

☐ Completed route-tracing worksheet (all seven rows).
☐ APPL_DB ROUTE_TABLE entry captured.
☐ Neighbor resolution state captured.
☐ ASIC_DB route-entry SCAN result captured.
☐ Data-plane test result captured.

Reflection

Which layer would you check first in a real 'route learned but not forwarding' incident, and why?
What did the VS simulation let you verify, and what would only a real ASIC confirm?

Lab notes & observations