BGP Route Lifecycle
This lesson traces a single prefix end to end: a BGP UPDATE arrives, bgpd runs best-path, zebra selects and exports the route, fpmsyncd publishes it to APPL_DB, SWSS/orchagent resolves next-hop and neighbor dependencies, SAI objects are created, syncd programs the ASIC, and only then do packets forward. Each hop is a checkpoint you can observe.
Prerequisites: Lessons 2.1 and 2.2. · Module 1 SWSS/syncd/SAI model.
Learning objectives
- →Trace a prefix from BGP UPDATE to hardware forwarding through every SONiC stage.
- →Name the process and datastore involved at each hop.
- →Explain what 'next-hop resolution' and 'dependency resolution' mean in the SWSS/orchagent context.
- →Use the lifecycle as a checklist to localize where a new route stops.
When a newly advertised prefix does not forward, the fastest path to root cause is to walk the lifecycle and find the first stage where the route is absent or unresolved.
The stages are asynchronous. Understanding the ordering — and that later stages wait on dependencies — explains both normal convergence delay and stuck states.
What your CCNP already gives you
- • BGP UPDATE processing and best-path — unchanged.
- • The concept that a route needs a resolvable next-hop and adjacency to be usable.
- • Convergence intuition: events propagate and take time.
What is new in SONiC
- • The propagation is across processes and databases you can watch live.
- • Next-hop/neighbor resolution is performed by orchagent against NEIGH/APPL_DB state, and gates hardware programming.
- • Route programming becomes concrete SAI objects in ASIC_DB, and can be counted and error-checked.
| In the analogy | In SONiC |
|---|---|
| A new route suggestion arrives from a fellow driver | BGP UPDATE received |
| You compare it against your other options and pick a winner | best-path in bgpd |
| You commit to the directions and hand them to the installer | zebra selects + exports |
| The installer verifies the road actually connects and the on-ramp exists before programming it | orchagent next-hop/neighbor resolution |
| The directions are written into the nav unit's memory | SAI objects + syncd |
| The car takes the turn at speed | ASIC forwards |
Where the analogy stops working
A human installer works one job at a time; SONiC's orchagent processes many updates concurrently and event-driven, so ordering is logical, not strictly serial wall-clock. And unlike a nav unit that simply stores directions, SAI programming can *fail* on real silicon (table full, unsupported combination), a failure mode with no clean everyday-navigation analog — covered in Lesson 2.4.
The lifecycle of one prefix
- BGP UPDATE arrives at bgpd from a peer for prefix P. Observe:
show ip bgp neighbors,show ip bgp P. - Best-path selection in bgpd chooses the winning path for P and installs it into the Loc-RIB, then offers it to zebra.
- zebra RIB selection: zebra applies administrative distance across sources, selects P, and marks it for installation. Observe:
show ip route P. - Export toward SONiC: zebra streams the selected route over FPM; fpmsyncd receives it and writes an APPL_DB
ROUTE_TABLE:Pentry. Observe:sonic-db-cli APPL_DB HGETALL ROUTE_TABLE:P. - SWSS picks it up: the route orchagent (in the swss container) consumes the APPL_DB entry as an event.
- Dependency resolution: orchagent resolves the next-hop — the neighbor's NEIGH entry and MAC, the egress interface, and any NEXTHOP_GROUP for multipath. If unresolved, the route is held pending, not programmed.
- SAI object creation: once dependencies resolve, orchagent issues SAI calls to create the route entry (and next-hop / next-hop-group objects). These appear in ASIC_DB as
SAI_OBJECT_TYPE_ROUTE_ENTRY(and related) objects. - Hardware programming: syncd consumes ASIC_DB intent and programs the ASIC through the vendor SAI implementation/SDK.
- Packets forward: the silicon forwarding entry now exists; data-plane traffic to P is switched in hardware. Verify with a data-plane test, not just table reads.
Next-hop and neighbor resolution, concretely
'Resolve the next-hop' means: the orchagent must know how to reach the route's next-hop IP. That requires a neighbor entry (ARP/ND result) giving the next-hop's MAC, an egress interface that is oper-up and configured for L3, and — for ECMP — a NEXTHOP_GROUP object aggregating the members. These are themselves APPL_DB/STATE_DB-backed objects. Until they exist, the route object cannot be fully formed, so orchagent keeps the route pending and re-evaluates it when the missing neighbor or interface event arrives.
This is why a freshly learned route can take a beat to forward even on a healthy box: the neighbor may not be resolved yet. It is also why a route can stay stuck indefinitely if the neighbor never resolves — the subject of the next lesson.
Watch a route being born (read-only, illustrative)
PFX=10.100.5.0/24 ; NH=10.0.0.1
docker exec bgp vtysh -c "show ip route $PFX" # zebra selected?
sonic-db-cli APPL_DB HGETALL "ROUTE_TABLE:$PFX" # reached APPL_DB?
sonic-db-cli APPL_DB HGETALL "NEIGH_TABLE:Ethernet4:$NH" # neighbor resolved?
# ASIC route objects (SCAN, then match dest == PFX)
sonic-db-cli ASIC_DB EVAL \
"return redis.call('SCAN',0,'MATCH','ASIC_STATE:SAI_OBJECT_TYPE_ROUTE_ENTRY:*','COUNT',200)" 0Correlate the same prefix across FRR, APPL_DB, neighbor state, and ASIC_DB. Use SCAN/targeted keys, never KEYS *. Output is illustrative and version-dependent.
New prefix not forwarding? Walk the nine steps and find the first one where it is absent. Missing at zebra → it's a routing/policy problem. Present in APPL_DB but no ASIC object → orchagent/dependency/SAI problem. Present in ASIC_DB but traffic drops → data-plane/hardware or a resolution detail. The stage that first fails is your investigation's center of gravity.
- Localize by finding the first lifecycle stage where the prefix is missing or pending.
- A route held pending on next-hop resolution is a control-plane success but a forwarding failure — check NEIGH/interface state.
- SAI object present but no forwarding points downstream to syncd/silicon or a subtle next-hop-group issue.
Key takeaways
- ✓A prefix travels UPDATE → bgpd best-path → zebra select → fpmsyncd/APPL_DB → orchagent → SAI/ASIC_DB → syncd → silicon.
- ✓orchagent gates hardware programming on next-hop/neighbor/interface dependency resolution.
- ✓The stages are asynchronous and event-driven; convergence delay and stuck-pending states both come from this.
- ✓The lifecycle doubles as a bisection checklist for 'route learned but not forwarding.'
Tested against: SONiC 202305 / 202311 lineage · Upstream unless noted · SONiC-VS for observation; real ASIC for programming · last reviewed 2026-07. Exact commands and behavior can vary by SONiC release, image, hardware platform, and enterprise distribution.
Knowledge check
What carries zebra's selected routes out of FRR and into SONiC's APPL_DB?
A prefix is present in zebra and in APPL_DB ROUTE_TABLE, but there is no ASIC_DB route object and traffic is dropping. Where do you look next?
A prefix is withdrawn at the source. Order the removal as it propagates through the system.
- 1bgpd removes the path
- 2syncd tears down the silicon entry; forwarding stops
- 3zebra withdraws the route and signals FPM
- 4fpmsyncd deletes the APPL_DB ROUTE_TABLE entry
- 5orchagent issues SAI remove; ASIC_DB object deleted
Related lessons