FRR Inside SONiC
SONiC does not ship a proprietary routing stack. It runs FRRouting — zebra and bgpd — inside the bgp container, driven by vtysh and by SONiC's own configuration path. The interesting part for a CCNP engineer is not the protocol (it is standard BGP) but the plumbing: how FRR is configured, who owns that configuration, and how learned routes leave FRR and enter the rest of SONiC.
Prerequisites: Module 1 complete (Linux host + containers + Redis + SAI mental model). · CCNP-level BGP: sessions, address families, best-path, route maps.
Learning objectives
- →Identify the processes inside the bgp container (zebra, bgpd, staticd, fpmsyncd, vtysh) and what each owns.
- →Explain the two configuration surfaces — SONiC config vs. direct FRR config — and why ownership is version- and distribution-dependent.
- →Describe how routes computed in FRR are handed to the rest of SONiC via the FIB Push Manager path.
- →Avoid the trap of assuming a single, universal 'source of truth' for routing configuration.
Your BGP knowledge transfers unchanged. What changes is where the routing daemon lives, how you configure it, and how its output reaches hardware — and each of those is a distinct place to look when a route misbehaves.
The single most common early mistake is configuring FRR one way while SONiC expects to own it another way. Understanding the two configuration surfaces prevents config that silently disappears on the next reload.
What your CCNP already gives you
- • BGP sessions, address families, attributes, best-path — identical semantics.
- • The RIB/FIB distinction and administrative distance / route preference.
- • Reading a routing table and a BGP table to reason about reachability.
- •
vtyshfeels like a familiar IOS-style CLI because FRR modeled it that way.
What is new in SONiC
- • The routing daemon is a container you can enter, inspect, and restart independently of the rest of the switch.
- • Two configuration surfaces can coexist: SONiC's CONFIG_DB-driven flow and FRR's native config files.
- • FRR's selected routes are exported over FPM to a SONiC process, not pushed straight to the ASIC.
- • You can watch routes leave FRR and appear in APPL_DB — the boundary is observable.
| In the analogy | In SONiC |
|---|---|
| The buyer who negotiates with suppliers (BGP peers) and picks winning offers (best-path) | bgpd |
| The receiving desk that reconciles all buyers' picks into one accepted-goods ledger (the RIB) | zebra |
| Standing orders the manager wrote by hand (static routes) | staticd |
| The runner who carries the accepted-goods ledger to the production coordinator | fpmsyncd |
| The department's front counter where you read and write its ledgers | vtysh |
Where the analogy stops working
In a real factory the buyer and receiving desk are people who remember decisions. FRR daemons are stateless-on-restart in the sense that their runtime RIB is rebuilt from configuration and re-learned adjacencies — not from a durable ledger — so 'who owns the written configuration' is a separate question from 'what is in the running RIB right now.' Also, the factory has one obvious manager; SONiC deliberately has two possible owners of routing config (SONiC's config layer and FRR's own files), and which one is authoritative varies.
What is actually running
The bgp container hosts standard FRRouting. The processes you will meet are bgpd (the BGP speaker and best-path engine), zebra (the RIB manager and next-hop/interface resolver), staticd (static routes), optionally bfdd, zebra's FPM client, and the SONiC-specific fpmsyncd and bgpcfgd helpers.
vtysh is FRR's integrated shell. From the host you reach it with docker exec -it bgp vtysh, or on many images simply vtysh from the SONiC CLI. Everything you know from IOS/EOS-style routing CLIs — show ip bgp, show ip route, show bgp summary — works here because FRR intentionally mirrors that grammar.
The two configuration surfaces
This is the part that trips up experienced engineers. There are two ways routing can be configured in SONiC, and which one is authoritative depends on your version and distribution:
(1) The SONiC configuration path. BGP intent (neighbors, ASNs, address families, some route policy) lives in CONFIG_DB, and a SONiC helper — historically bgpcfgd — renders FRR configuration from it using templates. On images using the `unified`/`split-unified` config modes, SONiC treats its own configuration as the driver and regenerates FRR config from it.
(2) The native FRR path. On images running FRR in `split` config mode, you configure FRR directly (via vtysh and /etc/sonic/frr/*.conf), and FRR — not SONiC — owns that routing configuration. frr.conf on disk is the durable source, edited with vtysh -c 'write memory'.
The mode is controlled by the `DEVICE_METADATA|localhost` `docker_routing_config_mode` setting (values seen in the field include separated, unified, and split), and enterprise distributions may add their own configuration front-ends on top. Never assume the workflow from one deployment carries to another.
It is tempting to say 'CONFIG_DB / Redis is always authoritative for routing.' That is not universally true. In split mode FRR's own config files are authoritative; in unified modes SONiC drives FRR. Enterprise distributions vary further. Before you edit anything, determine the routing config mode with show runningconfiguration all / sonic-cfggen and by checking docker_routing_config_mode — then edit on the surface that actually owns the config, or your change may be overwritten on the next config reload.
Determine what mode you are in (read-only)
# What routing config mode is this device in?
sonic-db-cli CONFIG_DB HGET 'DEVICE_METADATA|localhost' docker_routing_config_mode
# -> "split" (FRR files authoritative)
# -> "unified" (SONiC config drives FRR)
# Confirm the routing container and FRR daemons are alive
docker ps --filter name=bgp
docker exec bgp vtysh -c 'show watchfrr status'
# bgpd, zebra, staticd -> UpCheck the routing config mode and confirm the bgp container is up before deciding where to make changes. Output below is illustrative.
Generated config vs. authoritative running state
Even where SONiC generates FRR config, the running state inside bgpd/zebra is what actually forwards decisions — and it can diverge from any on-disk file after live changes, transient failures, or partial reloads. Treat frr.conf (or the CONFIG_DB BGP tables) as *intent*, and vtysh -c 'show running-config' plus show ip route as *what FRR is actually doing right now*. Reconciling those two is a routine SONiC troubleshooting move that has no exact analog on a monolithic NOS.
Common misconceptions
Myth: SONiC has its own BGP implementation.
Reality: It uses upstream FRRouting. The protocol behavior is FRR's; SONiC adds the configuration rendering and the FPM-to-APPL_DB bridge.
Myth: Editing frr.conf is always the right way to change BGP in SONiC.
Reality: Only in config modes where FRR owns its config. In unified modes a template regenerates FRR config from CONFIG_DB and can overwrite hand edits. Check the mode first.
Myth: If bgpd selected the route, it is in hardware.
Reality: bgpd's best-path only reaches zebra; zebra's selected routes only reach APPL_DB via fpmsyncd. Hardware programming is a separate downstream stage (Lesson 2.3).
Go deeperWhy fpmsyncd instead of letting zebra program the kernel/hardware
On a plain Linux router, zebra installs selected routes into the kernel FIB and that is the end of the story. On a SONiC switch, the kernel is not the forwarding plane — the ASIC is. FRR's FPM interface was designed exactly for this: instead of (or in addition to) programming the kernel, zebra streams route add/delete messages to an external forwarding-plane manager. SONiC's fpmsyncd is that manager. It translates FPM messages into APPL_DB `ROUTE_TABLE` entries, which the orchestration layer then turns into SAI route objects. This is the clean seam between 'standard FRR' and 'SONiC-specific hardware programming.'
- Before editing routing config, identify the config mode; edit on the surface that owns it or the change may be reverted.
- 'The route is in bgpd' and 'the route is installed' are different claims — bgpd → zebra → fpmsyncd → APPL_DB are four separate checkpoints.
- If the bgp container is down, adjacencies drop but already-programmed hardware routes may persist for a while — control-plane state and forwarding state are decoupled (Lesson 2.4).
Key takeaways
- ✓SONiC routes with upstream FRR: bgpd (BGP + best-path), zebra (RIB/next-hop), staticd, plus SONiC's fpmsyncd bridge.
- ✓There are two configuration surfaces; ownership is set by docker_routing_config_mode and varies by distribution — do not assume one universal source of truth.
- ✓zebra's selected routes leave FRR over FPM and land in APPL_DB via fpmsyncd; hardware programming happens downstream.
- ✓Always separate intent (config files / CONFIG_DB) from running FRR state (show running-config, show ip route).
Tested against: SONiC 202305 / 202311 lineage · Upstream (community) unless noted; enterprise differences called out · SONiC-VS baseline; FRR 8.x lineage · last reviewed 2026-07. Exact commands and behavior can vary by SONiC release, image, hardware platform, and enterprise distribution.
Knowledge check
Which process inside the bgp container maintains the RIB and resolves next-hops and interfaces?
The device's docker_routing_config_mode is a SONiC-driven (unified) mode.
You edit /etc/sonic/frr/frr.conf by hand to add a neighbor. After a config reload the neighbor is gone. What is the most likely explanation?
Related lessons