Linux Under the Switch
Underneath every SONiC container is an ordinary Linux host: systemd, docker, journald, /etc, and a management NIC. This lesson maps the host you already know onto the switch you are learning, and separates the host's own networking from the front-panel ports the ASIC owns.
Prerequisites: Lesson 1.1 complete. · Comfort with systemd, docker, and journalctl on any Linux host.
Learning objectives
- →Locate the Linux primitives (systemd, docker, journald, filesystem) that host the NOS.
- →Distinguish the Linux management shell from the SONiC CLI, and know when you have left one for the other.
- →Read SONiC's processes, services, and logs using standard Linux tools.
- →Separate host networking (eth0/mgmt, its own routing table) from the front-panel EthernetN interfaces the ASIC drives.
When a container will not start, a filesystem fills, or a service crash-loops, you troubleshoot it with the same Linux skills you already have — but only if you recognize that the host is plain Linux.
Confusing the management interface with a front-panel port, or the Linux netdev with the ASIC port, is one of the most common early SONiC mistakes and sends you debugging the wrong plane.
What your CCNP already gives you
- • Out-of-band management concepts: a dedicated mgmt interface in its own reachability domain.
- • The idea that control-plane traffic is punted to a CPU while data-plane traffic stays in hardware.
- • General Linux operations if you have run OS10, NX-OS guestshell, or any Linux server.
What is new in SONiC
- • The 'NOS' is directly inspectable as Linux processes and containers — no hidden monolith.
- • Configuration persists as files on disk (config_db.json) in addition to living in Redis.
- • Front-panel ports appear as Linux netdevs but are backed by ASIC objects, not the kernel's forwarding.
- • Logs are split between the host (journald/syslog) and per-container logs.
| In the analogy | In SONiC |
|---|---|
| The building, its power, plumbing, and security desk | The Linux host (kernel, init) |
| The facilities manager who starts/stops each department on schedule | systemd |
| The leased department suites inside the building | docker |
| The building-wide logbook and incident record | journald / rsyslog |
| The filing cabinets and the master blueprint on disk | /etc/sonic, config_db.json |
| The staff entrance and phone line — how people reach the office | eth0 / management port |
| The loading docks the production machinery actually ships through | EthernetN front-panel ports |
systemctl, journalctl, ps, df, ip, docker — works exactly as it does on a server. The management interface (eth0) is a normal Linux NIC with its own address and routing; the front-panel ports are a different animal owned by the ASIC.Where the analogy stops working
The building metaphor breaks in two places. First, the front-panel EthernetN interfaces you see with ip link are only *representations* of ASIC ports — the kernel netdev is a control/exception path, not the forwarding path; bulk traffic never touches the CPU. Second, unlike a normal server, the host's real job is to program silicon, so a 'healthy Linux host' can still be forwarding nothing if syncd or the ASIC path is broken.
It really is just Linux
Log into a SONiC device and you land in a bash shell on a Debian-based host. The switch's features are docker containers supervised by systemd. The state store is a redis-server process. The routing stack is FRRouting inside a container. Hardware programming is a process called syncd. All of these are ordinary Linux processes you can list with ps and inspect with docker.
This matters because your Linux troubleshooting reflexes transfer directly: a crash-looping service, a full /var/log, a wedged process, a bad DNS resolver — you diagnose all of these the same way you would on a server.
Two shells, one prompt
SONiC gives you the host bash shell and a set of `show`/`config`/`sonic-cli` commands layered on top. The show and config commands are Python programs that read and write Redis; bash is the underlying host. You frequently move between them in a single session, and it is important to know which one you are in.
A useful habit: show and config are the SONiC layer (they touch CONFIG_DB and friends); systemctl, docker, journalctl, ip, ps, and redis-cli/sonic-db-cli are the Linux/host layer beneath it. Some distributions (for example Dell Enterprise SONiC) also ship an industry-standard sonic-cli mode that feels like a classic NOS — its availability is distribution-dependent.
The Linux primitives that host the NOS
| Concern | Linux mechanism | How you inspect it | Notes |
|---|---|---|---|
| Service supervision | systemd units (e.g. swss.service, bgp.service) | systemctl status, systemctl --failed | Many SONiC units wrap a docker container lifecycle. |
| Feature containers | docker | docker ps, docker inspect | Container set varies by image/platform. |
| Logs | journald and/or rsyslog | journalctl -u <unit>, show logging | Split between host and per-container; backend varies by version. |
| Persistent config | files under /etc/sonic | ls /etc/sonic, cat config_db.json | config_db.json is the on-disk copy of CONFIG_DB. |
| Processes | kernel scheduler | ps -ef, top | orchagent, syncd, redis-server, zebra/bgpd are all here. |
| Host networking | kernel netdevs + routing tables | ip addr, ip route, ip link | eth0 = mgmt; EthernetN = ASIC-backed netdevs. |
Processes and services, seen as Linux
admin@sonic:~$ systemctl --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
0 loaded units listed.
admin@sonic:~$ docker ps --format '{{.Names}}'
database
swss
syncd
bgp
teamd
lldp
pmon
snmp
telemetry
admin@sonic:~$ ps -ef | grep -E 'orchagent|syncd|redis-server' | grep -v grep
redis ... redis-server *:6379
root ... /usr/bin/orchagent -d /var/log/swss ...
root ... /usr/bin/syncd --diag ...The same subsystems you configure through show/config appear here as plain processes and systemd units. Output is illustrative and will differ by image and platform.
The filesystem: where config actually lives on disk
Under /etc/sonic you will find config_db.json — the serialized, on-disk form of CONFIG_DB written by config save. This is a crucial nuance for later modules: Redis holds the live desired state, but it is populated from disk at boot, and unsaved Redis changes are lost on reboot. Redis is authoritative *at runtime*, not the sole source of truth across a power cycle.
Other artifacts you will meet: platform data under /usr/share/sonic, per-container logs, and (on some deployments) a minigraph.xml or YANG/GCU-based config pipeline. The exact config-generation path is version- and deployment-dependent — do not assume every device was configured the same way.
A change made only in Redis (or via a live config command without config save) behaves like an unsaved running-config: it works now and vanishes on reboot. sudo config save -y writes config_db.json. Treat this exactly like write memory discipline from your CCNP days.
Logs: journald, syslog, and per-container
SONiC logging is layered. The host uses journald and/or rsyslog (the backend and defaults vary by version); show logging is a convenience wrapper over the syslog stream. Individual containers also emit logs (for example swss writes orchagent logs under /var/log/swss). When a subsystem misbehaves, you correlate the host journal (journalctl -u swss) with the container's own logs.
Management interface vs front-panel interfaces
| Aspect | eth0 (management) | EthernetN (front-panel) |
|---|---|---|
| Owner | The Linux kernel — an ordinary NIC. | The ASIC; the netdev is only a representation/exception path. |
| Purpose | Out-of-band reachability to the host (SSH, mgmt). | Data-plane forwarding at line rate. |
| Routing | Uses a dedicated mgmt VRF/table on many images. | Forwarding is in silicon, not the host route table. |
| Traffic path | All traffic goes through the CPU/kernel. | Bulk traffic never touches the CPU; only punted/control traffic does. |
| You inspect with | ip addr show eth0, mgmt VRF tools. | show interfaces status, then the DB/ASIC path (Lessons 1.4–1.5). |
Seeing Ethernet0 up in ip link confirms the kernel representation is up — it does not confirm the ASIC is forwarding on that port. Equating the Linux netdev state with hardware state is a classic misdiagnosis. Confirm hardware-facing state through ASIC_DB/STATE_DB, not ip link alone.
Common misconceptions
Myth: SONiC has its own special OS underneath the CLI.
Reality: The host is a standard Debian-based Linux. The 'specialness' is the container + Redis + SAI model layered on top, not the OS primitives.
Myth: If the Linux netdev for Ethernet0 is up, the port is forwarding.
Reality: The netdev is a control/exception representation of the ASIC port. Forwarding state lives in the ASIC and is observed via ASIC_DB/STATE_DB.
Myth: Redis is the only place my configuration lives.
Reality: Live desired state is in Redis, but it is loaded from config_db.json on disk at boot. Unsaved Redis changes do not survive a reboot.
- Start systemic problems at the host:
systemctl --failedandjournalctlbefore assuming a networking bug. - Never conclude a port is forwarding from
ip linkstate alone — that is host-layer evidence only. - If config 'disappeared' after a reboot, suspect an unsaved change: check config_db.json vs live CONFIG_DB.
Key takeaways
- ✓The switch is a Debian-based Linux host; systemd, docker, journald, and /etc all apply.
- ✓
show/configare the SONiC layer; bash/systemctl/docker/journalctl/redis are the host layer beneath. - ✓eth0/mgmt is a normal kernel NIC; EthernetN are ASIC-backed representations, not the forwarding path.
- ✓Config lives both in Redis (runtime) and config_db.json (on disk); save to persist.
Tested against: SONiC 202305 / 202311 lineage · Upstream (community); Debian-based host · SONiC-VS (virtual) baseline · last reviewed 2026-07. Exact commands and behavior can vary by SONiC release, image, hardware platform, and enterprise distribution.
Knowledge check
How does the management interface (eth0) differ from a front-panel interface such as Ethernet0 on a SONiC switch?
Single best answer.
After a reboot, an interface IP you configured yesterday is gone, even though it worked all afternoon. The change was made via config interface ip add but nothing else. What most likely happened?
Related lessons