System Architecture

The SONiC operating system is built around a series of Docker containers, each controlling a specific functional component of the operating system. The containers in turn communicate via a Redis database that also provides persistent data storage, replication and a language-independent interface.

The containers are:

  • bgp: Runs the FRRouting stack. The BGP container runs the following services:
    • bgpd: The standard BGP service. The routing state from external parties is received through regular TCP or UDP sockets and pushed down to the forwarding plane through the zebra/fpmsyncd interface.
    • zebra: The traditional IP routing manager. It provides kernel routing table updates, interface lookups and route redistribution services across the various routing protocols. zebra also pushes the computed FIB down to the kernel via netlink and to southbound components involved in the forwarding process via the Forwarding Plane Manager (FPM).
    • fpmsyncd: This service collects the FIB state generated by zebra and dumps its contents into the application database (APPL_DB) within the redis engine.
  • database: The container where the Redis databases reside. The main databases hosted by the Redis engine are:
    • APPL_DB: Stores the state generated by all application containers — routes, next hops, neighbors and so forth. Any application that needs to interact with other SONiC subsystems does so through APPL_DB.
    • CONFIG_DB: Stores the configuration state created by SONiC applications, including port configurations, interfaces and VLANs.
    • STATE_DB: Stores the essential operational state for elements configured in the operating system. The state resolves dependencies between different SONiC subsystems. For example, a LAG port channel defined in the teamd submodule can potentially refer to physical ports that may or may not be present in the system. Another example is a VLAN configured via vlanmgrd, which may reference port members whose presence is undetermined in the system.
    • ASIC_DB: Stores the necessary state to drive the ASIC configuration and operation. The ASIC state is kept in an ASIC-friendly format to ease the interaction between syncd and the ASIC SDKs.
    • COUNTERS_DB: Stores counters and statistics associated with each port in the system. This state can be utilized for local CLI requests or to feed a telemetry channel for remote consumption.
  • dhcp-relay: The DHCP relay agent enables the relaying of DHCP requests from a subnet with no DHCP server to one or more DHCP servers on other subnets.
  • lldp: The link layer discovery protocol container, in which the following processes run:
    • lldpd: The LLDP service, which establishes the LLDP connections with external peers to advertise and receive system capabilities.
    • lldp_syncd: This service uploads the LLDP discovered state to the redis engine — the centralized system’s message infrastructure — delivering the LLDP state to applications that require it, such as SNMP.
    • lldpmgr: The configuration tool for the lldpd service.
  • pmon: This container is where the sensord service runs. sensord periodically logs sensor readings from hardware components and alerts when an alarm is triggered. The pmon container also hosts the fancontrol process, which collects fan-related state from the corresponding platform drivers.
  • radv: The route advertiser container.
  • snmp: Hosts SNMP features. There are two relevant processes within this container:
    • snmpd: The server handling incoming SNMP polls from external network elements.
    • snmp-agent: This is SONiC’s implementation of an AgentX SNMP subagent, sonic_ax_impl. The sonic_ax_impl subagent feeds the master agent (snmpd) with information collected from SONiC databases in the centralized Redis engine.
  • swss: The switch state service (swss) container is a suite of tools to coordinate communication among all the SONiC modules and between the modules and the redis engine. swss also hosts the following services, which interact with the SONiC application layer through netlink (except for the processes run in other containers, namely fpmsyncd, teamsyncd and lldp_syncd). The first three services below push state to the redis engine while the last three utilize and redistribute state gathered from the engine.
    • portsyncd: Listens to port-related netlink events. When the switch boots up, portsyncd parses the switch’s hardware configuration files to obtain physical port information, then pushes the collected state into APPL_DB. portsyncd transfers port speeds, lanes and MTU, and also injects state into STATE_DB.
    • intfsyncd: Listens to interface-related netlink events and pushes the collected state into APPL_DB. intfsyncd also manages elements such as new or changed IP addresses associated with an interface.
    • neighsyncd: Listens to neighbor-related netlink events triggered by newly discovered neighbors as a result of ARP processing and pushes the collected state into APPL_DB. neighsyncd manages attributes such as the MAC address and neighbor’s address family, and will eventually build the adjacency table required in the data plane for L2 rewrite purposes.
    • orchagent: The most critical component in the swss subsystem. orchagent extracts all the relevant state injected by the various syncd services, processes and messages this information accordingly, and finally pushes it to ASIC_DB. orchagent is unique among these services in that it is both a consumer (like getting the state from APPL_DB) and a producer (like pushing into ASIC_DB).
    • intfMgrd: Reacts to state arriving from APPL_DB, CONFIG_DB and STATE_DB to configure interfaces in the Linux kernel, provided there is no conflicting or inconsistent state within any of the databases being monitored.
    • vlanMgrd: Reacts to state arriving from APPL_DB, CONFIG_DB and STATE_DB to configure VLANs in the Linux kernel, provided there is no conflicting or inconsistent state within any of the databases being monitored.
  • syncd: Synchronizes the switch’s network state with its ASIC. This includes the initialization, configuration and collection of the ASIC’s current status. The primary logical components are:
    • syncd: The process that executes the synchronization logic. At compilation time, syncd links with the ASIC SDK library and injects state gathered from the interfaces into the ASIC. syncd subscribes to ASIC_DB to receive state from swss actors, and also pushes state coming from the hardware back to ASIC_DB.
    • SAI API: The Switch Abstraction Interface (SAI) defines the API to provide a vendor-independent way of controlling forwarding elements, such as a switching ASIC, an NPU or a software switch in a uniform manner.
    • ASIC SDK: An SAI-compatible implementation of the SDK required to drive the ASIC. The SDK hooks into syncd, which is responsible for driving its execution.
  • teamd: The link aggregation (LAG) container, which provides the functionality for configuring bonds on SONiC switches. The teamd service is a Linux-based open source implementation of the LAG protocol. The teamsyncd service manages the interaction between teamd and southbound subsystems.
  • telemetry:
  • what-just-happened: The container for the Mellanox What Just Happened functionality. This container is not installed in SONiC by default; you must add it manually.

To see which Docker containers are running in SONiC, run docker ps:

admin@leaf01:~$ docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS  NAMES
7693970157e4        docker-sonic-telemetry:latest     "/usr/bin/supervisord"   36 hours ago        Up 36 hours  telemetry
1346bf47d5aa        docker-snmp-sv2:latest            "/usr/bin/supervisord"   36 hours ago        Up 36 hours  snmp
b4927925be4a        docker-router-advertiser:latest   "/usr/bin/docker-ini…"   37 hours ago        Up 36 hours  radv
fd4006a177d1        docker-dhcp-relay:latest          "/usr/bin/docker_ini…"   37 hours ago        Up 36 hours  dhcp_relay
d90bf65d463f        docker-lldp-sv2:latest            "/usr/bin/docker-lld…"   37 hours ago        Up 36 hours  lldp
7e1c5217eff3        docker-syncd-vs:latest            "/usr/bin/supervisord"   4 days ago          Up 36 hours  syncd
dcede1ab4ac0        docker-teamd:latest               "/usr/bin/supervisord"   4 days ago          Up 36 hours  teamd
8e3c153cafdb        docker-orchagent:latest           "/usr/bin/docker-ini…"   4 days ago          Up 36 hours  swss
cdeb016623c5        docker-fpm-frr:latest             "/usr/bin/supervisord"   8 days ago          Up 36 hours  bgp
2f6ac855e3e1        docker-platform-monitor:latest    "/usr/bin/docker_ini…"   8 days ago          Up 36 hours  pmon
77917d9efb8a        docker-database:latest            "/usr/local/bin/dock…"   8 days ago          Up 37 hours  database

To see which Docker images are loaded in SONiC, run show version:

admin@switch:~$ show version

SONiC Software Version: SONiC.202012.15-cb79de1b
Distribution: Debian 10.7
Kernel: 4.19.0-9-2-amd64
Build commit: cb79de1b
Build date: Sat Jan 30 11:14:42 UTC 2021
Built by: johnar@jenkins-worker-11

Platform: x86_64-mlnx_msn2700-r0
ASIC: mellanox
ASIC Count: 1
Serial Number: MT0000000
Uptime: 16:16:22 up 15 days, 19:54,  1 user,  load average: 5.80, 5.62, 5.64

Docker images:
REPOSITORY                    TAG                  IMAGE ID            SIZE
docker-syncd-mlnx             202012.15-cb79de1b   10691c29472b        540MB
docker-syncd-mlnx             latest               10691c29472b        540MB
docker-teamd                  202012.15-cb79de1b   15135f94fe07        406MB
docker-teamd                  latest               15135f94fe07        406MB
docker-router-advertiser      202012.15-cb79de1b   11e5c3c840a8        395MB
docker-router-advertiser      latest               11e5c3c840a8        395MB
docker-nat                    202012.15-cb79de1b   f28128bcecb0        408MB
docker-nat                    latest               f28128bcecb0        408MB
docker-platform-monitor       202012.15-cb79de1b   68f6c90b7f0e        687MB
docker-platform-monitor       latest               68f6c90b7f0e        687MB
docker-lldp                   202012.15-cb79de1b   ee4cb85cb4bc        435MB
docker-lldp                   latest               ee4cb85cb4bc        435MB
docker-database               202012.15-cb79de1b   7a79adaa829c        394MB
docker-database               latest               7a79adaa829c        394MB
docker-orchagent              202012.15-cb79de1b   177b5e05d036        423MB
docker-orchagent              latest               177b5e05d036        423MB
docker-snmp                   202012.15-cb79de1b   0e1fd1ff2bc3        435MB
docker-snmp                   latest               0e1fd1ff2bc3        435MB
docker-dhcp-relay             202012.15-cb79de1b   0ae260811eaf        401MB
docker-dhcp-relay             latest               0ae260811eaf        401MB
docker-sonic-telemetry        202012.15-cb79de1b   4fcfc6efdcaa        469MB
docker-sonic-telemetry        latest               4fcfc6efdcaa        469MB
docker-sonic-mgmt-framework   202012.15-cb79de1b   b6f7412f2707        612MB
docker-sonic-mgmt-framework   latest               b6f7412f2707        612MB
docker-fpm-frr                202012.15-cb79de1b   4e6dfaf61388        423MB
docker-fpm-frr                latest               4e6dfaf61388        423MB
docker-sflow                  202012.15-cb79de1b   32b2b32b4bd7        406MB
docker-sflow                  latest               32b2b32b4bd7        406MB

For more details about the architecture, see the Azure SONiC documentation.