> This page is for version 1.4.0 (Latest) (default). > For other versions, use one of these documentation indexes: > - 1.4.0 (Latest) (default): https://docs.nvidia.com/halos/outside-in-safety/1.4.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/halos/outside-in-safety/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/halos/outside-in-safety/_mcp/server. # Integration Guide > Architecture of the Safety Core: its processes, the data path from perception to the PLC, where each process runs, the transports between them, and the process-group rule they share. The Safety Core turns perception output into a safety decision and sends that decision to a PLC. It checks every handoff on the way, watches the health of its own inputs, and puts every decision maker into a latched safe state when it can no longer trust what it receives. This guide is for integrators: people who connect their own perception, write their own decision maker or event source, or run the Safety Core without the reference launcher. To run the reference applications, use the [Deployment Guide](/halos/outside-in-safety/deployment). ![Safety Core overview: perception metadata, camera streams and perception containers feed the MDX client, SAIM and PCM; they report to the SEI daemon, which feeds the NvPSD Gateway, which feeds the SDMs on the CCPLEX or, through fsicom-agent, on the FSI; the SDMs command the PLC and accept a validated manual release from it](/halos/outside-in-safety/_fern-files/nvidia-halos.docs.buildwithfern.com/45b418282832f11cf46fffbda54d1213aec7aa4bbf44ecaec5cb0dce3604e8ee/assets/images/safety-core/safety-core-overview.svg) ## Components The components, their binaries and their roles are listed in [Safety Core Components](/halos/outside-in-safety/integration/components), with a page and an architecture diagram for each. Paths, package names, binaries and log lines use older short names: `pss` for the SEI, `psd` for the gateway and decision side, `psb` for the SBB, and `psf` for the Safety Core as a whole. For example, the gateway logs as `PSD-Gateway` and the SEI's audit records start with `PSS_`. ## Data path One safety event travels through these stages: | # | Stage | Input | What happens | Output | | - | ----------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Perception (SIPP) | Camera streams | Detection, tracking and analytics. | Kafka `mdx-events` (`Behavior` messages: tripwire and ROI alerts) and `mdx-frames` (`FrameMessage` messages: per-frame objects, ROI and proximity). | | 2 | MDX client | Kafka messages | Decodes them, matches each against the rules of the event-mapping file, and resolves the camera name to a pipeline id. | A `SafetyEvent` (`EVENT_0` to `EVENT_23`) reported to the SEI. | | 3 | Trust inputs | Camera streams; perception containers and `mdx-frames` | SAIM judges each camera's input. PCM judges the perception pipeline as a whole. | `SENSOR_INVALID` or `SENSOR_VALID` per camera; `AI_PIPELINE_INVALID` or `AI_PIPELINE_VALID` for the whole pipeline. | | 4 | SEI daemon | Events and verdicts | Checks each report, stamps each event with the standing verdicts (`isHealthy`, `isTrustedSource`), marks old events `STALE`, optionally fuses detections of one object from several cameras, sets the severity, and supervises every source. Sends a status no-op with a presence report at a fixed interval. | A `DecisionRequest` of up to 8 entries, with the SEI's mode (`NORMAL`, `DEGRADED`, `ERROR`). | | 5 | NvPSD Gateway | Decision requests | Checks the SEI's request numbering, turns its own link faults into `SW_FAIL` records, filters each request to each SDM's subscription, and wraps it in a numbered envelope per SDM. | One `DRPG` envelope per SDM. | | 6 | SDM | Envelopes | Validates the envelope, its CRC and its sequence, applies the verdicts and fault causes, decides, and latches the safe state on any fault. | A 64-byte command to the PLC. | | 7 | PLC or command receiver | Commands | Acts on the command and acknowledges it. Sends a release request when an operator confirms that the area is safe. | Acknowledgements; release requests. | How the perception messages map to safety events is described on [Event Mapping Configuration](/halos/outside-in-safety/reference/event-mapping-configuration). What makes an SDM latch, and how it is released, is described in the [safe-state model](/halos/outside-in-safety/integration/safe-state-model). The SEI daemon, the gateway and the MDX client write their records through the SBB to syslog. The SBB is not in the data path; see [Logs](/halos/outside-in-safety/deployment/troubleshooting#logs) for where each log lands. ## Where each process runs ### Platforms | Platform | Where the Safety Core runs | | ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | x86-64, Ubuntu with an NVIDIA GPU | Every process runs on the CPU under Linux. SAIM uses the GPU. | | IGX Thor (aarch64), SDM on the CCPLEX | Every process runs on the CCPLEX under Linux for Tegra. SAIM uses the PVA by default. | | IGX Thor (aarch64), SDM on the FSI | The SDM runs as firmware on the FSI's real-time OS. `fsicom-agent` runs on the CCPLEX in its place; everything else is as above. ATL and PXC only, 2D only. | ### Container or host The Safety Core can be started in three ways. The container's options are on [Container options](/halos/outside-in-safety/reference/launcher#container-options), and the launcher's on [`launch_hoisa.sh`](/halos/outside-in-safety/reference/launcher#launch_hoisa-options). | Process | Plain `docker run` of the image | `launch_hoisa.sh --runtime docker` (default) | `launch_hoisa.sh --runtime host` | | -------------------------------------------------- | ------------------------------------------------- | --------------------------------------------- | -------------------------------- | | `nvpss_daemon`, `nvpsd_gateway`, `mdx_client` | Container | Container `nv-psf` | Host, as root | | `safety_monitor` (unless `--saim-mode skip`) | Container | Container | Host | | `nv_perception_monitor` (unless `--pcm-mode skip`) | Container | Container | Host | | CCPLEX SDMs | Container; on the host only with `--sdm-external` | Host, as root with the container user's group | Host, as root | | `fsicom-agent` (SDM on the FSI) | Host, started by you, with `--sdm-on-fsi` | Host, as root with the container user's group | Host, as root | | Command receiver or PLC | Not started | Not started by the launcher | Not started by the launcher | Under `--runtime docker` the launcher starts the SDMs, or `fsicom-agent`, on the host as root, with the primary group of the container user `nvidia`, so the NvPSD Gateway in the container accepts them. See [Shared process group](#shared-process-group). With a plain `docker run`, every Safety Core process except `fsicom-agent` runs in the container by default, so no Safety Core link crosses the container boundary. The container needs: * **Docker socket.** Mount the host's `/var/run/docker.sock`, so PCM can check the perception containers from inside the container. * **Network.** The container uses the host network (`--network=host`), so the SDM-to-PLC UDP link and the Kafka connection behave as on the host. * **Files.** Mount each configuration file over the path the container reads it from. The monitor modes must match `required_safety_monitor` and `required_perception_monitor` in the `nvpss.conf` the container reads. A Safety Core process on the host that connects to the container, such as `fsicom-agent` or an SDM next to a container started with `--sdm-external`, needs two things more: * **Sockets.** All Safety Core sockets live in `/run/nvpsf`. Bind-mount that host directory into the container (`-v /run/nvpsf:/run/nvpsf`), so the process on the host and the processes in the container reach the same sockets. * **One group.** The process must run with the container user's primary group. See [Shared process group](#shared-process-group). `fsicom-agent` always runs on the host: it talks to the FSI through NvFsiCom, which needs the FSI channel in the host's device tree and the NvFsiCom daemon running on the host. ## 2D and 3D deployments `deploymentMode` in `nvpss.conf` is mandatory and states the topology. It is not inferred from which files are present. | Aspect | `deploymentMode = 2D` | `deploymentMode = 3D` | | -------------------------------------------------------- | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Event identity | Each camera reports its own events. | Cameras are calibrated into one BEV group, and events are reported under the group. | | `nvpss.conf` | Fusion may be on or off (`enableFusion`). | Requires `enableFusion = false` and `sensorGroupConfig`, the sensor group file. The daemon refuses to start otherwise. | | `mdx_client` identity file | `--sensor-config sensor_config.conf` (the cameras) | `--sensor-pipelines-config sensor_pipelines_config.conf` (the group only) | | SDM reviewed sensor set (`sensor_pipelines_config.conf`) | The cameras | The group only | | SAIM | Watches every camera in `sensor_config.conf`. | The same. | | Sensor verdicts the SDMs see | Per camera. | Per group. The group is untrusted only when every member camera is invalid and none is still initialising; one healthy member keeps it trusted. SAIM cannot report the group directly. | | SDM on the FSI | Supported | Not supported | The file formats are on [Sensor files](/halos/outside-in-safety/reference/sensor-files). The keys are on [SEI Configuration](/halos/outside-in-safety/reference/sei-configuration). ## Transports Every link inside the Safety Core is an `AF_UNIX` `SOCK_SEQPACKET` socket in `/run/nvpsf`. None of them can be switched to UDP or TCP. Only the links to the outside world use the network. | # | Link | Transport | Address | Server | Client | Credential check | | -- | ----------------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------ | ---------------------------------- | | 1 | Perception to `mdx_client` | Kafka | Topics `mdx-events`, `mdx-frames`. Broker from `--broker`, else `MDX_MSGBUS_BROKERS`, else `localhost:9092`. | Kafka broker | `mdx_client` | — | | 2 | Perception to PCM | Kafka; Docker Engine API | Topic `mdx-frames`, with its own consumer; `/var/run/docker.sock`, mounted into the container under the Docker runtime | Kafka broker; Docker | `nv_perception_monitor` | — | | 3 | Cameras to SAIM | RTSP | The `rtspUrl` of each camera in `sensor_config.conf` | Camera or stream server | `safety_monitor` | — | | 4 | Sources to the SEI (SEI RPC) | `AF_UNIX` `SOCK_SEQPACKET` | `/run/nvpsf/nvpssd`, mode `0660` | `nvpss_daemon` | `mdx_client`, `safety_monitor`, `nv_perception_monitor`, `nvpsd_gateway`, your own sources | Server checks the client | | 5 | SEI to gateway | `AF_UNIX` `SOCK_SEQPACKET` | `/run/nvpsf/nvpssd_to_psd`, mode `0660` | `nvpss_daemon` | `nvpsd_gateway` | Both ends | | 6 | Gateway to SDMs | `AF_UNIX` `SOCK_SEQPACKET` | `/run/nvpsf/nvpsd_gateway`, mode `0660` | `nvpsd_gateway` | Each SDM, or `fsicom-agent` | Both ends | | 7 | SDM to PLC | UDP, 64-byte packets | The SDM sends from an ephemeral port to `--cmd_rx_ip`:`--cmd_rx_port`; the receiver listens on `-p` (default `12345`). | PLC or command receiver | SDM | Source address, identifier and CRC | | 8 | `fsicom-agent` to the FSI SDM | NvFsiCom, channel 6 | Device-tree node compatible with `nvidia,tegra-fsicom-bufdemo-10k` | — | — | — | | 9 | `fsicom-agent` to PLC | UDP, 64-byte packets | `--ip`:`--port` | PLC or command receiver | `fsicom-agent` | Source address, identifier and CRC | | 10 | Display relays (optional) | UDP | ATL command receiver `--vst_ip`:`--vst_port`; `proximity_sdm` `--halo_ip`:`--halo_port` | VST | Receiver or SDM | — | Notes on the `AF_UNIX` links: * **Directory.** `/run/nvpsf` must exist before any server starts; no binary creates it. The runtime package creates it with mode `1777` at install and on every boot, and the launchers check it; see [Install](/halos/outside-in-safety/integration/install#what-the-runtime-package-sets-up). * **Socket mode.** Each server sets its socket to `0660` after binding. The parent directory protects nothing, so access control is the credential check. * **Second instance.** A server that finds a live server already listening on its path refuses to start and prints `nvpsf_transport: bind() refused -- a LIVE server is already listening on this path`. A stale socket left by a crash is removed and reused. * **Buffers.** The ends of link 6 request socket buffers for 516 decision packets. The kernel limits must allow them; see [Kernel socket buffers](/halos/outside-in-safety/integration/install#kernel-socket-buffers). ## Shared process group **Every Safety Core process must run with the same primary group.** On every `AF_UNIX` link, the server reads each client's credentials with `SO_PEERCRED` and compares the client's group with its own `getgid()`. On links 5 and 6 the client also checks the server the same way. The peer's primary group must match exactly; having the group as a supplementary group is not enough. The rule covers `nvpss_daemon`, `nvpsd_gateway`, `mdx_client`, `safety_monitor`, `nv_perception_monitor`, every SDM, `fsicom-agent`, and every process you add. There is no dedicated Safety Core group. Each process requires its own primary group of its peers, so the rule holds when every process runs with the same primary group, whichever it is. | Deployment | Primary group of the Safety Core processes | Meets the rule | | ---------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | | Plain `docker run` of the image, every process in the container | The group of the container user `nvidia`. The entrypoint runs the container launcher as `nvidia`, and the launcher keeps that group when it starts SAIM and PCM with extra device or Docker socket groups | Yes | | `launch_hoisa.sh --runtime host` | Root's group, `0`: the launcher starts every process that connects to another one with `sudo -n` | Yes | | Every process started by hand in one root shell, as in [Running without Docker](/halos/outside-in-safety/integration/install#running-without-docker) | `0` | Yes | | `launch_hoisa.sh --runtime docker`, control-plane run | The container user's group. The launcher reads it from the running container and starts the SDMs or `fsicom-agent` on the host as root, with that group as their primary group | Yes | The container user's group ID is assigned when the image is built. Read it from the image, or from a running container: ```bash docker run --rm --entrypoint id nvcr.io/nvidia/halos-outside-in/outside-in-safety:nv-psf-halos-1.4-6b92731 -g nvidia docker exec id -g nvidia ``` ### For a process you start yourself Start the process with the primary group of the processes it connects to. Next to processes that run as root on the host, start it as root. Next to the container, start it as root with the container user's group ID as its primary group, keeping root's user ID and supplementary groups: ```bash sudo setpriv --regid --keep-groups -- ``` `` is the number printed by the `id` command above. The container must also mount the host's `/run/nvpsf`, so that the process on the host sees the container's sockets. The [SDM on FSI](/halos/outside-in-safety/deployment/custom-deployment#sdm-on-fsi) procedure starts `fsicom-agent` this way, and `launch_hoisa.sh --runtime docker` starts its SDMs and `fsicom-agent` the same way. Docker must not use user-namespace remapping or rootless mode: the host then sees the container's group IDs shifted, and no host process can share them. ### Host accounts with the container's group When a process on the host connects to the container, `/run/nvpsf` is shared with the host, and the container user's group ID is the only credential the sockets check. A host account that can run with that group ID can then connect to the Safety Core sockets, or stand in for a Safety Core server whose socket is not bound, for example before the NvPSD Gateway starts or after it exits. That is an account whose primary group has that ID, or a member of a host group with that ID, who can switch to it with `newgrp` or `sg`. An SDM or `fsicom-agent` on the host would then accept such a process as its gateway. A host account with the container user's user ID owns the socket files on the host and can delete them, so that processes connecting or reconnecting afterwards cannot reach the Safety Core, and the decision makers then hold their safe state. `launch_hoisa.sh --runtime docker` prints a warning at every control-plane start when either ID is used on the host; see [Host processes and the container's group](/halos/outside-in-safety/reference/launcher#host-processes-and-the-container-group). To check a host yourself: ```bash getent group getent passwd | awk -F: '$4 == ' getent passwd ``` `` is printed by `docker exec id -u nvidia`. Each command prints nothing when no host group or account uses the ID. Keep such accounts off the host, or run every Safety Core process in the container without the `/run/nvpsf` mount, so that no socket is shared with the host. ### What a mismatch looks like A mismatched process is refused. This is the safe direction: it cannot register, so an SDM holds its safe output and latches on lost freshness and gateway heartbeat, and a required monitor that never registers is reported lost to every SDM. The refusing end prints both group ids to standard error: ```text nvpsf_transport: peer GID does not match the required PSF group . Every PSF process must share one GID; check the container UID/GID mapping across the boundary this link crosses. ``` followed by one of: | Printed by | Line | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | | The server (`nvpss_daemon`, `nvpsd_gateway`) | `nvpsf_transport: rejected connection, peer credential is not in the allowed group` | | A transport client (an SDM, `fsicom-agent`, or the gateway on link 5) | `nvpsf_transport: server credential rejected; refusing to treat the connection as trusted` | The gateway logs the group it requires at start-up, in a syslog record whose data reads `PSD-Gateway: peer credential allowlistrequiredPrimaryGid=`; search for `requiredPrimaryGid=`. To compare it with the running processes: ```bash ps -eo pid,egid,args | grep -E 'nvpss_daemon|nvpsd_gateway|mdx_client|safety_monitor|nv_perception_monitor|_sdm|fsicom-agent' ``` ## Integration Guide pages | Page | Contents | | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | [Install](/halos/outside-in-safety/integration/install) | Debian packages, install paths, runtime libraries, kernel settings, running every process without Docker, building the reference applications. | | [Writing a decision maker](/halos/outside-in-safety/integration/writing-a-decision-maker) | The contract a custom SDM must meet. | | [Safe-state model](/halos/outside-in-safety/integration/safe-state-model) | What latches an SDM, what clears each cause, and how the latch is released. | | [Custom event sources](/halos/outside-in-safety/integration/custom-event-sources) | Adding your own event source or monitor through the SEI client library. | | [SAIM distortion injection](/halos/outside-in-safety/integration/saim-distortion-injection) | Injecting camera faults into a test deployment to check SAIM and the decision makers. | The Reference section holds the full option and key lists: [Launcher](/halos/outside-in-safety/reference/launcher), [SEI Configuration](/halos/outside-in-safety/reference/sei-configuration), [Sensor files](/halos/outside-in-safety/reference/sensor-files), [Event Mapping Configuration](/halos/outside-in-safety/reference/event-mapping-configuration) and [Causes and Records](/halos/outside-in-safety/reference/causes-and-records).