Safety Event Integrator (SEI)

The Safety Event Integrator (SEI) collects the safety events that event sources report, checks them, fuses or passes them through, and sends the result to the decision makers as DecisionRequests through the NvPSD Gateway. Its daemon is nvpss_daemon; its configuration file is nvpss.conf.

SEI also tracks the trust verdicts from the Safety AI Monitor (SAIM) and the Perception Container Monitor (PCM), checks that every required source is present, and reports its own health in every request. Under the Docker runtime it runs inside the Safety Core container; under --runtime host it runs on the host.

Startup

The daemon reads /opt/nvidia/psf/bin/nvpss.conf, or the file given with --conf <path>. Every key is described in SEI configuration. The daemon refuses to start, and exits 1, when any of the conditions below holds. It does not report unknown keys: a misspelt optional key is ignored and its default applies.

Other numeric keys with a bad value fall back to a default or are clamped, with a warning.

The daemon serves two sockets in /run/nvpsf. It does not create the directory; a missing directory makes it fail with nvpsf_transport: bind(<path>) failed.

Both are AF_UNIX SOCK_SEQPACKET sockets with mode 0660. A peer is accepted only when its primary GID equals the daemon’s; see Shared process group.

Client admission

• Slots. The daemon has 8 client slots in total, shared by every client, including the gateway, SAIM and PCM. A connection takes a slot before it registers. When no slot is free the connection is dropped (admission dropped: no free client slot).
• Registration. A connection that has not registered 6.5 s after connecting is closed at the next 5 s check. A heartbeat or a report before registration closes the connection.
• Refusals. An unknown client type, an instance number on a non-MDX client, or an instance outside 1–8 is refused. A second client of a single-instance type, or a second MDX client with the same number, is refused with duplicate singleton client type rejected. If the existing holder has exhausted its heartbeats, it is evicted instead and the newcomer is accepted. A refused client is disconnected.
• Heartbeats. Clients send a heartbeat every 5 s. A client is stale 6.5 s after its last heartbeat; each further 5 s check counts one more miss. With M = max_hb_failures (range 1–255; 10 when the key is absent; the shipped file sets 50) and W = M / 2 rounded down:

The custom-source API is described in Custom event sources.

Report validation

Each SafetyEvent a client reports goes through these checks in order. The first five answer the client with a rejection code.

Accepted events are then checked against the sensor configuration. These checks run after the client was told the report was accepted, so their rejections are visible only in the log:

A client cannot set the severity; the daemon assigns it.

Rejection streak. Every rejected report adds to the client’s run of consecutive rejections; an accepted report resets it. The checks that run after acceptance do not count. When the run reaches client_reject_error_threshold (default 50, range 3–10000):

• If the source is required, by its own identity or because its last accepted report was for a pipeline in the sensor configuration (in 3D, the sensor group), SEI logs PSS_CLIENT_INPUT_UNUSABLE and sends SW_FAIL cause 13 once per episode. Every SDM latches. The SDMs clear the condition only when a later presence report confirms that source usable again.
• Otherwise SEI logs the warning PSS_CLIENT_REJECT_STREAK once and takes no action.

Trust and sensor health

SEI keeps three kinds of trust and stamps them on every event it forwards, as isHealthy and isTrustedSource.

• An event that is unhealthy or untrusted is not fused and not checked against the sensor configuration. It is forwarded as CRITICAL with status UNKNOWN (or STALE when old), so the SDMs can see it but do not act on it.
• A stamp can only get worse between queueing and sending: the daemon uses the trust at queue time and the trust at send time, whichever is lower.
• With every status no-op, SEI re-sends each standing INVALID verdict: the AI-pipeline verdict, each unhealthy camera in 2D, or the untrusted group in 3D. Valid verdicts are not re-sent.

Source presence

Three mandatory keys in nvpss.conf name the sources that must be present:

A source is present while it is registered, has sent a heartbeat since registering, and has not faulted. SEI reports a missing source to every SDM as a SW_FAIL:

• The never-registered budget is 6.5 s plus 5 s for each of max_hb_failures − 1 misses, counted from daemon start: about 4.2 minutes with the shipped value of 50.
• A loss is reported whether or not the source is required. It clears only when the source registers and sends heartbeats again.
• The log record names the source: PSS_SAFETY_MONITOR_LOST, PSS_PERCEPTION_MONITOR_LOST or PSS_MDX_CLIENT_LOST.
• Every status no-op also carries a presence report (cause 16) that lists the present and the expected sources. An SDM latches when the report shows an expected source as absent. The format is in Causes and records.

Fusion or pass-through

enableFusion in nvpss.conf selects how accepted events are processed.

SW_FAIL, SENSOR_INVALID, SENSOR_VALID, AI_PIPELINE_INVALID and AI_PIPELINE_VALID always bypass fusion. Each forwarded event carries a status:

The fusion keys are described in SEI configuration.

Fail-safe staging

Forwarded events wait in one of two lanes until they are sent.

• Each lane holds psd_critical_queue_depth and psd_operational_queue_depth events (default 32, range 8–512).
• A request carries at most 8 events. The critical lane is served first.
• Operational lane full. The new event is dropped and counted (PSD_STAGING_QUEUE_OVERFLOW, logged at most once a second). A status no-op that finds the lane full is retried after 100 ms.
• Critical lane full. An ordinary event is dropped. A SW_FAIL or trust report evicts the newest evictable entry, an event from an untrusted source or a stale one, outside the first 8 (PSD_CRITICAL_LANE_EVICTION). If nothing can be evicted, the report is refused but kept (up to 32), and every request is stamped ERROR (PSS_CRITICAL_LANE_OVERFLOW) until the lane is at most half full, an ERROR-stamped request has been acknowledged and every refused report has been delivered (PSS_CRITICAL_LANE_RECOVERED).

Severity is OPERATIONAL (0) or CRITICAL (1). One request carries one severity: if any entry is CRITICAL, or SEI is in ERROR mode, every entry is CRITICAL.

Delivery to the gateway

One sender thread sends one request at a time and waits for the gateway’s DecisionResponse.

Status no-op

So that the SDMs hear from SEI even when no events occur, SEI queues a PSS_STATUS_NOOP request every status_noop_interval_ms (default 5000, range 100–600000), whatever other traffic there is. It carries:

• one event of type PSS_STATUS_NOOP, severity OPERATIONAL, status PASSTHROUGH, with sensorIdentifier PSS_DAEMON;
• the current SEI mode;
• the presence report (cause 16).

Queueing a no-op also re-sends the standing INVALID trust verdicts. The SDMs use the no-op to refresh their decision-freshness timer; see Writing a decision maker.

Modes

Every request carries the SEI mode in pssStatus.mode.

When a request reports ERROR, the gateway sends every SDM a SW_FAIL with cause 12 and every SDM latches.

Audit records

The daemon writes its records to syslog through the Safety Black Box with the tag NVPSB_PSS_DAEMON; see Safety Black Box (SBB). The record formats are in Causes and records.

Diagnostics about the gateway link, such as Timeout receiving matching DecisionResponse from PSD client, and transport rejections go to standard error, not to syslog.