Skip to navigation

nvpsf_transport.h

View as Markdown

The message-oriented AF_UNIX SOCK_SEQPACKET transport that every Safety Core link uses: endpoint configuration, socket paths under /run/nvpsf, the peer group check and the socket-buffer sizing.

Part of the Transport. Generated from the header published in safety-core/, release 1.3-26.07.5. The development packages install it in /opt/nvidia/psf/include.

#include "nvpsf_transport.h"

Message-oriented inter-process transport for PSF links.

One API over two transports: AF_UNIX SOCK_SEQPACKET and UDP. The API is message-oriented rather than stream-oriented, because both preserve message boundaries. Two consequences the caller can rely on:

  • Recv returns exactly one whole message or an error. A partial message is not a reachable outcome, so no caller needs a recvAll loop.
  • Send transmits one whole message or reports WOULD_BLOCK. A partial send is not a reachable outcome, so no caller needs a sendAll loop.

A blanket “never call this API under your own lock” rule would be unsafe, not merely conservative: obeying it literally means dropping your lock before Send, which opens a window for a concurrent Close to free the handle mid-send. Consult the table instead.

Par: Send is always non-blocking at the syscall level. Waiting happens only in NvPSFTransportPoll, which takes an explicit timeout. There is no call in this API through which a caller can block indefinitely.

Par: The transport’s internal descriptor mutex is a leaf lock: this library never invokes a caller callback while holding it, so it can never acquire a caller’s lock. The obligation is therefore per entry point, not blanket.

Call Under a caller lock? NvPSFTransportSend Permitted. Non-blocking at the syscall level (MSG_DONTWAIT, and every socket is SOCK_NONBLOCK), acquires only the transport’s own leaf mutex, and makes no callback. Holding a caller lock across it is in fact the simplest way to keep the handle alive against a concurrent Close. NvPSFTransportRecv Permitted, for the same reasons. recvmsg uses MSG_DONTWAIT. NvPSFTransportPoll Not permitted while holding a lock any send path also takes. Poll is the ONLY call in this API that blocks, for up to its timeout, and it would stall every concurrent Send for that whole period. NvPSFTransportDropPeer Not permitted. It tears down peer state, and reclaim logic naturally sits inside a locked region which is what makes this the easy one to get wrong. Snapshot the peer set under your lock, release it, then call DropPeer. NvPSFTransportClose Not permitted. It frees the handle. No other thread may be inside any call on it, which is a stronger requirement than a lock can express.

The distinction that matters: Send and Recv are bounded and side-effect-free with respect to the caller’s state; Poll blocks; DropPeer and Close destroy state other callers may be using. Treat the last three as the rule and the first two as safe.

Par: Descriptors stay private to the transport object. No function returns or accepts a raw descriptor, so a caller cannot outlive-use one.

Par: Linux only. Depends on SO_PEERCRED, struct ucred, AF_UNIX SOCK_SEQPACKET, MSG_NOSIGNAL, and SOCK_CLOEXEC.

Functions

NvPSFTransportKindName

static inline const char * NvPSFTransportKindName(NvPSFTransportKind kind);

Operator-facing name of kind, for logs and diagnostics.

Returns: A static string; “INVALID” when kind is out of range. Never NULL, so callers may log the result without a null check.

NvPSFTransportLinkSupportsKind

static inline int NvPSFTransportLinkSupportsKind(NvPSFTransportLink link, NvPSFTransportKind kind);

Whether link can carry kind.

The single definition of the availability matrix. A selection outside it is a fatal startup error everywhere, never a silent downgrade, because a downgrade would move safety traffic onto a different path than the operator configured with no indication.

Deliberately a header-only inline so configuration parsers and test tooling can validate a selection without linking libnvpsf_transport. Duplicating the matrix in those callers instead would let the copies drift apart, which is the failure this placement prevents.

Returns: 1 when supported, 0 when not or when either argument is out of range.

NvPSFTransportLinkName

static inline const char * NvPSFTransportLinkName(NvPSFTransportLink link);

Operator-facing name of link, for diagnostics.

Returns: A static string; “INVALID” when out of range. Never NULL.

NvPSFGatewaySdmMinQueueDepth

static inline uint32_t NvPSFGatewaySdmMinQueueDepth(void);

Provisioned depth for the gateway -> SDM link, honouring a deployment override.

Returns NVPSF_GATEWAY_SDM_MIN_QUEUE_DEPTH unless NVPSF_MIN_QUEUE_DEPTH is set in the environment, in which case that value is used and a warning naming the risk is printed.

WHY AN OVERRIDE EXISTS AT ALL, since a lower value re-opens exactly the spurious-lockout path this floor closes:

The floor is a statement about what the HOST has been provisioned for, not a safety dial. A host whose net.core.rmem_max cannot reach 1096500 every stock distribution default is far below it cannot start the gateway or either SDM at all once this lands, and that includes every build and test machine. Without an explicit, loud override the only way to run any integration test would be root access to change a kernel parameter, which turns an unprovisioned CI box into a hard stop rather than a warning.

It is deliberately an environment variable and not a config key or CLI flag: it must be conspicuous, it must not look like normal operator tuning, and it must not be reachable from nvpss.conf. Startup-only and read once, so it is therefore immutable once Open returns.

Production must NOT set it. Release documentation states the sysctl instead.

NvPSFTransportOpen

NvPSFTransportErr NvPSFTransportOpen(const NvPSFTransportCfg *cfg, NvPSFTransport **out);

Create an endpoint.

Fails rather than degrades. An unsupported kind for cfg->link, a path too long for sun_path, or a buffer the kernel will not grant all return an error with a diagnostic on stderr; the caller is expected to treat any failure here as fatal at startup.

ParameterDescription
cfgConfiguration. Not retained.
outReceives the handle on success; set to NULL on failure.

NvPSFTransportClose

NvPSFTransportErr NvPSFTransportClose(NvPSFTransport *t);

Destroy an endpoint, closing every descriptor and unlinking a bound AF_UNIX path.

Call only after every thread that uses t has been joined. NULL is a no-op.

NvPSFTransportPoll

NvPSFTransportErr NvPSFTransportPoll(NvPSFTransport *t, uint32_t timeoutMs, NvPSFTransportEvent *events, size_t maxEvents, size_t *outCount);

Wait up to timeoutMs for peer or message activity.

Accepts pending connections, validating credentials, and reports the results as events. Returns NVPSF_TRANSPORT_TIMEOUT with *outCount == 0 when nothing happened.

Par: When the shared datagram socket is in an error state most often ICMP port-unreachable from a peer that is not listening Poll consumes the pending error, names it on stderr and returns NVPSF_TRANSPORT_FAIL. It does not report TIMEOUT: poll() does not block while an error is pending, so a caller told “nothing happened” would spin.

Par: A peer that sends and then closes delivers both in one wakeup. Poll reports the readable data and holds the slot; PEER_GONE follows on a later Poll, or as Recv’s return code, once the queue is empty. Draining the peer is therefore what surfaces the disconnect: a caller that acts on events but never calls Recv never observes PEER_GONE.

ParameterDescription
eventsCaller-provided array.
maxEventsCapacity of events.
outCountEvents written.

NvPSFTransportRecv

NvPSFTransportErr NvPSFTransportRecv(NvPSFTransport *t, NvPSFPeerId *fromPeer, void *buf, size_t bufLen, size_t *outLen);

Receive exactly one whole message.

With several peers ready, selection is round-robin from the slot after the one last serviced, and one call yields at most one message. Lowest-slot-first would let a chatty peer inside the allowed group starve the others; on the gateway link that would mean registrations and heartbeat acknowledgements from healthy SDMs go unserviced and those SDMs get reclaimed as dead.

ParameterDescription
fromPeerSender. May be NULL if the caller does not need it.
outLenBytes written to buf.
ParameterDescription
NVPSF_TRANSPORT_TRUNCATEDbufLen was too small; the message has been discarded by the kernel and cannot be re-read.
NVPSF_TRANSPORT_WOULD_BLOCKNo message was ready.

NvPSFTransportSend

NvPSFTransportErr NvPSFTransportSend(NvPSFTransport *t, NvPSFPeerId toPeer, const void *buf, size_t len);

Send one whole message. Never blocks.

ParameterDescription
NVPSF_TRANSPORT_WOULD_BLOCKBuffer full; nothing was sent. On the gateway link the caller must treat this as fault evidence, not a transient, because the buffer is provisioned for the worst legitimate burst.

NvPSFTransportSolePeer

NvPSFTransportErr NvPSFTransportSolePeer(NvPSFTransport *t, NvPSFPeerId *outPeer);

Identity of the single peer held by a non-server endpoint.

Required for a client to send first, which every PSF client does: an SDM registers with the gateway, the PSS daemon initiates to PSD, and an RPC client issues a request. A peer id embeds a generation counter that is assigned internally, so it cannot be constructed by the caller, and without this accessor a client could only learn its own peer after receiving a message making the send-first direction unreachable.

Valid for a client of any kind. A server has many peers and learns each from NVPSF_EVENT_PEER_CONNECTED or from Recv, so the question does not apply and the call is refused rather than answered arbitrarily.

ParameterDescription
outPeerReceives the peer id; NVPSF_PEER_INVALID on failure.
ParameterDescription
NVPSF_TRANSPORT_INVALID_ARGt is a socket server.
NVPSF_TRANSPORT_PEER_GONEThe connection has since dropped.

NvPSFTransportPeerCred

NvPSFTransportErr NvPSFTransportPeerCred(NvPSFTransport *t, NvPSFPeerId peer, NvPSFPeerCred *outCred);

Read the credential captured when peer was admitted.

Authorization must use this, never a field from a client-controlled message payload.

NvPSFTransportDropPeer

NvPSFTransportErr NvPSFTransportDropPeer(NvPSFTransport *t, NvPSFPeerId peer);

Disconnect peer and free its slot.

Must be called outside any caller lock. A stale identity whose generation no longer matches returns NVPSF_TRANSPORT_PEER_GONE without touching the slot’s current occupant.

NvPSFTransportPeerDropCount

NvPSFTransportErr NvPSFTransportPeerDropCount(NvPSFTransport *t, NvPSFPeerId peer, uint64_t *outDrops);

Messages dropped for peer because the buffer was full.

Exists so a caller can log backpressure as fault evidence with a count attached, rather than reporting each WOULD_BLOCK in isolation.

NvPSFTransportErrName

const char * NvPSFTransportErrName(NvPSFTransportErr err);

Human-readable name for err. Never NULL.

Types

NvPSFPeerCred_t

struct NvPSFPeerCred_t

Validated peer credential, captured once at accept/connect time.

FieldDeclarationDescription
pidpid_t pid
uiduid_t uid
gidgid_t gid

NvPSFTransportEvent_t

struct NvPSFTransportEvent_t
FieldDeclarationDescription
kindNvPSFTransportEventKind kind
peerNvPSFPeerId peer

NvPSFTransportCfg_t

struct NvPSFTransportCfg_t

Endpoint configuration. Startup-only; nothing here is mutable after NvPSFTransportOpen returns.

Only the fields belonging to kind are read. String fields are copied during Open, so the caller may free them afterwards.

FieldDeclarationDescription
kindNvPSFTransportKind kind
roleNvPSFTransportRole role
linkNvPSFTransportLink link
maxPeersuint32_t maxPeersPeers this endpoint may hold. Forced to 1 for every client role.
maxMessageBytessize_t maxMessageBytesLargest frame on this link. Bounds every internal buffer.
minQueueDepthuint32_t minQueueDepthWhole messages the transport buffer must hold. 0 disables the check. Non-zero only on the gateway -> SDM link, because that is the only link where a failed send burns a sequence number and therefore latches the SDM into fault-safe lockout. Elsewhere a send failure surfaces as a retried or failed call, so a depth floor buys nothing.
recvTimeoutMsuint32_t recvTimeoutMs
sendTimeoutMsuint32_t sendTimeoutMs
unixPathconst char * unixPath
unixModemode_t unixModeApplied with chmod() after bind(); the 1777 parent grants no protection.
allowedGidgid_t allowedGidSO_PEERCRED allowlist. Peers outside this group are refused.
inetAddrconst char * inetAddr
inetPortuint16_t inetPort
inetAllowListconst char * inetAllowListSource-address allowlist for UDP registration. Required for a UDP server; ignored for every other kind and role. Comma-separated IPv4 addresses or CIDR blocks, e.g. "127.0.0.1/32" or "10.0.4.0/24,10.0.5.7". A bare address means /32. At most NVPSF_ALLOWLIST_MAX entries. Making it optional would have closed the gap only for operators who happened to set it, which is the same as not closing it. An endpoint that cannot say who is allowed to register cannot be started; the message names the flag. Same fail-closed shape as the minQueueDepth shortfall and the retired-config-key rejection. Par: UDP carries no credential, so SO_PEERCRED the check that authenticates registration on the UNIX_SOCKET path has nothing to inspect. A datagram’s source address is the only identity the kind offers. Without a bound on which sources may register, any peer that can route to this port can occupy a slot and be sent DecisionRequests. This bounds who may register. It is not authentication: a source address on a routed network can be spoofed, and an allowlisted host that is itself compromised is still allowlisted. UNIX_SOCKET remains the authenticated option and the recommended deployment.

NvPSFTransportKind_t

enum NvPSFTransportKind_t

Transport used to carry PSF messages over one link.

Values are explicit and stable. Not every kind is valid on every link; each link declares the kinds it supports and rejects the rest at startup.

These enumerators must not be placed on the wire. Convert to and from a fixed-width integer at any serialization boundary and range-check against NVPSF_TRANSPORT_KIND_COUNT before casting back.

ValueInitializerDescription
NVPSF_TRANSPORT_UNIX_SOCKET0AF_UNIX SOCK_SEQPACKET. The default and the only kind the two upstream links accept.
NVPSF_TRANSPORT_UDP_SOCKET1AF_INET SOCK_DGRAM. Gateway -> SDM link only.
NVPSF_TRANSPORT_KIND_COUNT2One past the highest kind. Sizes dispatch tables; not a selectable value.
enum NvPSFTransportLink_t

Which PSF link an endpoint serves.

Lives here beside the kind enum rather than in nvpsf_transport.h because the two are one vocabulary: a transport selection is only meaningful paired with the link it applies to, and configuration parsers need both before any endpoint exists.

ValueInitializerDescription
NVPSF_LINK_PSS_RPC0mdx-client and other libnvpss clients to the PSS daemon.
NVPSF_LINK_PSS_TO_PSD1PSS daemon to the PSD gateway.
NVPSF_LINK_GATEWAY_TO_SDM2PSD gateway to SDM.
NVPSF_LINK_COUNT3Number of links. Sizes dispatch tables; not a selectable value.

NvPSFTransportRole_t

enum NvPSFTransportRole_t
ValueInitializerDescription
NVPSF_ROLE_SERVER0
NVPSF_ROLE_CLIENT1

NvPSFTransportErr_t

enum NvPSFTransportErr_t
ValueInitializerDescription
NVPSF_TRANSPORT_OK0
NVPSF_TRANSPORT_WOULD_BLOCKBackpressure. The caller applies its own drop policy; nothing was sent.
NVPSF_TRANSPORT_PEER_GONEPeer disconnected or was never connected. Fault evidence, per LT-7.
NVPSF_TRANSPORT_TRUNCATEDMessage exceeded the buffer or MSG_TRUNC was set. A protocol error.
NVPSF_TRANSPORT_TIMEOUT
NVPSF_TRANSPORT_DENIEDPeer credential rejected. No slot was allocated.
NVPSF_TRANSPORT_INVALID_ARG
NVPSF_TRANSPORT_FAIL

NvPSFTransportEventKind_t

enum NvPSFTransportEventKind_t
ValueInitializerDescription
NVPSF_EVENT_PEER_CONNECTED0
NVPSF_EVENT_PEER_GONE1
NVPSF_EVENT_MESSAGE_READY2

Type aliases

NameDefinitionDescription
NvPSFTransportKindtypedef enum NvPSFTransportKind_t NvPSFTransportKindTransport used to carry PSF messages over one link. Values are explicit and stable. Not every kind is valid on every link; each link declares the kinds it supports and rejects the rest at startup. These enumerators must not be placed on the wire. Convert to and from a fixed-width integer at any serialization boundary and range-check against NVPSF_TRANSPORT_KIND_COUNT before casting back.
NvPSFTransportLinktypedef enum NvPSFTransportLink_t NvPSFTransportLinkWhich PSF link an endpoint serves. Lives here beside the kind enum rather than in nvpsf_transport.h because the two are one vocabulary: a transport selection is only meaningful paired with the link it applies to, and configuration parsers need both before any endpoint exists.
NvPSFTransporttypedef struct NvPSFTransport NvPSFTransportOpaque endpoint handle.
NvPSFPeerIdtypedef uint32_t NvPSFPeerIdOpaque peer identity: slot index in the low 16 bits, generation counter in the high 16 bits. The generation makes a stale identity detectable rather than dangerous: a deferred disconnect that names a slot which has since been recycled fails the generation check and is ignored, instead of acting on the new occupant. Callers must treat the value as opaque and compare only for equality.
NvPSFTransportRoletypedef enum NvPSFTransportRole_t NvPSFTransportRole
NvPSFTransportErrtypedef enum NvPSFTransportErr_t NvPSFTransportErr
NvPSFPeerCredtypedef struct NvPSFPeerCred_t NvPSFPeerCredValidated peer credential, captured once at accept/connect time.
NvPSFTransportEventKindtypedef enum NvPSFTransportEventKind_t NvPSFTransportEventKind
NvPSFTransportEventtypedef struct NvPSFTransportEvent_t NvPSFTransportEvent
NvPSFTransportCfgtypedef struct NvPSFTransportCfg_t NvPSFTransportCfgEndpoint configuration. Startup-only; nothing here is mutable after NvPSFTransportOpen returns. Only the fields belonging to kind are read. String fields are copied during Open, so the caller may free them afterwards.

Macros

MacroValueDescription
NVPSF_RUNTIME_DIR"/run/nvpsf"Runtime directory holding every PSF AF_UNIX socket. Created by tmpfiles.d (see pkg/deb/create_debian_package.sh) and bind-mounted into containers. Mode 1777: the sticky bit stops one endpoint from unlinking another’s socket, but grants no confidentiality, so each socket sets its own mode after bind and validates SO_PEERCRED.
NVPSF_SOCKET_PATH_PSD_GATEWAYNVPSF_RUNTIME_DIR "/nvpsd_gateway"PSD gateway to SDM endpoint. Server: nvpsd_gateway. Defined here so packaging, launcher, and both endpoints agree on one string; the gateway and its SDM clients begin using it when UNIX_SOCKET support lands on that link. The PSS daemon’s two socket paths are deliberately not here. They live in components/event-integrator/daemon/include/pss_daemon.h, which many components already include; moving them would force every one of those components to acquire this header’s include path for no benefit. Each path string is still defined exactly once.
NVPSF_GATEWAY_SDM_MIN_QUEUE_DEPTH516UReceive-buffer floor for the gateway -> SDM link, in whole messages. DERIVED, NOT CHOSEN: with T_service_worst and T_emit_min measured on x86-lin64 against the real SDM decision path and the real gateway respectively. c depth = ceil((MAIN_LOOP_POLL_TIMEOUT_MS + T_service_worst) / T_emit_min) = ceil((50 + 0.146) / 0.1460) = 344, x1.5 margin = 516 This link and no other. On PSS -> PSD and the PSS RPC the depth stays 0: a failed send there surfaces as a retried or failed call, whereas here it burns an already-allocated sequence number, which the SDM cannot distinguish from a lost message so it latches into fault-safe lockout needing a validated manual release. That asymmetry is the whole reason the floor exists. The byte requirement is 516 x sizeof(NvPSDGatewayDecisionRequestPacket), computed from the real message size at runtime rather than hardcoded. That size fell from 2125 to 1621 bytes when SafetyEvent’s 64-byte ruleIdentifier was replaced by a 1-byte cause, so the requirement dropped from 1096500 to 1. Deployments already provisioning 1096500 remain correct and over-provisioned; there is no need to lower a working sysctl. A shortfall is a FATAL startup error, not a warning, and the transport names the granted value, the required value and the sysctl to change.
NVPSF_PEER_INVALID((NvPSFPeerId)UINT32_MAX)No peer. Not a collidable value: slot indices are capped well below 0xFFFF by NVPSF_TRANSPORT_MAX_PEERS_LIMIT, so no live identity can equal it.
NVPSF_TRANSPORT_MAX_PEERS_LIMIT64UHard upper bound on peers per endpoint, independent of configuration.
NVPSF_TRANSPORT_MAX_MESSAGE_BYTES65536ULargest message any link may carry. Bounds every internal buffer.