> This page is for version 5.0 (Latest) (default). > For other versions, use one of these documentation indexes: > - 5.0 (Latest) (default): https://docs.nvidia.com/holoscan/sdk-user-guide/5-0-latest/llms.txt > - ltsb-2.0: https://docs.nvidia.com/holoscan/sdk-user-guide/ltsb-2-0/llms.txt > - 4.6: https://docs.nvidia.com/holoscan/sdk-user-guide/4-6/llms.txt > - 4.5: https://docs.nvidia.com/holoscan/sdk-user-guide/4-5/llms.txt > - 4.4: https://docs.nvidia.com/holoscan/sdk-user-guide/4-4/llms.txt > - 4.3: https://docs.nvidia.com/holoscan/sdk-user-guide/4-3/llms.txt > - Archives: https://docs.nvidia.com/holoscan/sdk-user-guide/archives/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/holoscan/sdk-user-guide/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/holoscan/sdk-user-guide/_mcp/server. Holoscan 5 compiles graphs into execution partitions, activation rules, and bounded channels. Tune those resources after verifying the pipeline's delivery and termination behavior on the target machine. > **You will learn** > > * How operator boundaries and partition placement affect overhead > * Why queue depth and loss policy are separate from producer pacing > * Where to find reproducible SDK benchmarks and profiling guidance ## Operator granularity and execution placement A small `compute()` can cost less than its dispatch and data movement. Compare one combined operator with reusable operators placed in a suitable partition; measure both under the same workload. The compiler can optimize eligible linear chains, but eligibility depends on the contracts, payloads, and placement. Contract omission is supported; synthesized `OnAnyInput` has different fast-path and rate-sizing eligibility from explicit `OnEach`. Use [Graph composition](/holoscan/sdk-user-guide/composing-applications/graph-composition) for automatic partitioning, explicit partitions, and deployment thread placement. Measure dispatch and placement costs with the shipped benchmarks under the workload and deployment configuration you intend to use. ## Bounded queues and overload Increasing a queue can absorb a finite burst, but does not make sustained producer/consumer imbalance disappear. `kBlocking` preserves accepted samples; it does not guarantee that an emit waits until it succeeds. A refusal can return `kBackpressured`, and propagating it from `compute()` normally fails the run. `kDropOldest` changes which samples survive and must be an intentional choice. A slower nominal source is not a guarantee against scheduling stalls. See [Understanding failures](/holoscan/sdk-user-guide/debugging-and-testing/understanding-failures) for publication ownership and retry constraints, and [Performance model](/holoscan/sdk-user-guide/performance/performance-model) for the costs of dispatch, channels, and retention. ## Measure comparable workloads Start with the [Core Runtime benchmarks](/holoscan/sdk-user-guide/performance/core-runtime-benchmarks) and [Channel benchmarks](/holoscan/sdk-user-guide/performance/channel-benchmarks). Keep payload, offered rate, work, backend, queue geometry, partition cover, stream policy, affinity, and termination criteria fixed when comparing authoring choices. Report warmup, hardware, build revision, and measurement scope. A change in fast-path eligibility is not a measured speedup by itself. Use [Nsight profiling](/holoscan/sdk-user-guide/performance/nsight-profiling) to distinguish CPU scheduling, channel pressure, GPU work, and synchronization. A [CUDA Graph region](/holoscan/sdk-user-guide/composing-applications/cuda-graph-regions) can reduce launch overhead for eligible GPU chains; compare ordinary and captured execution with the same downstream Tensor lifetime obligations. ## Host CPU Isolation for Real-Time Schedulers Deployment thread placement controls where execution partitions and configured support threads run. It does not stop unrelated work from using those CPUs. Consider isolation only after traces show that host contention causes material jitter; reserve housekeeping CPUs for the OS and unpinned helpers. > **Note** > > For an end-to-end worked example of CPU core isolation on a real platform, see the Holohub high-performance networking tutorial, section 3.5 ["Isolate CPU Cores"](https://nvidia-holoscan.github.io/holohub/tutorials/high_performance_networking/#35-isolate-cpu-cores). The following kernel command-line parameters work together: * **`isolcpus=`** * *What it does:* Removes the listed cores from the kernel scheduler's general load-balancing domain so threads land on them only when explicitly pinned (via deployment affinity, `taskset`, or `sched_setaffinity`). * *Cost / tradeoff:* Those cores no longer participate in general workload balancing — your housekeeping cores absorb everything that is not explicitly pinned, so size the unisolated pool for the rest of the system's load. * *How to tune:* Apply to exactly the cores your pinned execution partitions and configured support threads use. Skip it if your pipeline already runs comfortably on a shared core pool; isolation only helps when measurable jitter from co-scheduled work is the bottleneck. * **`nohz_full=`** * *What it does:* Disables the periodic scheduler tick on the listed cores while only one task is runnable on the core, typically reducing timer-interrupt jitter for long-running RT workers. * *Cost / tradeoff:* Tick offload work shifts to the timekeeping (housekeeping) core, and on most kernels the tick is only suppressed when a single task is runnable on the core — additional pinned threads on the same core re-enable it. * *How to tune:* Use the same list as `isolcpus`, but ensure each `nohz_full` core hosts only one runnable RT task. Skip it if you have not observed tick-driven jitter in profiling traces; the benefit is small for workloads already dominated by GPU or I/O latency. * **`rcu_nocbs=`** * *What it does:* Offloads RCU (read-copy-update) callback processing from the listed cores onto dedicated kernel threads that run elsewhere. * *Cost / tradeoff:* The offloaded RCU callbacks consume cycles on your housekeeping cores instead, so a heavily loaded RCU subsystem can add load there. * *How to tune:* Match the list to `isolcpus` / `nohz_full`. This is generally cheap to enable and worth doing whenever you isolate cores. These three parameters are typically used with the same core list. ### CPU Frequency Governor * *What it does:* The `performance` governor pins each selected core to its maximum operating frequency, eliminating frequency-scaling and wake-from-idle latency that the default `ondemand` / `schedutil` governors introduce. * *Cost / tradeoff:* Idle power draw and thermal output rise on the affected cores; on battery- or thermally-constrained systems this is a real cost. * *How to tune:* Apply only to the cores hosting your real-time execution partitions. Skip it if you have not seen frequency-scaling-induced jitter in your traces — on many server-class systems the default governor is fast enough that pinning to `performance` is unnecessary. ```bash sudo cpupower frequency-set -g performance ``` To target only specific cores: ```bash sudo cpupower -c 1-6 frequency-set -g performance ``` ### Worked Example For an application that explicitly places its execution partitions on CPUs 1–6, a matching isolation configuration could be: ```text isolcpus=1-6 nohz_full=1-6 rcu_nocbs=1-6 ``` On x86 hosts, this is set in `/etc/default/grub` (the `GRUB_CMDLINE_LINUX` line) followed by `update-grub`. On IGX and Jetson, it is set in `/boot/extlinux/extlinux.conf` under the `APPEND` line of the active boot entry. Refer to your platform's documentation for the exact procedure and reboot requirements. After rebooting, set the governor for the selected cores, then run the application with its graph/deployment affinity configuration: ```bash sudo cpupower -c 1-6 frequency-set -g performance ./my_holoscan_app ``` ### Verification Confirm the kernel sees the isolated set: ```bash cat /sys/devices/system/cpu/isolated # Expected: 1-6 ``` Confirm a running thread is pinned where you expect (replace `` with a worker thread ID from `ps -eLo pid,tid,comm | grep my_app`): ```bash taskset -c -p # Expected: pid 's current affinity list: 2 ``` > **Warning** > > **Leave at least one core unisolated.** The kernel, userspace shell, container runtime, and unpinned Holoscan threads all need somewhere to run. Isolating every core (e.g. `isolcpus=0-N` on an N+1-core system) starves the kernel of a general-purpose CPU and can hang the system. Reserve core `0` (and ideally one more) for the OS. Holoscan keeps its own unpinned helper threads off the isolated set. The HoloMQ and Python Tensor release workers run on the process's CPUs minus the isolated ones. Before it pins any thread, the Runtime creates the CUDA context of each device a Graph uses, and launches one host function on it, from a thread on those CPUs. CUDA's own threads take the CPU set of the thread whose call starts them: context creation starts its event-handler threads, and the first host-function launch starts the thread that runs host functions. If your application initializes CUDA or launches host functions itself before running the Graph, do it from an unpinned thread. A CUDA thread started on an isolated, pinned thread stays there, which can delay CUDA host functions, and the HoloMQ buffer releases that depend on them, by milliseconds. > **Tip** > > `nohz_full` requires `CONFIG_NO_HZ_FULL=y` in the kernel, which is set on most stock distributions but not all. If isolation is configured but jitter persists, check `cat /sys/devices/system/cpu/nohz_full` — an empty result means the option was ignored. ## Recap * Measure operator boundaries and partitions on the target workload. * Queue size, loss policy, and producer pacing are separate decisions. * Keep experimental controls fixed and record provenance for performance claims. * Use the SDK's current benchmarks and deployment placement APIs.