DGX Kernel Settings and Parameters

Parameter Name

Description

Package Name

Location

ast.modeset=0

Disable the Aspeed display driver. The AST2xxx is the BMC used in our servers DGX-1 and DGX-2

nv-ast-modeset

/etc/default/grub.d/nomodeset.cfg

pci=realloc=on

Allows kernel to reallocate PCI resources if allocations done by BIOS are insufficient. This and pcie_ports=native are both required for NVME hot-plug on DGX2.

nv-enable-nvme-hot-plug

/etc/default/grub.d/enable-nvme-hot-plug.cfg

pcie_ports=native

Use Linux native services for PME, AER, DPC, PCIe hotplug. I.e. not firmware first. This and pci=realloc=on are both required for NVME hot-plug on DGX2.

nv-enable-nvme-hot-plug

/etc/default/grub.d/enable-nvme-hot-plug.cfg

transparent_hugepage=madvise

Disable huge pages system-wide and only enable them inside MADV_HUGEPAGE madvise regions to prevent applications from allocating more memory resources than necessary.

nv-hugepage

/etc/default/grub.d/hugepage.cfg

iommu=pt

Enable pass through mode only and disable DMA translations. This enables optimizations for the CPU inside the DGX A100.

nv-iommu-pt

/etc/default/grub.d/iommu.cfg

crashkernel

Amount of memory to use for crash dumps.

nvidia-crashdump

/etc/default/grub.d/ipmisol.cfg

console=ttyS[0-1],115200n8

Set console to serial port 0 or 1, using 115200 baud, no parity, 8 data bits For dgx-2, dgx-h100, dgx-h800: console=ttyS0,115200n8 Other system types: console=ttyS1,115200n8

nvidia-ipmisol

kernel cmdline

net.ipv4.conf.all.arp_announce = 2

Always use the best local address for this target

nvidia-kernel-defaults

/etc/sysctl.d/20-nvidia-defaults.conf

net.ipv4.conf.default.arp_announce = 2

Always use the best local address for this target

nvidia-kernel-defaults

/etc/sysctl.d/20-nvidia-defaults.conf

net.ipv4.conf.all.arp_ignore = 1

Only reply to ARP requests on the interface which contains the target IP address

nvidia-kernel-defaults

/etc/sysctl.d/20-nvidia-defaults.conf

net.ipv4.conf.default.arp_ignore = 1

Only reply to ARP requests on the interface which contains the target IP address

nvidia-kernel-defaults

/etc/sysctl.d/20-nvidia-defaults.conf

NVreg_EnablePCIERelaxedOrderingMode=1

Sets a reg-key to enable PCIe relaxed-ordering in the GPUs

nvidia-relaxed-ordering-gpu

/etc/modprobe.d/nvidia-relaxed-ordering.conf