Known Issues#
This topic provides a summary of known issues with DGX Spark systems.
Use the supplied power adapter for optimal performance#
For optimal performance, use the supplied power adapter with the DGX Spark system. Using a different adapter may reduce performance, prevent boot, or cause unexpected shutdowns.
nvidia-smi reports “Memory-Usage: Not Supported”#
On iGPU platforms, nvidia-smi
will display “Memory-Usage: Not Supported” even though per-process GPU memory is listed. This is expected because iGPUs do not have dedicated framebuffer memory.
Memory reporting differences with unified memory architecture#
DGX Spark systems use a unified memory architecture (UMA), where the GPU shares system memory (DRAM) with the CPU and other compute engines. This design reduces latency and allows larger amounts of memory to be used for GPU workloads. On UMA systems, the CPU can dynamically manage DRAM contents, including freeing up memory by swapping pages between DRAM and the system’s SWAP area. However, the cudaMemGetInfo API does not account for memory that could potentially be reclaimed from SWAP. As a result, the memory size reported by cudaMemGetInfo may be smaller than the actual allocatable memory, since the CPU may be able to release additional DRAM pages by moving them to SWAP.
To more accurately estimate the amount of allocatable device memory on DGX Spark platforms, CUDA application developers should consider the possibility of DRAM reclamation via SWAP and not rely solely on the values returned by cudaMemGetInfo. The following provides an example implementation using C standard libraries:
#include <stdio.h>
int getAvailableMemory(long *availableMemoryKb, long *freeSwapKb) {
FILE *meminfoFile = NULL;
char lineBuffer[256];
long hugeTlbTotalPages = -1;
long hugeTlbFreePages = -1;
long hugeTlbPageSize = -1;
if (availableMemoryKb == NULL || freeSwapKb == NULL) {
return 1;
}
meminfoFile = fopen("/proc/meminfo", "r");
if (meminfoFile == NULL) {
return 1;
}
*availableMemoryKb = -1;
*freeSwapKb = -1;
while (fgets(lineBuffer, sizeof(lineBuffer), meminfoFile)) {
long value;
if (sscanf(lineBuffer, "MemAvailable: %ld kB", &value) == 1) {
*availableMemoryKb = value;
} else if (sscanf(lineBuffer, "SwapFree: %ld kB", &value) == 1) {
*freeSwapKb = value;
} else if (sscanf(lineBuffer, "HugePages_Total: %ld", &value) == 1) {
hugeTlbTotalPages = value;
} else if (sscanf(lineBuffer, "HugePages_Free: %ld", &value) == 1) {
hugeTlbFreePages = value;
} else if (sscanf(lineBuffer, "Hugepagesize: %ld kB", &value) == 1) {
hugeTlbPageSize = value;
}
if (*availableMemoryKb != -1 &&
*freeSwapKb != -1 &&
hugeTlbTotalPages != -1 &&
hugeTlbFreePages != -1 &&
hugeTlbPageSize != -1) {
break;
}
}
fclose(meminfoFile);
if (hugeTlbTotalPages != 0 && hugeTlbTotalPages != -1) {
*availableMemoryKb = hugeTlbFreePages * hugeTlbPageSize;
// Hugetlbfs pages are not swappable.
*freeSwapKb = 0;
}
return 0;
}
As a workaround for debugging purposes, you can flush the buffer cache manually with the following command:
sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
After flushing the cache, restart your application.