Multi-Process Service#

The Multi-Process Service (MPS) is a lightweight runtime service, designed to transparently enable co-operative CUDA multi-process and multi-application workflows on NVIDIA GPUs. It consists of several components:

  • nvidia-cuda-mps-control (Control Daemon Process) – The control daemon is responsible for starting and stopping the server, as well as coordinating connections between clients and servers.

  • nvidia-cuda-mps-server (Server Process) – The server is the clients’ shared connection to the GPU and owner of the GPU scheduling resources.

  • libcuda.so (Client Runtime) – The MPS client runtime is built into the CUDA Driver library and may be used transparently by any CUDA application.

Enabling MPS provides the following benefits:

  • Improved GPU utilization: A single process may not utilize all the compute and memory-bandwidth capacity available on the GPU. MPS allows kernel and memcopy operations from different processes to overlap on the GPU, achieving higher utilization and shorter running times.

  • Reduced GPU context switching: Without MPS, when processes share the GPU their scheduling resources must be swapped on and off the GPU. The MPS server shares one set of scheduling resources between all of its clients, eliminating the overhead of swapping when the GPU is scheduling between those clients.

  • Reduced On-GPU context storage: Without MPS, each CUDA processes using a GPU allocates separate storage and scheduling resources on the GPU. In contrast, the MPS server allocates one copy of GPU storage and scheduling resources shared by all its clients.

Organization of This Document#

This document is organized as follows:

  • Quick Start – describes how to get started.

  • When to use MPS, limitations and best practices – describes what factors to consider when choosing to run an application with or choosing to deploy MPS for your users.

  • Architecture – describes the client-server architecture of MPS in detail and how it multiplexes clients onto the GPU.

  • Appendices – Reference information for the tools and interfaces used by the MPS system and guidance for common use-cases.

See Also#

  • Manpage for nvidia-cuda-mps-control (1)

  • Manpage for nvidia-smi (1)