Contents

• 1. Introduction
  • 1.1. At a Glance
    • 1.1.1. MPS
    • 1.1.2. Volta MPS
    • 1.1.3. Intended Audience
    • 1.1.4. Organization of This Document
  • 1.2. Prerequisites
  • 1.3. Concepts
    • 1.3.1. Why MPS is Needed
    • 1.3.2. What MPS Is
  • 1.4. See Also
• 2. When to Use MPS
  • 2.1. The Benefits of MPS
    • 2.1.1. GPU Utilization
    • 2.1.2. Reduced On-GPU Context Storage
    • 2.1.3. Reduced GPU Context Switching
  • 2.2. Identifying Candidate Applications
  • 2.3. Considerations
    • 2.3.1. System Considerations
      • 2.3.1.1. Limitations
      • 2.3.1.2. GPU Compute Modes
    • 2.3.2. Application Considerations
    • 2.3.3. Memory Protection and Error Containment
      • 2.3.3.1. Memory Protection
      • 2.3.3.2. Error Containment
    • 2.3.4. MPS on Multi-GPU Systems
    • 2.3.5. Performance
      • 2.3.5.1. Client-Server Connection Limits
      • 2.3.5.2. Volta MPS Execution Resource Provisioning
      • 2.3.5.3. Threads and Linux Scheduling
      • 2.3.5.4. Volta MPS Device Memory Limit
    • 2.3.6. Interaction with Tools
      • 2.3.6.1. Debugging and CUDA-GDB
      • 2.3.6.2. memcheck
      • 2.3.6.3. Profiling
    • 2.3.7. Client Early Termination
    • 2.3.8. Client Priority Level Control
• 3. Architecture
  • 3.1. Background
  • 3.2. Client-server Architecture
  • 3.3. Provisioning Sequence
    • 3.3.1. Server
    • 3.3.2. Client Attach/Detach
• 4. Appendix: Tools and Interface Reference
  • 4.1. Utilities and Daemons
    • 4.1.1. nvidia-cuda-mps-control
    • 4.1.2. nvidia-cuda-mps-server
    • 4.1.3. nvidia-smi
  • 4.2. Environment Variables
    • 4.2.1. CUDA_VISIBLE_DEVICES
    • 4.2.2. CUDA_MPS_PIPE_DIRECTORY
    • 4.2.3. CUDA_MPS_LOG_DIRECTORY
    • 4.2.4. CUDA_DEVICE_MAX_CONNECTIONS
    • 4.2.5. CUDA_MPS_ACTIVE_THREAD_PERCENTAGE
      • 4.2.5.1. MPS Control Daemon Level
      • 4.2.5.2. Client Process Level
      • 4.2.5.3. Client CUDA Context Level
    • 4.2.6. CUDA_MPS_ENABLE_PER_CTX_DEVICE_MULTIPROCESSOR_PARTITIONING
    • 4.2.7. CUDA_MPS_PINNED_DEVICE_MEM_LIMIT
    • 4.2.8. CUDA_MPS_CLIENT_PRIORITY
  • 4.3. MPS Logging Format
    • 4.3.1. Control Log
    • 4.3.2. Server Log
  • 4.4. MPS Known Issues
• 5. Appendix: Common Tasks
  • 5.1. Starting and Stopping MPS on Linux
    • 5.1.1. On a Multi-user System
      • 5.1.1.1. Starting MPS control daemon
      • 5.1.1.2. Shutting Down MPS Control Daemon
      • 5.1.1.3. Log Files
    • 5.1.2. On a Single-User System
      • 5.1.2.1. Starting MPS Control Daemon
      • 5.1.2.2. Starting MPS client application
      • 5.1.2.3. Shutting Down MPS
      • 5.1.2.4. Log Files
    • 5.1.3. Scripting a Batch Queuing System
      • 5.1.3.1. Basic Principles
      • 5.1.3.2. Per-Job MPS Control: A Torque/PBS Example
  • 5.2. Best Practice for SM Partitioning
  • 5.3. Troubleshooting Guide
    • 5.3.1. MPS Server Not Accepting New Client Requests
• 6. Notices
  • 6.1. Trademarks