runai inference

inference management

Options

  -h, --help   help for inference

Options inherited from parent commands

      --config-file string   config file name; can be set by environment variable RUNAI_CLI_CONFIG_FILE (default "config.json")
      --config-path string   config path; can be set by environment variable RUNAI_CLI_CONFIG_PATH
  -d, --debug                enable debug mode
  -q, --quiet                enable quiet mode, suppress all output except error messages
      --verbose              enable verbose mode

SEE ALSO

• runai - Run:ai Command-line Interface
• runai inference bash - open a bash shell in an inference workload
• runai inference delete - delete an inference workload
• runai inference describe - describe an inference workload
• runai inference distributed - Runs multiple coordinated inference processes across multiple nodes. Required for models too large to run on a single node.
• runai inference exec - execute a command in an inference workload
• runai inference list - list inference workloads
• runai inference logs - view logs of an inference workload
• runai inference nim - [Experimental] Runs NVIDIA NIM (NVIDIA Inference Microservices) workloads. Optimized for deploying foundation models.
• runai inference port-forward - forward one or more local ports to an inference workload
• runai inference standard - Runs a single inference process on one node. Suitable for smaller models or simpler inference tasks.
• runai inference submit - submit an inference workload
• runai inference update - update an inference workload