Classification#

Classify text by using classification NIM microservices through the NIM Proxy microservice.

You can use the following classification NIM microservices through the NIM Proxy microservice:

Prerequisites#

Before you start, make sure that you have:

  • Access to the NIM Proxy microservice through the base URL where the service is deployed. Store the base URL in an environment variable NIM_PROXY_BASE_URL.

  • A valid classification model name. To retrieve the list of models deployed as NIM microservices in your environment and exposed through the NIM Proxy microservice, use the ${NIM_PROXY_BASE_URL}/v1/models API. For more information, see List Models.

Deploy the Jailbreak Detection NIM#

If you need to deploy the Jailbreak Detection NIM, use the NeMo Deployment Management microservice. For more information about deploying NIM microservices, see Deploy NIM Microservices.

The following examples show how to deploy the Jailbreak Detection NIM.

from nemo_microservices import NeMoMicroservices
import os

client = NeMoMicroservices(
    base_url=os.environ["DEPLOYMENT_BASE_URL"]
)

# Deploy the jailbreak detection NIM
deployment = client.deployment.model_deployments.create(
    name="nemoguard-jailbreak-detect",
    namespace="nvidia",
    config={
        "model": "nvidia/nemoguard-jailbreak-detect",
        "nim_deployment": {
            "image_name": "nvcr.io/nim/nvidia/nemoguard-jailbreak-detect",
            "image_tag": "1.0.0",
            "pvc_size": "25Gi",
            "gpu": 1
        }
    }
)

print(f"Jailbreak detection NIM deployment created: {deployment.namespace}/{deployment.name}")

curl --location "${DEPLOYMENT_BASE_URL}/v1/deployment/model-deployments" \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "nemoguard-jailbreak-detect",
    "namespace": "nvidia",
    "config": {
      "model": "nvidia/nemoguard-jailbreak-detect",
      "nim_deployment": {
        "image_name": "nvcr.io/nim/nvidia/nemoguard-jailbreak-detect",
        "image_tag": "1.0.0",
        "pvc_size": "25Gi",
        "gpu": 1
      }
    }
  }' | jq

To Classify Text#

Choose one of the following options for classifying text.

Create a NeMoMicroservices client instance using the base URL of the NIM Proxy microservice and perform the task as follows.

from nemo_microservices import NeMoMicroservices
import os

client = NeMoMicroservices(
    base_url=os.environ["NEMO_BASE_URL"],
    inference_base_url=os.environ["NIM_PROXY_BASE_URL"]
)
response = client.classify.create(
    input="This might be a jailbreak attempt.",
    model="nvidia/nemoguard-jailbreak-detect"
)

print(response)

Use the following cURL command. The v1/classify endpoint requires both the input and model fields

curl -X 'POST' \
    "${NIM_PROXY_BASE_URL}/v1/classify" \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
        "input": "This might be a jailbreak attempt.",
        "model": "nvidia/nemoguard-jailbreak-detect"
    }'
Example Response
{
  "jailbreak": "true",
  "score": "0.8785208151759366"
}
On this page