GLiNER Integration

GLiNER is a generalist and lightweight model for named entity recognition. NVIDIA GLiNER-PII is an adaptation that detects a wide range of PII categories. This integration enables NeMo Guardrails to use GLiNER-PII for PII detection and masking in input, output, and retrieval flows.

Prerequisites

To use the NVIDIA-hosted NIMs, set your NVIDIA API key:

$
export NVIDIA_API_KEY="nvapi-..."

You can obtain an API key at build.nvidia.com.

You will also need to install the NeMo Guardrails library.

Configure Guardrails

Create a config/ directory with one subdirectory per use case. The examples below cover two flows — PII detection and PII masking — both targeting the NVIDIA-hosted GLiNER-PII and Llama 3.1 8B NIM endpoints.

config/
├── pii_detection/
│   └── config.yml
└── pii_masking/
    └── config.yml

NeMo Guardrails loads every .yml / .yaml file in the directory passed to --config and merges them into a single configuration. Keeping each flow in its own subdirectory prevents the detection and masking rule sets from colliding; the Chat CLI then selects a flow with --config config/pii_detection or --config config/pii_masking.

nvidia/gliner-pii does not appear in the configs below because it is the default value of rails.config.gliner.model. You only need to set that field explicitly if you want to use a different model.

PII Detection

The detection flow blocks any input or output that contains PII. To implement this flow, save the config below as config/pii_detection/config.yml.

1
models:
2
  - type: main
3
    engine: nim
4
    model: meta/llama-3.1-8b-instruct
5
    api_key_env_var: NVIDIA_API_KEY
6
7
rails:
8
  config:
9
    gliner:
10
      server_endpoint: https://integrate.api.nvidia.com/v1/chat/completions
11
      api_key_env_var: NVIDIA_API_KEY
12
      threshold: 0.5
13
      input:
14
        entities:
15
          - first_name
16
          - last_name
17
          - email
18
          - phone_number
19
      output:
20
        entities:
21
          - first_name
22
          - last_name
23
          - email
24
          - phone_number
25
  input:
26
    flows:
27
      - gliner detect pii on input
28
  output:
29
    flows:
30
      - gliner detect pii on output

PII Masking

The masking flow replaces detected PII with label placeholders before the LLM processes the text, rather than blocking the request outright. For example, Hi, I am John. My email is john@example.com becomes Hi, I am [FIRST_NAME]. My email is [EMAIL]. To implement this flow, save the config below as config/pii_masking/config.yml.

1
models:
2
  - type: main
3
    engine: nim
4
    model: meta/llama-3.1-8b-instruct
5
    api_key_env_var: NVIDIA_API_KEY
6
7
rails:
8
  config:
9
    gliner:
10
      server_endpoint: https://integrate.api.nvidia.com/v1/chat/completions
11
      api_key_env_var: NVIDIA_API_KEY
12
      threshold: 0.5
13
      input:
14
        entities:
15
          - first_name
16
          - last_name
17
          - email
18
          - phone_number
19
      output:
20
        entities:
21
          - first_name
22
          - last_name
23
          - email
24
          - phone_number
25
  input:
26
    flows:
27
      - gliner mask pii on input
28
  output:
29
    flows:
30
      - gliner mask pii on output

Run the Guardrails Chat CLI

Start an interactive chat session by pointing --config at the subdirectory for the flow you want to test:

$
# Detection: block messages that contain PII
$
nemoguardrails chat --config config/pii_detection
$
$
# Masking: replace PII with placeholders before the LLM sees the message
$
nemoguardrails chat --config config/pii_masking

With PII detection enabled, any message containing PII is blocked before reaching the LLM:

> Hello! My name is John and my email is john@example.com.
I'm sorry, I can't respond to that.

With PII masking enabled, PII is replaced in-place before the LLM sees the message:

> Hello! My name is John and my email is john@example.com.
Nice to meet you, [FIRST_NAME]! How can I help you today?

Use the Python SDK

1
import nest_asyncio
2
nest_asyncio.apply()
3
4
from nemoguardrails import LLMRails, RailsConfig
5
6
# Use config/pii_detection or config/pii_masking depending on the flow you want.
7
config = RailsConfig.from_path("./config/pii_detection")
8
rails = LLMRails(config)
9
10
response = rails.generate(
11
    messages=[{"role": "user", "content": "Hello! My name is John and my email is john@example.com."}]
12
)
13
print(response["content"])
14
15
# Inspect the guardrail execution trace
16
info = rails.explain()
17
print(info.colang_history)

Deploy NIMs Locally

Running both NIMs locally eliminates network round-trips and removes the NVIDIA API key requirement for inference. You still need an NGC Personal API key — generate one at org.ngc.nvidia.com/setup/api-keys with at least the NGC Catalog service selected — to pull the Docker images and download the model artifacts.

GPU Requirements

The Llama NIM auto-selects the optimal TensorRT-LLM profile (FP16 or INT8) based on available hardware. An A10G (24 GB) or L4 (24 GB) is the practical minimum for comfortable headroom; a T4 (16 GB) may work but is not officially supported.

Note: nvidia/gliner-pii is pre-GA (1.0.0-rc1). The GPU requirements above are estimates based on the GLiNER encoder-only architecture because NVIDIA has not officially published requirements yet.

Start the Containers

Export your NGC Personal API key as NGC_API_KEY:

$
export NGC_API_KEY="<your-ngc-key>"

Important: The key must start with nvapi-. You can generate one at org.ngc.nvidia.com/setup/api-keys (select at least the NGC Catalog service) or at build.nvidia.com — both portals issue interchangeable nvapi- keys. Legacy NGC keys (older format, not starting with nvapi-) will cause the GLiNER container to fail during model-artifact download.

If you already have an NVIDIA_API_KEY starting with nvapi-, you can reuse it:

$
export NGC_API_KEY="$NVIDIA_API_KEY"

Alternatively, you can pass the key directly at container runtime — this avoids overwriting any existing NGC_API_KEY in your environment:

$
docker run ... -e NGC_API_KEY="$NVIDIA_API_KEY" ...

On a multi-GPU host, pin each container to a distinct GPU with --gpus '"device=N"' instead of --gpus all. Without an explicit device, both NIMs default to GPU 0 and compete for memory. The examples below assign GLiNER to GPU 0 and Llama to GPU 1; adjust the indices to match your host.

The GLiNER-PII NIM runs on port 8000 (GPU 0):

$
# Authenticate with NGC (username: $oauthtoken, password: your NGC API key)
$
echo $NGC_API_KEY | docker login -u '$oauthtoken' --password-stdin nvcr.io
$
$
docker run --rm -it --gpus '"device=0"' \
>
  -e NGC_API_KEY \
>
  -p 8000:8000 \
>
  nvcr.io/nim/nvidia/gliner-pii:1.0.0-rc1

Map the Llama 3.1 8B Instruct NIM to port 8001 (GPU 1) to avoid a conflict with GLiNER:

$
docker run --rm -it --gpus '"device=1"' \
>
  -e NGC_API_KEY \
>
  -p 8001:8000 \
>
  nvcr.io/nim/meta/llama-3.1-8b-instruct:latest

Wait until both containers log Application startup complete before proceeding.

Update the Configuration

Update both config.yml files (under config/pii_detection/ and config/pii_masking/) with the local-endpoint versions below, removing the api_key_env_var fields:

PII detection:

1
models:
2
  - type: main
3
    engine: nim
4
    model: meta/llama-3.1-8b-instruct
5
    parameters:
6
      base_url: http://localhost:8001/v1
7
8
rails:
9
  config:
10
    gliner:
11
      server_endpoint: http://localhost:8000/v1/chat/completions
12
      threshold: 0.5
13
      input:
14
        entities:
15
          - first_name
16
          - last_name
17
          - email
18
          - phone_number
19
      output:
20
        entities:
21
          - first_name
22
          - last_name
23
          - email
24
          - phone_number
25
  input:
26
    flows:
27
      - gliner detect pii on input
28
  output:
29
    flows:
30
      - gliner detect pii on output

PII masking:

1
models:
2
  - type: main
3
    engine: nim
4
    model: meta/llama-3.1-8b-instruct
5
    parameters:
6
      base_url: http://localhost:8001/v1
7
8
rails:
9
  config:
10
    gliner:
11
      server_endpoint: http://localhost:8000/v1/chat/completions
12
      threshold: 0.5
13
      input:
14
        entities:
15
          - first_name
16
          - last_name
17
          - email
18
          - phone_number
19
      output:
20
        entities:
21
          - first_name
22
          - last_name
23
          - email
24
          - phone_number
25
  input:
26
    flows:
27
      - gliner mask pii on input
28
  output:
29
    flows:
30
      - gliner mask pii on output

Reuse the CLI and SDK Workflows

With the containers running and config updated, rerun the CLI and SDK commands from Run the Guardrails Chat CLI and Use the Python SDK. No other changes are required.

API Specification

The GLiNER-PII NIM exposes an OpenAI-compatible chat completions endpoint.

Chat Completions Endpoint

Extract entities from text.

Request Body:

Example Request:

1
{
2
  "model": "nvidia/gliner-pii",
3
  "messages": [{"role": "user", "content": "Hello, my name is John and my email is john@example.com"}],
4
  "labels": ["email", "first_name"],
5
  "threshold": 0.5
6
}

Response Body:

The response follows the OpenAI chat completions format. The choices[0].message.content field contains a JSON string with the detected entities.

Parsed Content Fields:

EntitySpan Object:

Example Parsed Content:

1
{
2
  "entities": [
3
    {
4
      "text": "John",
5
      "label": "first_name",
6
      "start": 18,
7
      "end": 22,
8
      "score": 0.95
9
    },
10
    {
11
      "text": "john@example.com",
12
      "label": "email",
13
      "start": 40,
14
      "end": 56,
15
      "score": 0.98
16
    }
17
  ],
18
  "total_entities": 2,
19
  "tagged_text": "Hello, my name is [John](first_name) and my email is [john@example.com](email)"
20
}

Supported Entity Types

The NVIDIA GLiNER-PII NIM supports these PII categories:

Configuration Options

Testing

The GLiNER integration tests in tests/test_gliner.py use mocked API responses, so they don’t require a running server. To run them:

$
pytest tests/test_gliner.py -v

For a self-hosted alternative using the nvidia/gliner-PII model directly, the examples/deployment/gliner_server/ directory provides a reference implementation. The server exposes a POST /v1/extract endpoint. If you use it, set server_endpoint to http://localhost:1235/v1/extract. Refer to the deployment README for setup instructions.

For more information on GLiNER, refer to the GLiNER GitHub repository.