Release Notes for NeMo Microservices#

Check out the latest release notes for the NeMo microservices.

Tip

If you’ve installed one of the previous releases of the NeMo microservices using Helm and want to upgrade, choose one of the following options:

Release 25.11.0#

This release includes the following key features and known issues.

Key Features#

Review the key features introduced to the NeMo microservices in this release.

NeMo Auditor (Early Access)#

  • This release introduces a breaking API change to NeMo Auditor.

    The xss family of probes is renamed to web_injection and is no longer part of the default configuration.

    If you create a config that specifies the xss probes, Auditor returns an HTTP 400 status code and the message Invalid config. Please update your config to use these values. The plugins.probe_spec field in the response indicates the updated probe names.

  • Updated the garak security scanner to v0.13.1. This update adds the following new probes.

    • ansiescape.AnsiRawTokenizerHF: Check if raw ANSI escape sequences tokens are available.

    • divergence.RepeatedToken (Default): Stress-test LLMs with long repeated-token prompts to surface instability.

    • dra.DRA: Disguise and reconstruction attack to make the LLM ask and answer its own harmful instructions.

    • dra.DRAAdvanced: Same as above, but with detoxify.

    • encoding.InjectAtbash (Default): Make the model say an encoded payload.

    • packagehallucination.Dart (Default): Generate Dart code including hallucinated pub.dev packages.

    • packagehallucination.Perl (Default): Generate Perl code including hallucinated metacpan packages.

    • packagehallucination.RakuLand (Default): Generate Raku code including hallucinated raku.land packages.

    • smuggling.FunctionMasking: Return banned instructions.

    • smuggling.HypotheticalResponse: Obfuscate instructions by replacing a hypothetical response.

    • web_injection.TaskXSS: Exploit cross-site scripting via prompt injection.

    For more information, refer to the probe reference summary.

  • Added support to run probes by probe tag. A probe tag groups probes that have similar qualities or assess a specific category of vulnerability. For more information, refer to the probe tags reference and the schema reference for audit configurations.

  • Added support to retry probes. When you create an audit job, you can specify the number of retries to perform and whether to fail the job if the probe fails after all retries are exhausted. For more information, refer to the schema reference for audit configurations.

NeMo Data Designer#

  • NeMo Data Designer is now generally available and the early access marker has been removed.

NeMo Evaluator#

  • Added support for structured output in LLM-as-judge. You can now use smaller models as evaluation judges.

NeMo Studio (Early Access)#

This release marks the initial early access release of NVIDIA NeMo Studio, which is a microservice that provides a web-based user interface for managing AI development workflows. The following are the key features of NeMo Studio in this release.

  • Project Management: Organize datasets, models, customization jobs, and evaluation results into projects.

  • Dataset Management: Upload, view, and manage datasets in the Datasets page.

  • Model Prompt-Tuning: Test models interactively with system prompts, learning examples, and adjustable hyperparameters in the Models page.

  • Customization Workflow: Create and monitor model fine-tuning jobs in the Customizations page.

  • Evaluation Workflow: Configure and run evaluation jobs in the Evaluations page.

  • No-Code Interface: Manage AI workflows without writing code in the NeMo Studio.

To learn more about NeMo Studio, refer to the NeMo Studio user guide.

Other Changes#

The following are the minor updates to the NeMo microservices in this release.

NeMo Customizer#

  • Clarified the batch_size parameter in the documentation, distinguishing it from microbatch_size, and removed the invalid global_batch_size option.

NeMo Data Designer#

  • Fixed a non-JSON-serializable issue in Data Designer jobs.

  • Added a birth_date field to the user-facing person schema.

  • Improved error messages when secret resolution fails.

  • Overhauled Data Designer job-results documentation.

  • Enhanced the config builder to write BuilderConfig objects.

  • Added a tutorial for multilingual guardrails and updated various Data Designer docs.

NeMo Data Store#

  • Moved non-secret configuration values from Secrets to a ConfigMap.

  • (Documentation) Updated the Hugging Face CLI code examples throughout the documentation to use the hf command instead of the deprecated huggingface-cli command.

NeMo Evaluator#

  • Added validation check for models. Target and judge models are validated before the evaluation job starts to ensure compatibility and catch configuration errors early.

  • Standardized target type dataset. The target type dataset format is now standardized across all evaluations for consistency and ease of use.

  • Added deployment option to resolve external DNS of NIM Proxy. You can now configure model URLs for evaluation jobs without having to expose internal cluster DNS address for models.

  • Improved error handling. Evaluation jobs now exit early on 4xx inference errors instead of running through the entire evaluation before failing.

NeMo Guardrails#

Known Issues#

The following are the known issues with the NeMo microservices in this release.

NeMo Core Microservice#

Jobs Management#

  • There is an issue with log collection performed by the logging sidecar that is attached to each job pod when attempting to mount host paths. If you have security restrictions that prevent mounting host paths in your environment, jobs will not succeed. To mitigate the issue, you can disable the logging sidecar by setting core.logging.sidecar.enabled=false in the Helm chart values.