Evaluation Target Schema Reference#

This page provides a complete schema reference for evaluation targets, including all fields, data types, and nested properties. Use this reference when you need detailed technical specifications for API requests or to understand the full structure of target objects.

When to use this page:

  • You need detailed field specifications for API integration

  • You want to understand all available configuration options

  • You are troubleshooting schema validation errors

  • You need to reference specific nested object structures

For task-oriented guidance, refer to Create Evaluation Target for step-by-step instructions on creating targets, or browse the target type guides for conceptual overviews and examples.

Important

Each target is uniquely identified by a combination of namespace and name. For example, my-organization/my-target.

Available Target Types#

Evaluation targets define what you’re evaluating. NeMo Evaluator supports the following target types:

  • Model – LLM model with inference endpoint

  • RAG – Retrieval-Augmented Generation pipeline

  • Retriever – Document retrieval pipeline

  • Rows – Inline evaluation data

  • Dataset – Reference to stored evaluation data

Primary Schemas#

These are the main schemas you’ll use when creating targets or reading target information from the API.

EvaluationTargetInput#

Use this schema when creating a new evaluation target through POST requests to the /evaluation/targets endpoint.

EvaluationTargetInput object
An entity representing the target of the evaluation.
Properties
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
description string
The description of the entity.
type * string
The type of the evaluation target, e.g., 'model', 'retriever', 'rag'.
Allowed values:
modelcached_outputsretrieverragrowsdataset
cached_outputs object
The model to be evaluated.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
model string | object
The model to be evaluated.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
retriever object
Retriever to be evaluated.
Properties
pipeline object
Retriever pipeline run by Evaluator.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10
cached_outputs object
Cached pre-generated retriever outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated retriever outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
rag object
RAG to be evaluated.
Properties
pipeline object
RAG pipeline.
Properties
retriever * object
The retriever pipeline included in the RAG pipeline.
Properties
pipeline object
Retriever pipeline run by Evaluator.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10
cached_outputs object
Cached pre-generated retriever outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated retriever outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
model * string | object
The generation model for the RAG pipeline.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
context_ordering string
The context ordering for the RAG pipeline.
Default: desc
cached_outputs object
Cached pre-generated RAG outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated RAG outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
rows array
Rows to be evaluated.
Array items:
item object
Allows additional properties: Yes
dataset string | object
Dataset to be evaluated.
Any of:
Option 1: string - A reference to DatasetInputEV.
Option 2: object - A dataset that can be used for fine-tuning or evaluation.
project string
The URN of the project associated with this entity.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string

EvaluationTargetOutput#

Use this schema to understand evaluation target responses from GET requests when retrieving or managing existing targets.

EvaluationTargetOutput object
An entity representing the target of the evaluation.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
Default: eval-target
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
type * string
The type of the evaluation target, e.g., 'model', 'retriever', 'rag'.
Allowed values:
modelcached_outputsretrieverragrowsdataset
cached_outputs object
The model to be evaluated.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
model string | object
The model to be evaluated.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
retriever object
Retriever to be evaluated.
Properties
pipeline object
Retriever pipeline run by Evaluator.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10
cached_outputs object
Cached pre-generated retriever outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated retriever outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
rag object
RAG to be evaluated.
Properties
pipeline object
RAG pipeline.
Properties
retriever * object
The retriever pipeline included in the RAG pipeline.
Properties
pipeline object
Retriever pipeline run by Evaluator.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10
cached_outputs object
Cached pre-generated retriever outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated retriever outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
model * string | object
The generation model for the RAG pipeline.
Any of:
Option 1: string - A reference to ModelOutputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
context_ordering string
The context ordering for the RAG pipeline.
Default: desc
cached_outputs object
Cached pre-generated RAG outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated RAG outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
rows array
Rows to be evaluated.
Array items:
item object
Allows additional properties: Yes
dataset string | object
Dataset to be evaluated.
Any of:
Option 1: string - A reference to DatasetEV.
Option 2: object - A dataset that can be used for fine-tuning or evaluation.

Nested Configuration Schemas#

The schemas in this section define inline objects you can provide when creating evaluation targets. Use these schemas when you want to configure resources directly in the target creation request rather than referencing existing resources by name.

Understanding String References vs Inline Objects#

When creating an evaluation target, fields like model, rag, retriever, and dataset accept two types of values:

  1. String reference – Reference an existing resource by namespace and name (for example, "my-org/my-model")

  2. Inline object – Provide complete configuration using one of the schemas below

Use string references when you have pre-configured resources. Use inline objects when you want to define everything in a single request.

Example using a model target:

# Option 1: Reference an existing model by name
client.evaluation.targets.create(
    type="model",
    model="my-org/existing-model"
)

# Option 2: Provide inline model configuration (uses ModelInput schema)
client.evaluation.targets.create(
    type="model",
    model={
        "inference": {
            "url": "https://api.example.com/v1/chat/completions",
            "model_id": "meta/llama-3.1-8b-instruct",
            "format": "nim"
        }
    }
)

This pattern applies to all target types—use string references for existing resources or inline objects for complete configuration in a single request.

Target Type Configuration Schemas#

These schemas define the structure of inline objects for different target types.

ModelInput#

Define an inline model configuration in the model field when creating a target with type: "model".

ModelInput object
Information about a machine learning model, typically an LLM.
Properties
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
description string
The description of the entity.
spec object
Detailed specification for the model.
Properties
num_parameters * integer
The total number of trainable parameters in the model's neural network architecture.
context_size * integer
The maximum number of tokens to process together in a single forward pass through the model.
num_virtual_tokens * integer
The number of virtual tokens the model can support for techniques such as prompt tuning, where special trainable embeddings are prepended to inputs.
is_chat * boolean
Indicates if the model is designed for multi-turn conversation rather than single-prompt completion.
artifact object
A set of checkpoint files, configs, and other auxiliary info associated with this model.
Properties
gpu_arch string
The GPU architecture the model is optimized for.
precision string
The numerical precision format used to store model weights.
Allowed values:
int8bf16fp16fp32fp8-mixedbf16-mixed
tensor_parallelism integer
The number of GPU devices to split and process the model's neural network layers.
backend_engine string
The inference engine framework to run the model.
Allowed values:
nemotrt_llmvllmfaster_transformerhugging_face
status * string
The current state of the model artifact in the upload/deployment lifecycle.
Allowed values:
createdupload_failedupload_completed
files_url * string
The location where the artifact files are stored.
Constraints: min length: 1
base_model string | object
Link to another model which is used as a base for the current model.
Any of:
Option 1: string - A reference to Model.
Option 2: object - Information about a machine learning model, typically an LLM.
api_endpoint object
Data about the inference endpoint for this model.
Properties
url * string
The API endpoint URL.
Constraints: min length: 1
model_id * string
The id of the model. How this is used depends on the API endpoint format.
api_key string
The API key that should be used to access the endpoint.
format string
The format for the API endpoint, e.g., NVIDIA NIM, OpenAI, Llama-Stack.
Allowed values:
nimopenailama_stack
Default: nim
peft object
Configuration parameters for parameter efficient fine-tuning (PEFT) built into this model. PEFT methods like LoRA provide efficient model customization by training a small number of additional parameters while keeping the base model frozen.
Properties
finetuning_type * string
The type of finetuning.
Allowed values:
loralora_mergedall_weights
lora object
Configuration parameters for low-rank adaptation (LoRA) fine tuning, which injects trainable rank decomposition matrices into transformer layers to reduce parameters while maintaining quality. These parameters control adaptation strength using alpha scaling and layer selection (MLP/output) to optimize the balance between model size and performance.
Properties
alpha * integer
A scaling factor that controls how much influence the LoRA adaptations have on the base model's behavior. The alpha parameter should typically be set to dim or 0.5 * dim as the actual scaling applied in the training loop is alpha / dim.
apply_lora_to_mlp * boolean
Controls whether to adapt the model's feed-forward neural network layers using LoRA.
apply_lora_to_output * boolean
Controls whether to adapt the model's final output layer using LoRA.
p_tuning object
Configuration details for P-tuning optimization of the model.
Properties
token_embeddings * array
Learned continuous prompt embeddings that help optimize the model for specific tasks.
Array items:
item array
Array items:
item number
prompt object
Configuration for prompt engineering, which involves crafting input text to guide the model's behavior and responses through system prompts, few-shot examples, and other techniques.
Properties
system_prompt string
Initial instructions that define the model's role and behavior for the conversation.
inference_params object
Custom settings that control the model's text generation behavior.
Properties
top_p number
top_k integer
temperature number
max_tokens integer
stop array
Array items:
item string
reasoning_params object
Custom settings that control the model's reasoning behavior.
Properties
end_token string
Configure the end token to trim reasoning context based on the model's reasoning API. Used for omitting Nemotron reasoning steps from output denoted with tags
include_if_not_finished boolean
Configure whether to include reasoning context if the model has not finished reasoning.
effort string
Option for OpenAI models to specify low, medium, or high reasoning effort which balances between speed and reasoning accuracy.
icl_few_shot_examples string
Example input-output pairs that guide the model in understanding the desired task format and behavior.
system_prompt_template string
The template which will be used to compile the final prompt used for prompting the LLM. Currently supports only 'icl_few_shot_examples'
guardrails object
Data about the guardrail configuration built into this model. Guardrails provide safety and quality controls around model outputs by defining rules and filters that validate, modify or block responses based on criteria like toxicity, factuality, or custom business logic.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
Default: guardrail
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
data object
Guardrail configuration data
Allows additional properties: Yes
Properties
models * array
The list of models used by the rails configuration.
Array items:
item object
Configuration of a model used by the rails engine. Typically, the main model is configured e.g.: { "type": "main", "engine": "openai", "model": "gpt-3.5-turbo-instruct" }
Properties
type * string
engine * string
model string
The name of the model. If not specified, it should be specified through the parameters attribute.
api_key_env_var string
Optional environment variable with model's API Key. Do not include "$".
reasoning_config object
Configuration parameters for reasoning LLMs.
Properties
remove_reasoning_traces boolean
For reasoning models (e.g. DeepSeek-r1), if the output parser should remove reasoning traces.
Default: True
remove_thinking_traces boolean
start_token string
The start token used for reasoning traces.
Default:
end_token string
The end token used for reasoning traces.
Default:
parameters object
Allows additional properties: Yes
mode string
Whether the mode is 'text' completion or 'chat' completion. Allowed values are 'chat' or 'text'.
Allowed values:
chattext
Default: chat
instructions array
List of instructions in natural language that the LLM should use.
Default: [{'type': 'general', 'content': 'Below is a conversation between a helpful AI assistant and a user. The bot is designed to generate human-like text based on the input that it receives. The bot is talkative and provides lots of specific details. If the bot does not know the answer to a question, it truthfully says it does not know.'}]
Array items:
item object
Configuration for instructions in natural language that should be passed to the LLM.
Properties
type * string
content * string
actions_server_url string
The URL of the actions server that should be used for the rails.
sample_conversation string
The sample conversation that should be used inside the prompts.
Default: user "Hello there!" express greeting bot express greeting "Hello! How can I assist you today?" user "What can you do for me?" ask about capabilities bot respond about capabilities "As an AI assistant, I can help you with a wide range of tasks. This includes question answering on various topics, generating text for various purposes and providing suggestions based on your preferences." user "Tell me a bit about the history of NVIDIA." ask general question bot response for general question "NVIDIA is a technology company that specializes in designing and manufacturing graphics processing units (GPUs) and other computer hardware. The company was founded in 1993 by Jen-Hsun Huang, Chris Malachowsky, and Curtis Priem." user "tell me more" request more information bot provide more information "Initially, the company focused on developing 3D graphics processing technology for the PC gaming market. In 1999, NVIDIA released the GeForce 256, the world's first GPU, which was a major breakthrough for the gaming industry. The company continued to innovate in the GPU space, releasing new products and expanding into other markets such as professional graphics, mobile devices, and artificial intelligence." user "thanks" express appreciation bot express appreciation and offer additional help "You're welcome. If you have any more questions or if there's anything else I can help you with, please don't hesitate to ask."
prompts array
The prompts that should be used for the various LLM tasks.
Array items:
item object
Configuration for prompts that will be used for a specific task.
Properties
task * string
The id of the task associated with this prompt.
content string
The content of the prompt, if it's a string.
messages array
The list of messages included in the prompt. Used for chat models.
Array items:
item object | string
Any of:
Option 1: object - Template for a message structure.
Option 2: string
models array
If specified, the prompt will be used only for the given LLM engines/models. The format is a list of strings with the format: or /.
Array items:
item string
output_parser string
The name of the output parser to use for this prompt.
max_length integer
The maximum length of the prompt in number of characters.
Default: 16000
mode string
Corresponds to the `prompting_mode` for which this prompt is fetched. Default is 'standard'.
Default: standard
stop array
If specified, will be configure stop tokens for models that support this.
Array items:
item string
max_tokens integer
The maximum number of tokens that can be generated in the chat completion.
prompting_mode string
Allows choosing between different prompting strategies.
Default: standard
lowest_temperature number
The lowest temperature that should be used for the LLM.
Default: 0.001
enable_multi_step_generation boolean
Whether to enable multi-step generation for the LLM.
Default: False
colang_version string
The Colang version to use.
Default: 1.0
custom_data object
Any custom configuration data that might be needed.
Allows additional properties: Yes
rails object
Configuration for the various rails (input, output, etc.).
Allows additional properties: No
Properties
config object
Configuration data for specific rails that are supported out-of-the-box.
Properties
fact_checking object
Configuration data for the fact-checking rail.
Properties
parameters object
Allows additional properties: Yes
fallback_to_self_check boolean
Whether to fall back to self-check if another method fail.
Default: False
autoalign object
Configuration data for the AutoAlign guardrails API.
Properties
parameters object
Allows additional properties: Yes
input object
Input configuration for AutoAlign guardrails
Properties
guardrails_config object
The guardrails configuration that is passed to the AutoAlign endpoint
Allows additional properties: Yes
output object
Output configuration for AutoAlign guardrails
Properties
guardrails_config object
The guardrails configuration that is passed to the AutoAlign endpoint
Allows additional properties: Yes
patronus object
Configuration data for the Patronus Evaluate API.
Properties
input object
Patronus Evaluate API configuration for an Input Guardrail
Properties
evaluate_config object
Configuration passed to the Patronus Evaluate API
Properties
success_strategy string
Strategy to determine whether the Patronus Evaluate API Guardrail passes or not.
Allowed values:
all_passany_pass
Default: all_pass
params object
Parameters to the Patronus Evaluate API
Allows additional properties: Yes
output object
Patronus Evaluate API configuration for an Output Guardrail
Properties
evaluate_config object
Configuration passed to the Patronus Evaluate API
Properties
success_strategy string
Strategy to determine whether the Patronus Evaluate API Guardrail passes or not.
Allowed values:
all_passany_pass
Default: all_pass
params object
Parameters to the Patronus Evaluate API
Allows additional properties: Yes
sensitive_data_detection object
Configuration for detecting sensitive data.
Properties
recognizers array
Additional custom recognizers. Check out https://microsoft.github.io/presidio/tutorial/08_no_code/ for more details.
Array items:
item object
Allows additional properties: Yes
input object
Configuration of the entities to be detected on the user input.
Properties
entities array
The list of entities that should be detected. Check out https://microsoft.github.io/presidio/supported_entities/ forthe list of supported entities.
Array items:
item string
mask_token string
The token that should be used to mask the sensitive data.
Default: *
score_threshold number
The score threshold that should be used to detect the sensitive data.
Default: 0.2
output object
Configuration of the entities to be detected on the bot output.
Properties
entities array
The list of entities that should be detected. Check out https://microsoft.github.io/presidio/supported_entities/ forthe list of supported entities.
Array items:
item string
mask_token string
The token that should be used to mask the sensitive data.
Default: *
score_threshold number
The score threshold that should be used to detect the sensitive data.
Default: 0.2
retrieval object
Configuration of the entities to be detected on retrieved relevant chunks.
Properties
entities array
The list of entities that should be detected. Check out https://microsoft.github.io/presidio/supported_entities/ forthe list of supported entities.
Array items:
item string
mask_token string
The token that should be used to mask the sensitive data.
Default: *
score_threshold number
The score threshold that should be used to detect the sensitive data.
Default: 0.2
jailbreak_detection object
Configuration for jailbreak detection.
Properties
server_endpoint string
The endpoint for the jailbreak detection heuristics/model container.
length_per_perplexity_threshold number
The length/perplexity threshold.
Default: 89.79
prefix_suffix_perplexity_threshold number
The prefix/suffix perplexity threshold.
Default: 1845.65
nim_base_url string
Base URL for jailbreak detection model. Example: http://localhost:8000/v1
nim_server_endpoint string
Classification path uri. Defaults to 'classify' for NemoGuard JailbreakDetect.
Default: classify
api_key string
Secret String with API key for use in Jailbreak requests. Takes precedence over api_key_env_var
api_key_env_var string
Environment variable containing API key for jailbreak detection model
nim_url string
DEPRECATED: Use nim_base_url instead
nim_port integer
DEPRECATED: Include port in nim_base_url instead
embedding string
injection_detection object
Configuration for injection detection.
Properties
injections array
The list of injection types to detect. Options are 'sqli', 'template', 'code', 'xss'.Currently, only SQL injection, template injection, code injection, and markdown cross-site scripting are supported. Custom rules can be added, provided they are in the `yara_path` and have a `.yara` file extension.
Array items:
item string
action string
Action to take. Options are 'reject' to offer a rejection message, 'omit' to mask the offending content, and 'sanitize' to pass the content as-is in the safest way. These options are listed in descending order of relative safety. 'sanitize' is not implemented at this time.
Constraints: pattern: ^(reject|omit)$
Default: reject
yara_rules object
Dictionary mapping rule names to YARA rule strings. If provided, these rules will be used instead of loading rules from yara_path. Each rule should be a valid YARA rule string.
Additional properties schema:
[key: string] string
privateai object
Configuration for Private AI.
Properties
server_endpoint string
The endpoint for the private AI detection server.
Default: http://localhost:8080/process/text
input object
Configuration of the entities to be detected on the user input.
Properties
entities array
The list of entities that should be detected.
Array items:
item string
output object
Configuration of the entities to be detected on the bot output.
Properties
entities array
The list of entities that should be detected.
Array items:
item string
retrieval object
Configuration of the entities to be detected on retrieved relevant chunks.
Properties
entities array
The list of entities that should be detected.
Array items:
item string
fiddler object
Configuration for Fiddler Guardrails.
Properties
fiddler_endpoint string
The global endpoint for Fiddler Guardrails requests.
Default: http://localhost:8080/process/text
safety_threshold number
Fiddler Guardrails safety detection threshold.
Default: 0.1
faithfulness_threshold number
Fiddler Guardrails faithfulness detection threshold.
Default: 0.05
clavata object
Configuration for Clavata.
Properties
server_endpoint string
The endpoint for the Clavata API
Default: https://gateway.app.clavata.ai:8443
policies object
A dictionary of policy aliases and their corresponding IDs.
Additional properties schema:
[key: string] string
label_match_logic string
The logic to use when deciding whether the evaluation matched. If ANY, only one of the configured labels needs to be found in the input or output. If ALL, all configured labels must be found in the input or output.
Allowed values:
ANYALL
Default: ANY
input object
Clavata configuration for an Input Guardrail
Properties
policy * string
The policy alias to use when evaluating inputs or outputs.
labels array
A list of labels to match against the policy. If no labels are provided, the overall policy result will be returned. If labels are provided, only hits on the provided labels will be considered a hit.
Array items:
item string
output object
Clavata configuration for an Output Guardrail
Properties
policy * string
The policy alias to use when evaluating inputs or outputs.
labels array
A list of labels to match against the policy. If no labels are provided, the overall policy result will be returned. If labels are provided, only hits on the provided labels will be considered a hit.
Array items:
item string
pangea object
Configuration for Pangea.
Properties
input object
Pangea configuration for an Input Guardrail
Properties
recipe * string
Recipe key of a configuration of data types and settings defined in the Pangea User Console. It specifies the rules that are to be applied to the text, such as defang malicious URLs.
output object
Pangea configuration for an Output Guardrail
Properties
recipe * string
Recipe key of a configuration of data types and settings defined in the Pangea User Console. It specifies the rules that are to be applied to the text, such as defang malicious URLs.
guardrails_ai object
Configuration for Guardrails AI validators.
Properties
validators array
List of Guardrails AI validators to apply. Each validator can have its own parameters and metadata.
Array items:
item object
Configuration for a single Guardrails AI validator.
Properties
name * string
Unique identifier or import path for the Guardrails AI validator (e.g., 'toxic_language', 'pii', 'regex_match', or 'guardrails/competitor_check').
parameters object
Parameters to pass to the validator during initialization (e.g., threshold, regex pattern).
Allows additional properties: Yes
metadata object
Metadata to pass to the validator during validation (e.g., valid_topics, context).
Allows additional properties: Yes
trend_micro object
Configuration for Trend Micro.
Properties
v1_url string
The endpoint for the Trend Micro AI Guard API
Default: https://api.xdr.trendmicro.com/beta/aiSecurity/guard
api_key_env_var string
Environment variable containing API key for Trend Micro AI Guard
input object
Configuration of the input rails.
Properties
parallel boolean
If True, the input rails are executed in parallel.
Default: False
flows array
The names of all the flows that implement input rails.
Array items:
item string
output object
Configuration of the output rails.
Properties
parallel boolean
If True, the output rails are executed in parallel.
Default: False
flows array
The names of all the flows that implement output rails.
Array items:
item string
streaming object
Configuration for streaming output rails.
Properties
enabled boolean
Enables streaming mode when True.
Default: False
chunk_size integer
The number of tokens in each processing chunk. This is the size of the token block on which output rails are applied.
Default: 200
context_size integer
The number of tokens carried over from the previous chunk to provide context for continuity in processing.
Default: 50
stream_first boolean
If True, token chunks are streamed immediately before output rails are applied.
Default: True
apply_to_reasoning_traces boolean
If True, output rails will apply guardrails to both reasoning traces and output response. If False, output rails will only apply guardrails to the output response excluding the reasoning traces, thus keeping reasoning traces unaltered.
Default: False
retrieval object
Configuration of the retrieval rails.
Properties
flows array
The names of all the flows that implement retrieval rails.
Array items:
item string
dialog object
Configuration of the dialog rails.
Properties
single_call object
Configuration for the single LLM call option.
Properties
enabled boolean
Default: False
fallback_to_multiple_calls boolean
Whether to fall back to multiple calls if a single call is not possible.
Default: True
user_messages object
Configuration for how the user messages are interpreted.
Properties
embeddings_only boolean
Whether to use only embeddings for computing the user canonical form messages.
Default: False
embeddings_only_similarity_threshold number
The similarity threshold to use when using only embeddings for computing the user canonical form messages.
embeddings_only_fallback_intent string
Defines the fallback intent when the similarity is below the threshold. If set to None, the user intent is computed normally using the LLM. If set to a string value, that string is used as the intent.
actions object
Configuration of action rails.
Properties
instant_actions array
The names of all actions which should finish instantly.
Array items:
item string
tool_output object
Configuration of tool output rails.
Properties
flows array
The names of all the flows that implement tool output rails.
Array items:
item string
parallel boolean
If True, the tool output rails are executed in parallel.
Default: False
tool_input object
Configuration of tool input rails.
Properties
flows array
The names of all the flows that implement tool input rails.
Array items:
item string
parallel boolean
If True, the tool input rails are executed in parallel.
Default: False
enable_rails_exceptions boolean
If set, the pre-defined guardrails raise exceptions instead of returning pre-defined messages.
Default: False
passthrough boolean
Weather the original prompt should pass through the guardrails configuration as is. This means it will not be altered in any way.
tracing object
Configuration for tracing.
Properties
enabled boolean
Default: False
adapters array
The list of tracing adapters to use. If not specified, the default adapters are used.
Array items:
item object
Allows additional properties: Yes
Properties
name string
The name of the adapter.
Default: FileSystem
span_format string
The span format to use. Options are 'legacy' (simple metrics) or 'opentelemetry' (OpenTelemetry semantic conventions).
Default: opentelemetry
enable_content_capture boolean
Capture prompts and responses (user/assistant/tool message content) in tracing/telemetry events. Disabled by default for privacy and alignment with OpenTelemetry GenAI semantic conventions. WARNING: Enabling this may include PII and sensitive data in your telemetry backend.
Default: False
files_url string
The location where the artifact files are stored.
model_providers array
List of ModelProvider namespace/name resource names that provide inference for this Model Entity
Array items:
item string
project string
The URN of the project associated with this entity.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string

RAGTargetInput#

Define an inline RAG pipeline configuration in the rag field when creating a target with type: "rag".

RAGTargetInput object
Properties
pipeline object
RAG pipeline.
Properties
retriever * object
The retriever pipeline included in the RAG pipeline.
Properties
pipeline object
Retriever pipeline run by Evaluator.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10
cached_outputs object
Cached pre-generated retriever outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated retriever outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
model * string | object
The generation model for the RAG pipeline.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
context_ordering string
The context ordering for the RAG pipeline.
Default: desc
cached_outputs object
Cached pre-generated RAG outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated RAG outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.

RetrieverTargetInput#

Define an inline retriever configuration in the retriever field when creating a target with type: "retriever".

RetrieverTargetInput object
Properties
pipeline object
Retriever pipeline run by Evaluator.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10
cached_outputs object
Cached pre-generated retriever outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated retriever outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.

Pipeline Configuration Schemas#

These schemas define pipeline-specific settings used within RAG and retriever target configurations.

RAGPipelineDataInput#

Nested pipeline configuration schema used within RAGTargetInput. Defines retrieval and generation settings for RAG pipelines.

RAGPipelineDataInput object
Data for evaluating a RAG pipeline.
Properties
retriever * object
The retriever pipeline included in the RAG pipeline.
Properties
pipeline object
Retriever pipeline run by Evaluator.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10
cached_outputs object
Cached pre-generated retriever outputs.
Properties
files_url * string
The files URL of the cached outputs.
Constraints: min length: 1
dataset object
Cached pre-generated retriever outputs dataset.
Properties
schema_version string
The version of the schema for the object. Internal use only.
Default: 1.0
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
description string
The description of the entity.
type_prefix string
The type prefix of the entity ID. If not specified, it will be inferred from the entity type name, but this will likely result in long prefixes.
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
project string
The URN of the project associated with this entity.
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes
ownership object
Ownership information for the entity
Properties
created_by string
The ID of the user that created this entity.
Default:
updated_by string
The ID of the user that last updated this entity.
access_policies object
A general object for capturing access policies which can be used by an external service to determine ACLs
Default: {}
Additional properties schema:
[key: string] string
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
version_id string
A unique, immutable id for the version. This is similar to the commit hash.
Default: main
version_tags array
The list of version tags associated with this entity.
Array items:
item object
A version tag marks a particular version for an entity. The name must be unique across all version tags for the same entity name.
Properties
name * string
The name of the version. Typically should be kept lower case, no space, no '/', no `@`.
created_by string
The ID of the user that created this entity.
Default:
created_at string
Timestamp for when the version tag was created.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
model * string | object
The generation model for the RAG pipeline.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
context_ordering string
The context ordering for the RAG pipeline.
Default: desc

RetrieverPipelineDataInput#

Nested pipeline configuration schema used within RetrieverTargetInput. Defines embedding and reranking settings for retriever pipelines.

RetrieverPipelineDataInput object
Data for evaluating a retriever pipeline.
Properties
query_embedding_model * string | object
The query embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
index_embedding_model * string | object
The index embedding model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
reranker_model string | object
The reranker model.
Any of:
Option 1: string - A reference to ModelInputEV.
Option 2: object - Information about a machine learning model, typically an LLM.
top_k integer
The top k results to be used.
Default: 10

Data Source Schemas#

Dataset#

Define an inline dataset configuration in the dataset field when creating a target with type: "dataset".

Dataset object
A dataset that can be used for fine-tuning or evaluation.
Properties
created_at string
Timestamp for when the entity was created.
updated_at string
Timestamp for when the entity was last updated.
name string
The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
Constraints: max length: 255, pattern: ^[\w\-\+.@:]*$
Default:
namespace string
The namespace of the entity. This can be missing for namespace entities or in deployments that don't use namespaces.
Default: default
description string
The description of the entity.
format string
Specifies the dataset format, referring to the schema of the dataset rather than the file format. Examples include SQuAD, BEIR, etc.
files_url * string
The location where the artifact files are stored. This can be a URL pointing to NDS, Hugging Face, S3, or any other accessible resource location.
Constraints: min length: 1
hf_endpoint string
For HuggingFace URLs, the endpoint that should be used. By default, this is set to the Data Store URL. For HuggingFace Hub, this should be set to "https://huggingface.co".
split string
The split of the dataset. Examples include train, validation, test, etc.
limit integer
The maximum number of items to be used from the dataset.
id string
The ID of the entity. With the exception of namespaces, this is always a semantically-prefixed base58-encoded uuid4 [-base58(uuid4())].
project string
The URN of the project associated with this entity.
custom_fields object
A set of custom fields that the user can define and use for various purposes.
Allows additional properties: Yes