`nemo_microservices.resources.evaluation.targets`#

Module Contents#

Classes#

`AsyncTargetsResource`
`AsyncTargetsResourceWithRawResponse`
`AsyncTargetsResourceWithStreamingResponse`
`TargetsResource`
`TargetsResourceWithRawResponse`
`TargetsResourceWithStreamingResponse`

API#

class nemo_microservices.resources.evaluation.targets.AsyncTargetsResource( client: nemo_microservices._client.AsyncNeMoMicroservices, )#

Bases: nemo_microservices._resource.AsyncAPIResource

Initialization

async create( *, type: nemo_microservices.types.target_type.TargetType, cached_outputs: nemo_microservices.types.cached_outputs_data_param.CachedOutputsDataParam | nemo_microservices._types.Omit = omit, custom_fields: Dict[str, object] | nemo_microservices._types.Omit = omit, dataset: nemo_microservices.types.evaluation.target_create_params.Dataset | nemo_microservices._types.Omit = omit, description: str | nemo_microservices._types.Omit = omit, model: nemo_microservices.types.evaluation.target_create_params.Model | nemo_microservices._types.Omit = omit, name: str | nemo_microservices._types.Omit = omit, namespace: str | nemo_microservices._types.Omit = omit, ownership: nemo_microservices.types.shared_params.ownership.Ownership | nemo_microservices._types.Omit = omit, project: str | nemo_microservices._types.Omit = omit, rag: nemo_microservices.types.rag_target_param.RagTargetParam | nemo_microservices._types.Omit = omit, retriever: nemo_microservices.types.retriever_target_param.RetrieverTargetParam | nemo_microservices._types.Omit = omit, rows: Iterable[Dict[str, object]] | nemo_microservices._types.Omit = omit, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.evaluation_target.EvaluationTarget#

Create a new evaluation target.

Args: type: The type of the evaluation target, e.g., ‘model’, ‘retriever’, ‘rag’.

cached_outputs: An evaluation target which contains cached outputs.

custom_fields: A set of custom fields that the user can define and use for various purposes.

dataset: Dataset to be evaluated.

description: The description of the entity.

model: The model to be evaluated.

name: The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.

namespace: The namespace of the entity. This can be missing for namespace entities or in deployments that don’t use namespaces.

ownership: Information about ownership of an entity.

  If the entity is a namespace, the `access_policies` will typically apply to all
  entities inside the namespace.

project: The URN of the project associated with this entity.

rag: RAG to be evaluated.

retriever: Retriever to be evaluated.

rows: Rows to be evaluated.

extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

async delete( target_name: str, *, namespace: str, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.shared.delete_response.DeleteResponse#

Delete Evaluation Target

Args: extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

List available evaluation targets.

Args: filter: Filter targets on various criteria.

page: Page number.

page_size: Page size.

search: Search evaluation targets using substring matching. You can combine multiple search fields and filters.

  For example:

  - `?search[name]=llama-eval`: searches all targets with 'llama-eval' in the
    name.
  - `?search[type]=model`: searches all targets with 'model' in the type.
  - `?search[model]=llama`: searches all targets with 'llama' in the model field.
  - `?search[dataset]=validation`: searches all targets with 'validation' in the
    dataset field.
  - `?search[name]=llama-eval&search[type]=model`: searches all targets with
    'llama-eval' in the name AND 'model' in the type.
  - `?search[updated_at][start]=2024-01-01T00:00:00` finds all targets updated on
    or after the start date
  - `?search[created_at][start]=2022-01-01&search[updated_at][end]=2024-01-01`
    finds all targets created from start date up to and including end date

sort: The field to sort by. To sort in decreasing order, use - in front of the field name.

extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

async retrieve( target_name: str, *, namespace: str, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.evaluation_target.EvaluationTarget#

Get evaluation target info.

Args: extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

async update( target_name: str, *, namespace: str, cached_outputs: nemo_microservices.types.cached_outputs_data_param.CachedOutputsDataParam | nemo_microservices._types.Omit = omit, custom_fields: Dict[str, object] | nemo_microservices._types.Omit = omit, dataset: nemo_microservices.types.evaluation.target_update_params.Dataset | nemo_microservices._types.Omit = omit, description: str | nemo_microservices._types.Omit = omit, model: nemo_microservices.types.evaluation.target_update_params.Model | nemo_microservices._types.Omit = omit, ownership: nemo_microservices.types.shared_params.ownership.Ownership | nemo_microservices._types.Omit = omit, project: str | nemo_microservices._types.Omit = omit, rag: nemo_microservices.types.rag_target_param.RagTargetParam | nemo_microservices._types.Omit = omit, retriever: nemo_microservices.types.retriever_target_param.RetrieverTargetParam | nemo_microservices._types.Omit = omit, rows: Iterable[Dict[str, object]] | nemo_microservices._types.Omit = omit, type: nemo_microservices.types.target_type.TargetType | nemo_microservices._types.Omit = omit, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.evaluation_target.EvaluationTarget#

Update an evaluation target.

Args: cached_outputs: An evaluation target which contains cached outputs.

custom_fields: A set of custom fields that the user can define and use for various purposes.

dataset: Dataset to be evaluated.

description: The description of the entity.

model: The model to be evaluated.

ownership: Information about ownership of an entity.

  If the entity is a namespace, the `access_policies` will typically apply to all
  entities inside the namespace.

project: The URN of the project associated with this entity.

rag: RAG to be evaluated.

retriever: Retriever to be evaluated.

rows: Rows to be evaluated.

type: The type of the evaluation target, e.g., ‘model’, ‘retriever’, ‘rag’.

extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

property with_raw_response: nemo_microservices.resources.evaluation.targets.AsyncTargetsResourceWithRawResponse#

This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#accessing-raw-response-data-e-g-headers

property with_streaming_response: nemo_microservices.resources.evaluation.targets.AsyncTargetsResourceWithStreamingResponse#

An alternative to .with_raw_response that doesn’t eagerly read the response body.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#with_streaming_response

class nemo_microservices.resources.evaluation.targets.AsyncTargetsResourceWithRawResponse( targets: nemo_microservices.resources.evaluation.targets.AsyncTargetsResource, )#: Initialization

class nemo_microservices.resources.evaluation.targets.AsyncTargetsResourceWithStreamingResponse( targets: nemo_microservices.resources.evaluation.targets.AsyncTargetsResource, )#: Initialization

class nemo_microservices.resources.evaluation.targets.TargetsResource(client: nemo_microservices._client.NeMoMicroservices)#

Bases: nemo_microservices._resource.SyncAPIResource

Initialization

create( *, type: nemo_microservices.types.target_type.TargetType, cached_outputs: nemo_microservices.types.cached_outputs_data_param.CachedOutputsDataParam | nemo_microservices._types.Omit = omit, custom_fields: Dict[str, object] | nemo_microservices._types.Omit = omit, dataset: nemo_microservices.types.evaluation.target_create_params.Dataset | nemo_microservices._types.Omit = omit, description: str | nemo_microservices._types.Omit = omit, model: nemo_microservices.types.evaluation.target_create_params.Model | nemo_microservices._types.Omit = omit, name: str | nemo_microservices._types.Omit = omit, namespace: str | nemo_microservices._types.Omit = omit, ownership: nemo_microservices.types.shared_params.ownership.Ownership | nemo_microservices._types.Omit = omit, project: str | nemo_microservices._types.Omit = omit, rag: nemo_microservices.types.rag_target_param.RagTargetParam | nemo_microservices._types.Omit = omit, retriever: nemo_microservices.types.retriever_target_param.RetrieverTargetParam | nemo_microservices._types.Omit = omit, rows: Iterable[Dict[str, object]] | nemo_microservices._types.Omit = omit, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.evaluation_target.EvaluationTarget#

Create a new evaluation target.

Args: type: The type of the evaluation target, e.g., ‘model’, ‘retriever’, ‘rag’.

cached_outputs: An evaluation target which contains cached outputs.

custom_fields: A set of custom fields that the user can define and use for various purposes.

dataset: Dataset to be evaluated.

description: The description of the entity.

model: The model to be evaluated.

name: The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.

namespace: The namespace of the entity. This can be missing for namespace entities or in deployments that don’t use namespaces.

ownership: Information about ownership of an entity.

  If the entity is a namespace, the `access_policies` will typically apply to all
  entities inside the namespace.

project: The URN of the project associated with this entity.

rag: RAG to be evaluated.

retriever: Retriever to be evaluated.

rows: Rows to be evaluated.

extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

delete( target_name: str, *, namespace: str, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.shared.delete_response.DeleteResponse#

Delete Evaluation Target

Args: extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

List available evaluation targets.

Args: filter: Filter targets on various criteria.

page: Page number.

page_size: Page size.

search: Search evaluation targets using substring matching. You can combine multiple search fields and filters.

  For example:

  - `?search[name]=llama-eval`: searches all targets with 'llama-eval' in the
    name.
  - `?search[type]=model`: searches all targets with 'model' in the type.
  - `?search[model]=llama`: searches all targets with 'llama' in the model field.
  - `?search[dataset]=validation`: searches all targets with 'validation' in the
    dataset field.
  - `?search[name]=llama-eval&search[type]=model`: searches all targets with
    'llama-eval' in the name AND 'model' in the type.
  - `?search[updated_at][start]=2024-01-01T00:00:00` finds all targets updated on
    or after the start date
  - `?search[created_at][start]=2022-01-01&search[updated_at][end]=2024-01-01`
    finds all targets created from start date up to and including end date

sort: The field to sort by. To sort in decreasing order, use - in front of the field name.

extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

retrieve( target_name: str, *, namespace: str, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.evaluation_target.EvaluationTarget#

Get evaluation target info.

Args: extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

update( target_name: str, *, namespace: str, cached_outputs: nemo_microservices.types.cached_outputs_data_param.CachedOutputsDataParam | nemo_microservices._types.Omit = omit, custom_fields: Dict[str, object] | nemo_microservices._types.Omit = omit, dataset: nemo_microservices.types.evaluation.target_update_params.Dataset | nemo_microservices._types.Omit = omit, description: str | nemo_microservices._types.Omit = omit, model: nemo_microservices.types.evaluation.target_update_params.Model | nemo_microservices._types.Omit = omit, ownership: nemo_microservices.types.shared_params.ownership.Ownership | nemo_microservices._types.Omit = omit, project: str | nemo_microservices._types.Omit = omit, rag: nemo_microservices.types.rag_target_param.RagTargetParam | nemo_microservices._types.Omit = omit, retriever: nemo_microservices.types.retriever_target_param.RetrieverTargetParam | nemo_microservices._types.Omit = omit, rows: Iterable[Dict[str, object]] | nemo_microservices._types.Omit = omit, type: nemo_microservices.types.target_type.TargetType | nemo_microservices._types.Omit = omit, extra_headers: nemo_microservices._types.Headers | None = None, extra_query: nemo_microservices._types.Query | None = None, extra_body: nemo_microservices._types.Body | None = None, timeout: float | httpx.Timeout | None | nemo_microservices._types.NotGiven = not_given, ) → nemo_microservices.types.evaluation_target.EvaluationTarget#

Update an evaluation target.

Args: cached_outputs: An evaluation target which contains cached outputs.

custom_fields: A set of custom fields that the user can define and use for various purposes.

dataset: Dataset to be evaluated.

description: The description of the entity.

model: The model to be evaluated.

ownership: Information about ownership of an entity.

  If the entity is a namespace, the `access_policies` will typically apply to all
  entities inside the namespace.

project: The URN of the project associated with this entity.

rag: RAG to be evaluated.

retriever: Retriever to be evaluated.

rows: Rows to be evaluated.

type: The type of the evaluation target, e.g., ‘model’, ‘retriever’, ‘rag’.

extra_headers: Send extra headers

extra_query: Add additional query parameters to the request

extra_body: Add additional JSON properties to the request

timeout: Override the client-level default timeout for this request, in seconds

property with_raw_response: nemo_microservices.resources.evaluation.targets.TargetsResourceWithRawResponse#

This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#accessing-raw-response-data-e-g-headers

property with_streaming_response: nemo_microservices.resources.evaluation.targets.TargetsResourceWithStreamingResponse#

An alternative to .with_raw_response that doesn’t eagerly read the response body.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#with_streaming_response

class nemo_microservices.resources.evaluation.targets.TargetsResourceWithRawResponse( targets: nemo_microservices.resources.evaluation.targets.TargetsResource, )#: Initialization

class nemo_microservices.resources.evaluation.targets.TargetsResourceWithStreamingResponse( targets: nemo_microservices.resources.evaluation.targets.TargetsResource, )#: Initialization

nemo_microservices.resources.evaluation.targets#

Module Contents#

Classes#

API#

`nemo_microservices.resources.evaluation.targets`#