Evaluation Configs Resource#

class nemo_microservices.resources.evaluation.ConfigsResource(client: NeMoMicroservices)#

Bases: SyncAPIResource

property with_raw_response: ConfigsResourceWithRawResponse#

This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#accessing-raw-response-data-e-g-headers

property with_streaming_response: ConfigsResourceWithStreamingResponse#

An alternative to .with_raw_response that doesn’t eagerly read the response body.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#with_streaming_response

Create a new evaluation config.

Parameters:

type – The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to custom.
custom_fields – A set of custom fields that the user can define and use for various purposes.
description – The description of the entity.
groups – Evaluation tasks belonging to the evaluation.
name – The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
namespace – The namespace of the entity. This can be missing for namespace entities or in deployments that don’t use namespaces.
ownership –
Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.
params – Global parameters for an evaluation.
project – The URN of the project associated with this entity.
tasks – Evaluation tasks belonging to the evaluation.
extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

Get evaluation config info.

Parameters:

extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

Update the evaluation config.

Parameters:

custom_fields – A set of custom fields that the user can define and use for various purposes.
description – The description of the entity.
groups – Evaluation tasks belonging to the evaluation.
name – The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
body_namespace – The namespace of the entity. This can be missing for namespace entities or in deployments that don’t use namespaces.
ownership –
Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.
params – Global parameters for an evaluation.
project – The URN of the project associated with this entity.
tasks – Evaluation tasks belonging to the evaluation.
type – The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to custom.
extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

List available evaluation configs.

Parameters:

filter – Filter configs on various criteria.
page – Page number.
page_size – Page size.
sort – The field to sort by. To sort in decreasing order, use - in front of the field name.
extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

Delete Evaluation Config

Parameters:

extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

create_from_dict(data: dict[str, object]) → object#

class nemo_microservices.resources.evaluation.AsyncConfigsResource(client: AsyncNeMoMicroservices)#

Bases: AsyncAPIResource

property with_raw_response: AsyncConfigsResourceWithRawResponse#

This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#accessing-raw-response-data-e-g-headers

property with_streaming_response: AsyncConfigsResourceWithStreamingResponse#

An alternative to .with_raw_response that doesn’t eagerly read the response body.

For more information, see https://docs.nvidia.com/nemo/microservices/latest/pysdk/index.html#with_streaming_response

Create a new evaluation config.

Parameters:

type – The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to custom.
custom_fields – A set of custom fields that the user can define and use for various purposes.
description – The description of the entity.
groups – Evaluation tasks belonging to the evaluation.
name – The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
namespace – The namespace of the entity. This can be missing for namespace entities or in deployments that don’t use namespaces.
ownership –
Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.
params – Global parameters for an evaluation.
project – The URN of the project associated with this entity.
tasks – Evaluation tasks belonging to the evaluation.
extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

Get evaluation config info.

Parameters:

extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

Update the evaluation config.

Parameters:

custom_fields – A set of custom fields that the user can define and use for various purposes.
description – The description of the entity.
groups – Evaluation tasks belonging to the evaluation.
name – The name of the entity. Must be unique inside the namespace. If not specified, it will be the same as the automatically generated id.
body_namespace – The namespace of the entity. This can be missing for namespace entities or in deployments that don’t use namespaces.
ownership –
Information about ownership of an entity.

If the entity is a namespace, the access_policies will typically apply to all entities inside the namespace.
params – Global parameters for an evaluation.
project – The URN of the project associated with this entity.
tasks – Evaluation tasks belonging to the evaluation.
type – The type of the evaluation, e.g., ‘mmlu’, ‘big_code’.For custom evaluations, this is set to custom.
extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

List available evaluation configs.

Parameters:

filter – Filter configs on various criteria.
page – Page number.
page_size – Page size.
sort – The field to sort by. To sort in decreasing order, use - in front of the field name.
extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

Delete Evaluation Config

Parameters:

extra_headers – Send extra headers
extra_query – Add additional query parameters to the request
extra_body – Add additional JSON properties to the request
timeout – Override the client-level default timeout for this request, in seconds

create_from_dict( data: dict[str, object], ) → object#