aitune.torch.task.correctness | NVIDIA AITune Documentation

Correctness checking utilities.

Module Contents

Classes

Name	Description
`CorrectnessDynamicShapeError`	Error raised when dynamic-shape boundary inference fails.
`CorrectnessTensorShapeError`	Error raised when tensor shapes do not match.
`CorrectnessValueError`	Error raised when value is not finite i.e. NaN or infinity.

Functions

Name	Description
`_check_output_correctness`	Check if model outputs contain any NaN or infinity values.
`_check_output_tensor_shapes`	Check if the output tensor shapes are the same as the original output tensor shapes.
`_dynamic_shape_boundary_samples`	Create min/max input samples for dynamic-shape correctness checks.
`_resize_sample_to_shape`	Resize each tensor input in a sample to the requested TensorSpec shape attribute.
`_resize_tensor`	Resize a tensor by slicing or repeating each axis to match a target shape.
`check_dynamic_shape_boundary_inference`	Check dynamic-shape boundary inference.
`check_inference_output_correctness`	Check recorded-sample inference output correctness.

API

class aitune.torch.task.correctness.CorrectnessDynamicShapeError()

Bases: ValueError

Error raised when dynamic-shape boundary inference fails.

class aitune.torch.task.correctness.CorrectnessTensorShapeError()

Bases: ValueError

Error raised when tensor shapes do not match.

class aitune.torch.task.correctness.CorrectnessValueError()

Bases: ValueError

Error raised when value is not finite i.e. NaN or infinity.

aitune.torch.task.correctness._check_output_correctness(
    output: typing.Any,
    name: str = 'output',
    depth: int = 0
)

Check if model outputs contain any NaN or infinity values.

Parameters:

output

Any

Model outputs. Dict, list, tensors or scalars. Strings are ignored.

name

strDefaults to 'output'

Name of the output. If not provided, the name will be inferred from the output type.

depth

intDefaults to 0

Depth of the output. Auxiliary variable for recursive calls.

Raises:

CorrectnessCheckError: If any output contains NaN or infinity values
ValueError: If output does not contain tensors or scalars

aitune.torch.task.correctness._check_output_tensor_shapes(
    expected: aitune.torch.module.sample_metadata.SampleMetadata,
    actual: aitune.torch.module.sample_metadata.SampleMetadata
)

Check if the output tensor shapes are the same as the original output tensor shapes.

aitune.torch.task.correctness._dynamic_shape_boundary_samples(
    sample: tuple[tuple, dict],
    graph_spec: aitune.torch.module.graph_spec.GraphSpec
) -> list[tuple[tuple, dict]]

Create min/max input samples for dynamic-shape correctness checks.

Note: These samples are intended only for validating min and max shapes. Their tensor values may be numerically invalid and can produce NaN or Inf outputs.

aitune.torch.task.correctness._resize_sample_to_shape(
    sample: tuple[tuple, dict],
    graph_spec: aitune.torch.module.graph_spec.GraphSpec,
    shape_attr: str
) -> tuple[tuple, dict]

Resize each tensor input in a sample to the requested TensorSpec shape attribute.

Note: These samples are intended only for validating min and max shapes. Their tensor values may be numerically invalid and can produce NaN or Inf outputs.

aitune.torch.task.correctness._resize_tensor(
    tensor: torch.Tensor,
    target_shape: list[int]
) -> torch.Tensor

Resize a tensor by slicing or repeating each axis to match a target shape.

Note: These samples are intended only for validating min and max shapes. Their tensor values may be numerically invalid and can produce NaN or Inf outputs.

aitune.torch.task.correctness.check_dynamic_shape_boundary_inference(
    sample: tuple[tuple, dict],
    graph_spec: aitune.torch.module.graph_spec.GraphSpec,
    infer: collections.abc.Callable[..., typing.Any],
    name: str
)

Check dynamic-shape boundary inference.

Parameters:

sample

tuple[tuple, dict]

Sample inputs used as a template for min and max shape checks.

graph_spec

GraphSpec

Recorded graph metadata used to construct and validate boundary inputs.

infer

Callable[..., Any]

Inference callable to validate.

name

str

Name of the module or backend being checked.

Returns:

None.

Raises:

CorrectnessDynamicShapeError: If inference fails on min or max shape inputs.
CorrectnessTensorShapeError: If inferred output tensor shapes do not match the expected metadata.

aitune.torch.task.correctness.check_inference_output_correctness(
    data: list[tuple[tuple, dict]],
    output_spec: aitune.torch.module.sample_metadata.SampleMetadata,
    infer: collections.abc.Callable[..., typing.Any],
    name: str
)

Check recorded-sample inference output correctness.

Parameters:

data

list[tuple[tuple, dict]]

Recorded samples to run through inference.

output_spec

SampleMetadata

Recorded output metadata with expected output tensor shapes.

infer

Callable[..., Any]

Inference callable to validate.

name

str

Name of the module or backend being checked.

Returns:

None.

Raises:

CorrectnessValueError: If any output value is NaN or infinite.
CorrectnessTensorShapeError: If inferred output tensor shapes do not match the expected metadata.