nemoguardrails.eval.utils

Module Contents

Name	Description
`_collect_span_metrics`	Collects and aggregates the metrics from all the spans.
`get_output_paths`	Helper to return the output paths from the current dir.
`load_dict_from_file`	Helper to load a dict from a file based on its extension.
`load_dict_from_path`	Load a dict from a path.
`save_dict_to_file`	Helper to write data to a file in the chosen format.
`save_eval_output`	Writes the evaluation output to a folder.
`update_dict_at_path`	Updates a dictionary at the specified path.

nemoguardrails.eval.utils._collect_span_metrics(
    spans: typing.List[Span]
) -> typing.Dict[str, typing.Union[int, float]]

Collects and aggregates the metrics from all the spans.

nemoguardrails.eval.utils.get_output_paths() -> typing.List[str]

Helper to return the output paths from the current dir.

nemoguardrails.eval.utils.load_dict_from_file(
    file_path: str
) -> typing.Dict[str, typing.Any]

Helper to load a dict from a file based on its extension.

nemoguardrails.eval.utils.load_dict_from_path(
    path: str
) -> typing.Dict[str, typing.Any]

Load a dict from a path.

It recursively loads all .json and .yaml files in the given path. Top level arrays are joined.

nemoguardrails.eval.utils.save_dict_to_file(
    val: typing.Any,
    output_path: str,
    output_format: str = 'yaml'
)

Helper to write data to a file in the chosen format.

nemoguardrails.eval.utils.save_eval_output(
    eval_output: EvalOutput,
    output_path: str,
    output_format: str = 'yaml'
)

Writes the evaluation output to a folder.

nemoguardrails.eval.utils.update_dict_at_path(
    path: str,
    d: dict
)

Updates a dictionary at the specified path.

It recursively looks at all .json and .yaml files in the given path. for every file that contains a top level key found in d, it is updated.