ReferencePython SDK ReferenceNemoguardrailsEval

nemoguardrails.eval.cli

View as Markdown

Module Contents

Functions

NameDescription
_launch_uiHelper to launch a Streamlit UI.
check_complianceCheck the policy compliance of the interactions in the output_path.
runRun the interactions for an evaluation.
uiLaunches the Evaluation UI.

Data

app

API

nemoguardrails.eval.cli._launch_ui(
    script: str,
    port: int = 8501
)

Helper to launch a Streamlit UI.

nemoguardrails.eval.cli.check_compliance(
    llm_judge: str = typer.Option(help='The name...,
    eval_config_path: str = typer.Option('config', '-e'...,
    output_path: typing.List[str] = typer.Option([], '-o', '--o...,
    policy_ids: typing.List[str] = typer.Option([], '-p', '--p...,
    verbose: bool = typer.Option(False, '-v', '...,
    force: bool = typer.Option(False, '-f', '...,
    disable_llm_cache: bool = typer.Option(default=False,...,
    reset: bool = typer.Option(default=False,...,
    parallel: int = typer.Option(1, '--parallel...
)

Check the policy compliance of the interactions in the output_path.

nemoguardrails.eval.cli.run(
    eval_config_path: str = typer.Option('config', '-e'...,
    guardrail_config_path: str = typer.Option(None, '-g', '-...,
    output_path: str = typer.Option('', '-o', '--o...,
    output_format: str = typer.Option(default='json'...,
    parallel: int = typer.Option(1, '--parallel...
)

Run the interactions for an evaluation.

nemoguardrails.eval.cli.ui(
    eval_config_path: str = typer.Option(default='confi...,
    output_path: typing.List[str] = typer.Option(default=[], he...
)

Launches the Evaluation UI.

nemoguardrails.eval.cli.app = typer.Typer()