nat.profiler.calc.data_models

Classes

FitConfig	Configuration parameters for linear fit and outlier detection.
CalcRunnerConfig	Parameters used for a calc runner.
SizingMetricPerItem	Sizing metrics per dataset entry item.
SizingMetricsAlerts	Sizing metrics alerts.
SizingMetrics	Sizing metrics for a single concurrency.
LinearFitResult	Result of linear regression including slope, intercept, and quality metrics.
FitResults	Linear fit results for both LLM latency and workflow runtime analysis.
GPUEstimates	GPU estimates.
CalcAlerts	Calc runner alerts.
CalcData	Output of the calc runner per concurrency.
CalcRunnerOutput	Output of the calc runner.

Module Contents

class FitConfig(/, **data: Any)

Bases: pydantic.BaseModel

Configuration parameters for linear fit and outlier detection.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

small_concurrency_range_threshold: int = 8

extreme_outlier_threshold: float = 2.0

conservative_outlier_threshold: float = 1.5

min_r_squared: float = 0.7

remove_outliers: bool = True

class CalcRunnerConfig(/, **data: Any)

Bases: pydantic.BaseModel

Parameters used for a calc runner.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

config_file: pathlib.Path | None = None

endpoint: str | None = None

endpoint_timeout: int = 300

offline_mode: bool = False

num_passes: int = 0

concurrencies: list[int] = [1, 2, 4, 8]

target_llm_latency_p95: float = 0

target_workflow_runtime_p95: float = 0

target_users: int = 0

test_gpu_count: int = 0

output_dir: pathlib.Path | None = None

append_job: bool = False

plot_data: bool = True

fit_config: FitConfig = None

class SizingMetricPerItem(/, **data: Any)

Bases: pydantic.BaseModel

Sizing metrics per dataset entry item.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

llm_latency: float

workflow_runtime: float

class SizingMetricsAlerts(/, **data: Any)

Bases: pydantic.BaseModel

Sizing metrics alerts.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

workflow_interrupted: bool = False

class SizingMetrics(/, **data: Any)

Bases: pydantic.BaseModel

Sizing metrics for a single concurrency.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

alerts: SizingMetricsAlerts = None

llm_latency_p95: float = 0.0

workflow_runtime_p95: float = 0.0

total_runtime: float = 0.0

per_item_metrics: dict[Any, SizingMetricPerItem]

class LinearFitResult(/, **data: Any)

Bases: pydantic.BaseModel

Result of linear regression including slope, intercept, and quality metrics.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

slope: float

intercept: float

r_squared: float

outliers_removed: list[int]

class FitResults(/, **data: Any)

Bases: pydantic.BaseModel

Linear fit results for both LLM latency and workflow runtime analysis.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

llm_latency_fit: LinearFitResult | None = None

wf_runtime_fit: LinearFitResult | None = None

class GPUEstimates(/, **data: Any)

Bases: pydantic.BaseModel

GPU estimates.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

gpu_estimate_by_wf_runtime: float | None = None

gpu_estimate_by_llm_latency: float | None = None

class CalcAlerts(/, **data: Any)

Bases: pydantic.BaseModel

Calc runner alerts.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

outlier_workflow_runtime: bool = False

outlier_llm_latency: bool = False

num_items_greater_than_target_latency: int = 0

num_items_greater_than_target_runtime: int = 0

class CalcData(/, **data: Any)

Bases: pydantic.BaseModel

Output of the calc runner per concurrency.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

gpu_estimates: GPUEstimates = None

alerts: CalcAlerts = None

sizing_metrics: SizingMetrics = None

class CalcRunnerOutput(/, **data: Any)

Bases: pydantic.BaseModel

Output of the calc runner.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

gpu_estimates: GPUEstimates = None

fit_results: FitResults = None

calc_data: dict[int, CalcData]