Skip to navigation

Metrics Flow

View as Markdown
Parsed Inference Results(Load Balanced) MetricRecordProcessor(Distributed instance 1) MetricRecordProcessor(Distributed instance 2) MetricRecordProcessor(Distributed instance N) RECORD: RequestLatencyMetricparse_record() → 125ms(Individual value per request) RECORD: RequestLatencyMetricparse_record() → 87ms(Individual value per request) RECORD: RequestLatencyMetricparse_record() → 203ms(Individual value per request) AGGREGATE: TotalRequestsMetricparse_record() → +1(Individual contribution) AGGREGATE: TotalRequestsMetricparse_record() → +1(Individual contribution) AGGREGATE: TotalRequestsMetricparse_record() → +1(Individual contribution) MetricRecordDict(Per-record results) MetricRecordDict(Per-record results) MetricRecordDict(Per-record results) RecordsManager → MetricsAccumulator(Single centralized instance) RECORD Collectionappend(125ms)append(87ms)append(203ms)(Collect all individual values) AGGREGATE Columnappend(1) append(1) append(1)fold via aggregation_kind (SUM) → 3(Numpy column folded in MetricsAccumulator) MetricResultsDict(Full profile run results) Summarize Functionsummarize()(Process all collected results) RECORD Statisticsp50=125ms, p95=203msmean=138ms, std=58msmin=87ms, max=203ms(Full statistical analysis) AGGREGATE Resultsfinal_value=3count=1(Single accumulated total) DERIVED: ThroughputMetricderive_value(results)= total_requests / duration= 3 / 5.2s = 0.58 req/s(Computed from other metrics) MetricResult ListComplete performance analysis