Pipeline components

Short overview of ZAPP pipeline components and target wiring.

A ZAPP pipeline is a YAML map of named components. Each component declares a type, component-specific fields, and optional targets. Targets connect one component output to another component input.

Components usually fit one of three roles:

  • Source components receive or collect telemetry
  • Processing components convert or route telemetry
  • Terminal components forward telemetry to a service, network endpoint, or consumer API

Channels

Pipeline components pass data through typed channels. Source and processing components expose outputs with targets, and downstream components receive those outputs as inputs. During startup, ZAPP creates the channels, connects each target to the named component, and validates that the receiving component accepts that input type.

Each target can tune how the channel behaves during back pressure:

Setting or PatternBehavior
Blocking channelThe default behavior when leaky is omitted or set to false. The cap field sets the channel buffer capacity. If the buffer is full, the upstream component waits until the downstream component reads data.
Leaky channelEnabled with leaky: true. The cap field sets the channel buffer capacity. If the buffer is full, the item being sent is dropped instead of blocking upstream collection. Set dropCounter to record those drops as a Prometheus counter.
Fan-outSends one component output to multiple target channels. Each target keeps its own cap, leaky, and dropCounter settings.
Fan-inCombines multiple input channels of the same data type for a component that accepts more than one upstream source.

Use blocking channels when every telemetry item must be preserved and back pressure is acceptable. Use leaky channels for high-rate telemetry paths where keeping collection moving is more important than retaining every sample during a downstream slowdown.

For the telemetry data model shared by these components, see Time-series telemetry .


gRPC pipeline component

Configuration reference for the ZAPP grpcServer pipeline component.

Redfish listener pipeline

Configuration overview for Redfish listener pipelines and the redfishCollector component.

UDP listener and writer

Configuration reference for the ZAPP UDP listener and writer pipeline components.