Function Autoscaling

Function Autoscaler Architecture

View as Markdown

The Function Autoscaler runs as a Kubernetes Deployment in the control-plane cluster. It reads metrics from a PromQL-compatible backend, stores coordination state in Cassandra, and writes desired instance counts to the NVCF API.

The work is split into two loops. A leader-elected discovery loop scans the timeseries database for active function versions and upserts them into Cassandra. A scaling loop runs on every replica, but each replica only handles the functions whose IDs hash into its assigned buckets, so the active set is sharded across replicas.

Sequence Diagram

The diagram shows the logical metrics flow and omits cluster boundaries.

The discovery loop runs on one leader-elected replica. The scaling loop runs on every replica, but each replica only processes its assigned function buckets.

Metrics backend

The autoscaler is a read-only client of a PromQL-compatible backend. It uses range queries to discover active functions and read instance, request, and utilization metrics.

The autoscaler does not scrape metrics. Self-hosted deployments use control-plane request metrics for function activity and utilization, and State Metrics for instance and concurrency data. These metrics must reach the backend that the autoscaler queries.

The backend can be bundled VictoriaMetrics or an existing PromQL-compatible service. See Observability Configuration for backend, endpoint, and authentication settings.

The autoscaler reports not ready until the query endpoint responds.

Coordination and Self-Healing

Coordination relies on Cassandra TTLs to recover from failures without operator intervention:

  • The discovery lock self-expires if the leader replica crashes or is partitioned, so a new leader takes over on the next loop iteration.
  • Bucket ownership is recomputed when replicas join or leave. During a reshuffle a function may be skipped for a single scaling cycle or briefly picked up by a different replica, and a short-lived per-function lock prevents two replicas from racing on the same function in that window.
  • Each active function row carries a TTL refreshed by every scaling cycle, so functions that stop emitting metrics age out of the active set automatically.

See Also