gRPC Function Invocation

gRPC invocation executes requests against Cloud Functions functions that expose a gRPC service. gRPC functions use the gRPC proxy instead of the HTTP invocation route.

In self-hosted deployments, the gRPC route is exposed on the Gateway TCP listener. See Gateway Routing for listener and DNS configuration.

Invocation Path

$
export GRPC_GATEWAY_ADDR=<grpc-gateway-address>
$
export FUNCTION_ID=<function-id>
$
export FUNCTION_VERSION_ID=<function-version-id>
$
export API_KEY=<api-key>

Multi-Cluster View

In a global deployment, DNS selects a regional public gRPC endpoint. Each region keeps its own gRPC Proxy, NVCF API and NATS stateful request path, worker CONNECT registration, and customer gRPC service placement. The cross-cluster line shows NATS chatter for regional stateful request-path coordination when configured.

Metadata

Set these gRPC metadata values when invoking a function:

The data sent to your gRPC function is defined by the Protobuf messages your function implements. gRPC functions do not have an input request size limit.

gRPC connections stay alive for 30 seconds when idle. Close the gRPC client connection after your client is finished so function workers are not held longer than needed.

Python Example

This example uses a plaintext local or test gateway on port 10081. For a production TLS endpoint, use grpc.secure_channel("grpc.<domain>:443", grpc.ssl_channel_credentials()).

1
import os
2
import grpc
3
4
import grpc_service_pb2_grpc
5
6
7
def call_grpc(model_infer_request) -> None:
8
    channel = grpc.insecure_channel(f"{os.environ['GRPC_GATEWAY_ADDR']}:10081")
9
    grpc_client = grpc_service_pb2_grpc.GRPCInferenceServiceStub(channel)
10
11
    metadata = [
12
        ("function-id", os.environ["FUNCTION_ID"]),
13
        ("function-version-id", os.environ["FUNCTION_VERSION_ID"]),
14
        ("authorization", f"Bearer {os.environ['API_KEY']}"),
15
    ]
16
17
    infer = grpc_client.ModelInfer(model_infer_request, metadata=metadata)
18
    _ = infer
19
20
    channel.close()

Connection Reuse and Streaming

The gRPC proxy pins sessions to the TCP connection to support unmodified gRPC clients that ignore cookie headers. This matters when an intermediary proxy for streaming, such as Kit streaming or Low Latency Streaming (LLS), uses HTTP/2 and reuses connections.

For requirements and a sample intermediary proxy implementation, see Intermediary Proxy.