Frontend APIUtilities

Compile-time constant tensors

View as Markdown

Some scalars should be fixed when the graph is built so the compiler and backend can fold them (constants in the plan), rather than supplied on every execute() like ordinary pass-by-value tensors.

The graph API distinguishes this with ScalarType:

ScalarTypeRole
RUNTIME_PARAMPass-by-value scalar whose value can change each launch (pointer in variant pack).
COMPILE_TIME_CONSTScalar baked into the graph; not a separate runtime buffer for that value.

C++ API

Create tensors from a scalar and ScalarType:

std::shared_ptr<Tensor_attributes> tensor(float const& scalar, ScalarType scalar_type);
std::shared_ptr<Tensor_attributes> tensor(half const& scalar, ScalarType scalar_type);
// ... int32, int64, double, bfloat16 overloads

Under the hood, COMPILE_TIME_CONST sets the tensor’s compile-time constant payload (Tensor_attributes::set_compile_time_constant / get_has_compile_time_constant()). Such tensors must be pass-by-value, non-virtual, and must not also carry a runtime pass_by_value payload.

Python API

  • pygraph.tensor_scalar(value, scalar_type)
    • Overloads for floating and integer scalars; scalar_type is cudnn.scalar_type.RUNTIME_PARAM or cudnn.scalar_type.COMPILE_TIME_CONST.

You can still query flags on cudnn.tensor:

  • get_has_compile_time_constant() / set_compile_time_constant (advanced; prefer tensor_scalar for scalars).

Example

See samples/cpp/misc/compile_time_constant_example.cpp for a fused multiply-add using ScalarType::COMPILE_TIME_CONST.