Skip to navigation

Cagra

View as Markdown

Source header: cuvs/neighbors/cagra.h

cuvsCagraGraphBuildAlgo

Enum to denote which ANN algorithm is used to build CAGRA graph

enum cuvsCagraGraphBuildAlgo {
AUTO_SELECT = 0,
IVF_PQ = 1,
NN_DESCENT = 2,
ITERATIVE_CAGRA_SEARCH = 3,
ACE = 4
};

Values

NameValue
AUTO_SELECT0
IVF_PQ1
NN_DESCENT2
ITERATIVE_CAGRA_SEARCH3
ACE4

cuvsCagraHnswHeuristicType

A strategy for selecting the graph build parameters based on similar HNSW index parameters.

Define how cuvsCagraIndexParamsFromHnswParams should construct a graph to construct a graph that is to be converted to (used by) a CPU HNSW index.

enum cuvsCagraHnswHeuristicType {
CUVS_CAGRA_HEURISTIC_SIMILAR_SEARCH_PERFORMANCE = 0,
CUVS_CAGRA_HEURISTIC_SAME_GRAPH_FOOTPRINT = 1
};

Values

NameValue
CUVS_CAGRA_HEURISTIC_SIMILAR_SEARCH_PERFORMANCE0
CUVS_CAGRA_HEURISTIC_SAME_GRAPH_FOOTPRINT1

cuvsCagraCompressionParams

Parameters for PQ dataset compression.

The cuvsCagraCompressionParams name is retained for ABI compatibility and is planned for removal in the 27.02 ABI-breaking release. Use cuvsPqParams in new code.

struct cuvsCagraCompressionParams {
uint32_t pq_bits;
uint32_t pq_dim;
uint32_t vq_n_centers;
uint32_t kmeans_n_iters;
double vq_kmeans_trainset_fraction;
double pq_kmeans_trainset_fraction;
};

Fields

NameTypeDescription
pq_bitsuint32_tThe bit length of the vector element after compression by PQ.

Possible values: [4, 5, 6, 7, 8].

Hint: the smaller the pq_bits, the smaller the index size and the better the search performance, but the lower the recall.
pq_dimuint32_tThe dimensionality of the vector after compression by PQ. When zero, an optimal value is selected using a heuristic.

TODO: at the moment dim must be a multiple pq_dim.
vq_n_centersuint32_tVector Quantization (VQ) codebook size - number of “coarse cluster centers”. When zero, an optimal value is selected using a heuristic.
kmeans_n_itersuint32_tThe number of iterations searching for kmeans centers (both VQ & PQ phases).
vq_kmeans_trainset_fractiondoubleThe fraction of data to use during iterative kmeans building (VQ phase). When zero, an optimal value is selected using a heuristic.
pq_kmeans_trainset_fractiondoubleThe fraction of data to use during iterative kmeans building (PQ phase). When zero, an optimal value is selected using a heuristic.

cuvsAceParams

Parameters for ACE (Augmented Core Extraction) graph build.

ACE enables building indexes for datasets too large to fit in GPU memory by:

  1. Partitioning the dataset in core (closest) and augmented (second-closest) partitions using balanced k-means.
  2. Building sub-indexes for each partition independently
  3. Concatenating sub-graphs into a final unified index
struct cuvsAceParams {
size_t npartitions;
size_t ef_construction;
const char* build_dir;
bool use_disk;
double max_host_memory_gb;
double max_gpu_memory_gb;
};

Fields

NameTypeDescription
npartitionssize_tNumber of partitions for ACE (Augmented Core Extraction) partitioned build.

When set to 0 (default), the number of partitions is automatically derived based on available host and GPU memory to maximize partition size while ensuring the build fits in memory.

Small values might improve recall but potentially degrade performance and increase memory usage. Partitions should not be too small to prevent issues in KNN graph construction. The partition size is on average 2 * (n_rows / npartitions) * dim * sizeof(T). 2 is because of the core and augmented vectors. Please account for imbalance in the partition sizes (up to 3x in our tests).

If the specified number of partitions results in partitions that exceed available memory, the value will be automatically increased to fit memory constraints and a warning will be issued.
ef_constructionsize_tThe index quality for the ACE build.

Bigger values increase the index quality. At some point, increasing this will no longer improve the quality.
build_dirconst char*Directory to store ACE build artifacts (e.g., KNN graph, optimized graph).

Used when use_disk is true or when the graph does not fit in host and GPU memory. This should be the fastest disk in the system and hold enough space for twice the dataset, final graph, and label mapping. The directory may already exist, but ACE’s named artifacts must not already exist. Simultaneous builds must use different directories. On failure, ACE removes only artifacts it created and never deletes unrelated directory contents.
use_diskboolWhether to use disk-based storage for ACE build.

When true, enables disk-based operations for memory-efficient graph construction.
max_host_memory_gbdoubleMaximum host memory to use for ACE build in GiB.

When set to 0 (default), uses available host memory. When set to a positive value, limits host memory usage to the specified amount. Useful for testing or when running alongside other memory-intensive processes.
max_gpu_memory_gbdoubleMaximum GPU memory to use for ACE build in GiB.

When set to 0 (default), uses available GPU memory. When set to a positive value, limits GPU memory usage to the specified amount. Useful for testing or when running alongside other memory-intensive processes.

cuvsCagraIndexParams

Supplemental parameters to build CAGRA Index

struct cuvsCagraIndexParams {
cuvsDistanceType metric;
size_t intermediate_graph_degree;
size_t graph_degree;
enum cuvsCagraGraphBuildAlgo build_algo;
size_t nn_descent_niter;
void* graph_build_params;
};

Fields

NameTypeDescription
metriccuvsDistanceTypeDistance type.
intermediate_graph_degreesize_tDegree of input graph for pruning.
graph_degreesize_tDegree of output graph.
build_algoenum cuvsCagraGraphBuildAlgoANN algorithm to build knn graph.
nn_descent_nitersize_tNumber of Iterations to run if building with NN_DESCENT
graph_build_paramsvoid*Optional: specify graph build params based on build_algo
- IVF_PQ: cuvsIvfPqParams_t
- ACE: cuvsAceParams_t
- Others: nullptr

cuvsCagraMergeAlgo

Algorithm used to merge physical CAGRA indices.

enum cuvsCagraMergeAlgo {
CUVS_CAGRA_MERGE_AUTO = 0,
CUVS_CAGRA_MERGE_FASTENER = 1,
CUVS_CAGRA_MERGE_REBUILD = 2
};

Values

NameValue
CUVS_CAGRA_MERGE_AUTO0
CUVS_CAGRA_MERGE_FASTENER1
CUVS_CAGRA_MERGE_REBUILD2

cuvsCagraMergeParams

Parameters controlling how physical CAGRA indices are merged.

struct cuvsCagraMergeParams {
uint32_t levels;
uint32_t root_fanout;
uint32_t lower_fanout;
double leader_fraction;
uint32_t max_leaders;
uint32_t leaf_size;
uint32_t leaf_degree;
};

Fields

NameTypeDescription
levelsuint32_t
root_fanoutuint32_t
lower_fanoutuint32_t
leader_fractiondouble
max_leadersuint32_t
leaf_sizeuint32_t
leaf_degreeuint32_t

cuvsCagraIndexParamsCreate

Allocate CAGRA Index params, and populate with default values

cuvsError_t cuvsCagraIndexParamsCreate(cuvsCagraIndexParams_t* params);

Parameters

NameDirectionTypeDescription
paramsincuvsCagraIndexParams_t*cuvsCagraIndexParams_t to allocate

Returns

cuvsError_t

cuvsCagraIndexParamsDestroy

De-allocate CAGRA Index params

cuvsError_t cuvsCagraIndexParamsDestroy(cuvsCagraIndexParams_t params);

Parameters

NameDirectionTypeDescription
paramsincuvsCagraIndexParams_t

Returns

cuvsError_t

cuvsCagraMergeParamsCreate

Allocate CAGRA merge params and populate them with AUTO defaults.

cuvsError_t cuvsCagraMergeParamsCreate(cuvsCagraMergeParams_t* params);

Parameters

NameDirectionTypeDescription
paramscuvsCagraMergeParams_t*

Returns

cuvsError_t

cuvsCagraMergeParamsDestroy

De-allocate CAGRA merge params.

cuvsError_t cuvsCagraMergeParamsDestroy(cuvsCagraMergeParams_t params);

Parameters

NameDirectionTypeDescription
paramscuvsCagraMergeParams_t

Returns

cuvsError_t

cuvsCagraCompressionParamsCreate

Allocate CAGRA Compression params, and populate with default values.

cuvsError_t cuvsCagraCompressionParamsCreate(cuvsCagraCompressionParams_t* params);

Deprecated: Use cuvsPqParamsCreate. This compatibility API is planned for removal in the 27.02 ABI-breaking release.

Parameters

NameDirectionTypeDescription
paramsincuvsCagraCompressionParams_t*cuvsCagraCompressionParams_t to allocate

Returns

cuvsError_t

cuvsCagraCompressionParamsDestroy

De-allocate CAGRA Compression params.

cuvsError_t cuvsCagraCompressionParamsDestroy(cuvsCagraCompressionParams_t params);

Deprecated: Use cuvsPqParamsDestroy. This compatibility API is planned for removal in the 27.02 ABI-breaking release.

Parameters

NameDirectionTypeDescription
paramsincuvsCagraCompressionParams_t

Returns

cuvsError_t

cuvsAceParamsCreate

Allocate ACE params, and populate with default values

cuvsError_t cuvsAceParamsCreate(cuvsAceParams_t* params);

Parameters

NameDirectionTypeDescription
paramsincuvsAceParams_t*cuvsAceParams_t to allocate

Returns

cuvsError_t

cuvsAceParamsDestroy

De-allocate ACE params

cuvsError_t cuvsAceParamsDestroy(cuvsAceParams_t params);

Parameters

NameDirectionTypeDescription
paramsincuvsAceParams_t

Returns

cuvsError_t

cuvsCagraIndexParamsFromHnswParams

Create CAGRA index parameters similar to an HNSW index

cuvsError_t cuvsCagraIndexParamsFromHnswParams(cuvsCagraIndexParams_t params,
int64_t n_rows,
int64_t dim,
int M,
int ef_construction,
enum cuvsCagraHnswHeuristicType heuristic,
cuvsDistanceType metric);

This factory function creates CAGRA parameters that yield a graph compatible with an HNSW graph with the given parameters.

Parameters

NameDirectionTypeDescription
paramsoutcuvsCagraIndexParams_tThe CAGRA index params to populate
n_rowsinint64_tNumber of rows in the dataset
diminint64_tNumber of dimensions in the dataset
MinintHNSW index parameter M
ef_constructioninintHNSW index parameter ef_construction
heuristicinenum cuvsCagraHnswHeuristicTypeStrategy for parameter selection
metricincuvsDistanceTypeDistance metric to use

Returns

cuvsError_t

cuvsCagraIndexParamsFromDataset

Create CAGRA index parameters heuristically tuned for a dataset

cuvsError_t cuvsCagraIndexParamsFromDataset(cuvsCagraIndexParams_t params,
int64_t n_rows,
int64_t dim,
size_t graph_degree,
cuvsDistanceType metric,
size_t build_quality);

This factory function selects the graph build algorithm and its parameters based on the shape of the dataset.

Parameters

NameDirectionTypeDescription
paramsoutcuvsCagraIndexParams_tThe CAGRA index params to populate
n_rowsinint64_tNumber of rows in the dataset
diminint64_tNumber of dimensions in the dataset
graph_degreeinsize_tDegree of the output graph
metricincuvsDistanceTypeDistance metric to use
build_qualityinsize_tHigher values increase build quality (and cost) up to a point

Returns

cuvsError_t

C API for CUDA ANN Graph-based nearest neighbor search

cuvsCagraExtendParams

Supplemental parameters to extend CAGRA Index

struct cuvsCagraExtendParams {
uint32_t max_chunk_size;
};

Fields

NameTypeDescription
max_chunk_sizeuint32_tThe additional dataset is divided into chunks and added to the graph. This is the knob to adjust the tradeoff between the recall and operation throughput. Large chunk sizes can result in high throughput, but use more working memory (O(max_chunk_size*degree^2)). This can also degrade recall because no edges are added between the nodes in the same chunk. Auto select when 0.

cuvsCagraExtendParamsCreate

Allocate CAGRA Extend params, and populate with default values

cuvsError_t cuvsCagraExtendParamsCreate(cuvsCagraExtendParams_t* params);

Parameters

NameDirectionTypeDescription
paramsincuvsCagraExtendParams_t*cuvsCagraExtendParams_t to allocate

Returns

cuvsError_t

cuvsCagraExtendParamsDestroy

De-allocate CAGRA Extend params

cuvsError_t cuvsCagraExtendParamsDestroy(cuvsCagraExtendParams_t params);

Parameters

NameDirectionTypeDescription
paramsincuvsCagraExtendParams_t

Returns

cuvsError_t

cuvsCagraExtend

Extend a CAGRA index with a caller-owned pre-concatenated padded dataset.

cuvsError_t cuvsCagraExtend(cuvsResources_t res,
cuvsCagraExtendParams_t params,
cuvsDataset_t extended_dataset,
int64_t new_start_row,
cuvsCagraIndex_t index);

The caller must build extended_dataset as old \|\| new (size n_old + n_new) before calling. Rows [0, new_start_row) are the original vectors; rows [new_start_row, n_rows) are the additional vectors. new_start_row must equal the current index size. The library only extends the graph and rebinds the index to extended_dataset; keep that dataset alive for the index lifetime.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
paramsincuvsCagraExtendParams_tcuvsCagraExtendParams_t used to extend CAGRA index
extended_datasetincuvsDataset_tcuvsDataset_t caller-owned device-padded dataset of old || new
new_start_rowinint64_trow index where the additional vectors begin
indexin,outcuvsCagraIndex_tcuvsCagraIndex_t CAGRA index

Returns

cuvsError_t

C API for CUDA ANN Graph-based nearest neighbor search

cuvsCagraSearchAlgo

Enum to denote algorithm used to search CAGRA Index

enum cuvsCagraSearchAlgo {
SINGLE_CTA = 0,
MULTI_CTA = 1,
MULTI_KERNEL = 2,
AUTO = 100
};

Values

NameValue
SINGLE_CTA0
MULTI_CTA1
MULTI_KERNEL2
AUTO100

cuvsCagraHashMode

Enum to denote Hash Mode used while searching CAGRA index

enum cuvsCagraHashMode {
HASH = 0,
SMALL = 1,
AUTO_HASH = 100
};

Values

NameValue
HASH0
SMALL1
AUTO_HASH100

cuvsCagraSearchParams

Supplemental parameters to search CAGRA index

struct cuvsCagraSearchParams {
size_t max_queries;
size_t itopk_size;
size_t max_iterations;
enum cuvsCagraSearchAlgo algo;
size_t team_size;
size_t search_width;
size_t min_iterations;
size_t thread_block_size;
enum cuvsCagraHashMode hashmap_mode;
size_t hashmap_min_bitlen;
float hashmap_max_fill_rate;
uint32_t num_random_samplings;
uint64_t rand_xor_mask;
bool persistent;
float persistent_lifetime;
float persistent_device_usage;
};

Fields

NameTypeDescription
max_queriessize_tMaximum number of queries to search at the same time (batch size). Auto select when 0.
itopk_sizesize_tNumber of intermediate search results retained during the search.

This is the main knob to adjust trade off between accuracy and search speed. Higher values improve the search accuracy.
max_iterationssize_tUpper limit of search iterations. Auto select when 0.
algoenum cuvsCagraSearchAlgoWhich search implementation to use.
team_sizesize_tNumber of threads used to calculate a single distance. 4, 8, 16, or 32.
search_widthsize_tNumber of graph nodes to select as the starting point for the search in each iteration. aka search width?
min_iterationssize_tLower limit of search iterations.
thread_block_sizesize_tThread block size. 0, 64, 128, 256, 512, 1024. Auto selection when 0.
hashmap_modeenum cuvsCagraHashModeHashmap type. Auto selection when AUTO.
hashmap_min_bitlensize_tLower limit of hashmap bit length. More than 8.
hashmap_max_fill_ratefloatUpper limit of hashmap fill rate. More than 0.1, less than 0.9.
num_random_samplingsuint32_tNumber of iterations of initial random seed node selection. 1 or more.
rand_xor_maskuint64_tBit mask used for initial random seed node selection.
persistentboolWhether to use the persistent version of the kernel (only SINGLE_CTA is supported a.t.m.)
persistent_lifetimefloatPersistent kernel: time in seconds before the kernel stops if no requests received.
persistent_device_usagefloatSet the fraction of maximum grid size used by persistent kernel. Value 1.0 means the kernel grid size is maximum possible for the selected device. The value must be greater than 0.0 and not greater than 1.0.

One may need to run other kernels alongside this persistent kernel. This parameter can be used to reduce the grid size of the persistent kernel to leave a few SMs idle.
Note: running any other work on GPU alongside with the persistent kernel makes the setup fragile.
- Running another kernel in another thread usually works, but no progress guaranteed
- Any CUDA allocations block the context (this issue may be obscured by using pools)
- Memory copies to not-pinned host memory may block the context

Even when we know there are no other kernels working at the same time, setting kDeviceUsage to 1.0 surprisingly sometimes hurts performance. Proceed with care. If you suspect this is an issue, you can reduce this number to ~0.9 without a significant impact on the throughput.

cuvsCagraSearchParamsCreate

Allocate CAGRA search params, and populate with default values

cuvsError_t cuvsCagraSearchParamsCreate(cuvsCagraSearchParams_t* params);

Parameters

NameDirectionTypeDescription
paramsincuvsCagraSearchParams_t*cuvsCagraSearchParams_t to allocate

Returns

cuvsError_t

cuvsCagraSearchParamsDestroy

De-allocate CAGRA search params

cuvsError_t cuvsCagraSearchParamsDestroy(cuvsCagraSearchParams_t params);

Parameters

NameDirectionTypeDescription
paramsincuvsCagraSearchParams_t

Returns

cuvsError_t

C API for CUDA ANN Graph-based nearest neighbor search

cuvsCagraIndex

Struct holding the CAGRA index storage address and vector element dtype (DLPack-style)

Matches the usual cuVS C index pattern (addr + dtype). addr points at implementation-owned storage (not always a bare cagra::index*); free only via cuvsCagraIndexDestroy. dtype describes index vector elements for queries and template dispatch.

typedef struct cuvsCagraIndex {
uintptr_t addr;
DLDataType dtype;
} cuvsCagraIndex;

Fields

NameTypeDescription
addruintptr_t
dtypeDLDataType

cuvsCagraIndexCreate

Allocate CAGRA index

cuvsError_t cuvsCagraIndexCreate(cuvsCagraIndex_t* index);

Parameters

NameDirectionTypeDescription
indexincuvsCagraIndex_t*cuvsCagraIndex_t to allocate

Returns

cuvsError_t

cuvsCagraIndexDestroy

De-allocate CAGRA index

cuvsError_t cuvsCagraIndexDestroy(cuvsCagraIndex_t index);

Parameters

NameDirectionTypeDescription
indexincuvsCagraIndex_tcuvsCagraIndex_t to de-allocate

Returns

cuvsError_t

cuvsCagraIndexGetDims

Get dimension of the CAGRA index

cuvsError_t cuvsCagraIndexGetDims(cuvsCagraIndex_t index, int64_t* dim);

Parameters

NameDirectionTypeDescription
indexincuvsCagraIndex_tCAGRA index
dimoutint64_t*return dimension of the index

Returns

cuvsError_t

cuvsCagraIndexGetSize

Get size of the CAGRA index

cuvsError_t cuvsCagraIndexGetSize(cuvsCagraIndex_t index, int64_t* size);

Parameters

NameDirectionTypeDescription
indexincuvsCagraIndex_tCAGRA index
sizeoutint64_t*return number of vectors in the index

Returns

cuvsError_t

cuvsCagraIndexGetGraphDegree

Get graph degree of the CAGRA index

cuvsError_t cuvsCagraIndexGetGraphDegree(cuvsCagraIndex_t index, int64_t* graph_degree);

Parameters

NameDirectionTypeDescription
indexincuvsCagraIndex_tCAGRA index
graph_degreeoutint64_t*return graph degree

Returns

cuvsError_t

cuvsCagraIndexGetDataset

Returns a view of the CAGRA dataset

cuvsError_t cuvsCagraIndexGetDataset(cuvsCagraIndex_t index, DLManagedTensor* dataset);

This function returns a non-owning view of the CAGRA dataset. The output will be referencing device memory that is directly used in CAGRA, without copying the dataset at all. This means that the output is only valid as long as the CAGRA index is alive, and once cuvsCagraIndexDestroy is called on the cagra index - the returned dataset view will be invalid.

Note that the DLManagedTensor dataset returned will have an associated ‘deleter’ function that must be called when the dataset is no longer needed. This will free up host memory that stores the shape of the dataset view.

Parameters

NameDirectionTypeDescription
indexincuvsCagraIndex_tCAGRA index
datasetoutDLManagedTensor*the dataset used in cagra

Returns

cuvsError_t

cuvsCagraIndexGetGraph

Returns a view of the CAGRA graph

cuvsError_t cuvsCagraIndexGetGraph(cuvsCagraIndex_t index, DLManagedTensor* graph);

This function returns a non-owning view of the CAGRA graph. The output will be referencing device memory that is directly used in CAGRA, without copying the graph at all. This means that the output is only valid as long as the CAGRA index is alive, and once cuvsCagraIndexDestroy is called on the cagra index - the returned graph view will be invalid.

Note that the DLManagedTensor graph returned will have an associated ‘deleter’ function that must be called when the graph is no longer needed. This will free up host memory that stores the metadata for the graph view.

Parameters

NameDirectionTypeDescription
indexincuvsCagraIndex_tCAGRA index
graphoutDLManagedTensor*the output knn graph.

Returns

cuvsError_t

cuvsCagraUpdateDataset

Update a CAGRA index with a device-padded or device-PQ dataset.

cuvsError_t cuvsCagraUpdateDataset(cuvsResources_t res,
cuvsDataset_t dataset,
cuvsCagraIndex_t index);

This is the centralized dataset update operation for C callers. The index’s opaque handle is rebound to an index over the supplied dataset layout. Device-padded and device-PQ datasets can be attached to any supported index layout. The caller retains ownership of dataset and must keep it alive while index uses it.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
datasetincuvsDataset_towning or non-owning device-padded or device-PQ dataset handle
indexinoutcuvsCagraIndex_tCAGRA index handle

Returns

cuvsError_t

C API for CUDA ANN Graph-based nearest neighbor search

cuvsCagraBuild

Build a CAGRA index from a dataset handle. Acceptable underlying types are:

  1. kDLDataType.code == kDLFloat and kDLDataType.bits = 32
  2. kDLDataType.code == kDLFloat and kDLDataType.bits = 16
  3. kDLDataType.code == kDLInt and kDLDataType.bits = 8
  4. kDLDataType.code == kDLUInt and kDLDataType.bits = 8
cuvsError_t cuvsCagraBuild(cuvsResources_t res,
cuvsCagraIndexParams_t params,
cuvsDataset_t dataset,
cuvsCagraIndex_t index);

The memory space and layout dataset was constructed with select the C++ build overload. Build the handle with an owning factory or the matching dataset view factory (cuvsDatasetMakePaddedView, cuvsDatasetMakeStandardView, or cuvsDatasetMakeBbqView).

Note that a dataset residing in host memory produces a host-backed index, which must be made search-ready with cuvsCagraUpdateDataset (using a device-padded dataset) before calling cuvsCagraSearch.

A CUVS_DATASET_LAYOUT_PQ dataset created by cuvsDatasetMakePQ builds an iterative CAGRA-Q index. VPQ input requires L2Expanded and ITERATIVE_CAGRA_SEARCH (or AUTO_SELECT), and the VPQ dataset must outlive the index because the index stores a non-owning view. A CUVS_DATASET_LAYOUT_BBQ dataset builds a graph-only index; attach a searchable dataset with cuvsCagraUpdateDataset before search.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
paramsincuvsCagraIndexParams_tcuvsCagraIndexParams_t used to build CAGRA index
datasetincuvsDataset_tcuvsDataset_t training dataset or dataset view
indexinoutcuvsCagraIndex_tcuvsCagraIndex_t Newly built CAGRA index. This index needs to be already created with cuvsCagraIndexCreate.

Returns

cuvsError_t

C API for CUDA ANN Graph-based nearest neighbor search

cuvsCagraSearch

Search a CAGRA index with a DLManagedTensor which has underlying DLDeviceType equal to kDLCUDA, kDLCUDAHost, kDLCUDAManaged. It is also important to note that the CAGRA Index must have been built with the same type of queries, such that index.dtype.code == queries.dl_tensor.dtype.code Types for input are:

  1. queries: a. kDLDataType.code == kDLFloat and kDLDataType.bits = 32 b. kDLDataType.code == kDLFloat and kDLDataType.bits = 16 c. kDLDataType.code == kDLInt and kDLDataType.bits = 8 d. kDLDataType.code == kDLUInt and kDLDataType.bits = 8
  2. neighbors: kDLDataType.code == kDLUInt and kDLDataType.bits = 32 or kDLDataType.code == kDLInt and kDLDataType.bits = 64
  3. distances: kDLDataType.code == kDLFloat and kDLDataType.bits = 32
cuvsError_t cuvsCagraSearch(cuvsResources_t res,
cuvsCagraSearchParams_t params,
cuvsCagraIndex_t index,
DLManagedTensor* queries,
DLManagedTensor* neighbors,
DLManagedTensor* distances,
cuvsFilter filter);

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
paramsincuvsCagraSearchParams_tcuvsCagraSearchParams_t used to search CAGRA index
indexincuvsCagraIndex_tcuvsCagraIndex which has been returned by cuvsCagraBuild
queriesinDLManagedTensor*DLManagedTensor* queries dataset to search
neighborsoutDLManagedTensor*DLManagedTensor* output k neighbors for queries
distancesoutDLManagedTensor*DLManagedTensor* output k distances for queries
filterincuvsFiltercuvsFilter input filter that can be used to filter queries and neighbors based on the given bitset.

Returns

cuvsError_t

cuvsCagraSearchMultiPartition

Search multiple CAGRA index partitions concurrently and return the global top-k per query.

cuvsError_t cuvsCagraSearchMultiPartition(cuvsResources_t res,
cuvsCagraSearchParams_t params,
uint32_t num_partitions,
cuvsCagraIndex_t* indices,
DLManagedTensor* queries,
DLManagedTensor* partition_ids,
DLManagedTensor* neighbors,
DLManagedTensor* distances,
cuvsFilter* filters);

For each query row, the function searches all partitions in parallel into an internal intermediate buffer, applies per-partition distance post-processing, runs a batched top-k merge across partitions, and writes the final outputs to the caller-supplied device tensors. All work is submitted to the CUDA stream associated with res; use cuvsStreamSync to wait for completion.

The index element type may be float32, float16, int8, or uint8. All partitions must share the same element type, and the queries must use that same type.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
paramsincuvsCagraSearchParams_tsearch parameters (shared across partitions)
num_partitionsinuint32_tnumber of index partitions
indicesincuvsCagraIndex_t*array of num_partitions cuvsCagraIndex_t pointers, all of the same element type
queriesinDLManagedTensor*DLManagedTensor* (device, same dtype as the indices, [n_queries, dim]); the queries matrix is searched against every partition
partition_idsoutDLManagedTensor*DLManagedTensor* (device, uint32, [n_queries, k]); which partition each returned neighbor came from
neighborsoutDLManagedTensor*DLManagedTensor* (device, uint32 or int64, [n_queries, k]); ordinal in the corresponding partition’s dataset
distancesoutDLManagedTensor*DLManagedTensor* (device, float32, [n_queries, k]); post-processed distance for each (query, neighbor)
filtersincuvsFilter*array of num_partitions filters, one per partition (or NULL for a fully unfiltered search). filters[i] applies to partition i: use {.type=NO_FILTER, .addr=0} for no filter on that partition, or {.type=BITSET, .addr=ptr} where ptr is a uintptr_t-cast DLManagedTensor* holding that partition’s own bitset (one bit per vector in that partition; standard 32-bit packing).

Returns

cuvsError_t

CAGRA C-API serialize functions

cuvsCagraSerializeGraph

Save the CAGRA graph to file without its dataset.

cuvsError_t cuvsCagraSerializeGraph(cuvsResources_t res,
const char* filename,
cuvsCagraIndex_t index);

This supports dense, PQ-backed, and BBQ-built indexes. The serialized file does not contain vector data. After deserialization the index cannot be searched until a compatible dataset is attached with cuvsCagraUpdateDataset.

Experimental, both the API and the serialization format are subject to change.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
filenameinconst char*the file name for saving the graph
indexincuvsCagraIndex_tCAGRA index

Returns

cuvsError_t

cuvsCagraSerializeGraphAndDataset

Save the CAGRA graph and its attached dataset to file.

cuvsError_t cuvsCagraSerializeGraphAndDataset(cuvsResources_t res,
const char* filename,
cuvsCagraIndex_t index);

The index stores a non-owning dataset view. The caller must keep the memory of the dataset backing that view alive while this function runs. Returns CUVS_ERROR without modifying the destination file if the index has no attached dataset. PQ and BBQ datasets are not serialized by this function.

Experimental, both the API and the serialization format are subject to change.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
filenameinconst char*the file name for saving the graph and dataset
indexincuvsCagraIndex_tCAGRA index with an attached host or device dataset

Returns

cuvsError_t

cuvsCagraSerializeToHnswlib

Save the CAGRA index to file in hnswlib format.

cuvsError_t cuvsCagraSerializeToHnswlib(cuvsResources_t res,
const char* filename,
cuvsCagraIndex_t index);

NOTE: The saved index can only be read by the hnswlib wrapper in cuVS, as the serialization format is not compatible with the original hnswlib.

Experimental, both the API and the serialization format are subject to change.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
filenameinconst char*the file name for saving the index
indexincuvsCagraIndex_tCAGRA index

Returns

cuvsError_t

cuvsCagraDeserializeGraph

Load the CAGRA graph from file without retaining a serialized dataset.

cuvsError_t cuvsCagraDeserializeGraph(cuvsResources_t res,
const char* filename,
cuvsCagraIndex_t index);

This succeeds whether or not the file contains a dataset. Use cuvsCagraUpdateDataset to attach a caller-owned device-padded or device-PQ dataset before searching the graph-only index.

Experimental, both the API and the serialization format are subject to change.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
filenameinconst char*the name of the file that stores the index
indexinoutcuvsCagraIndex_tpre-created CAGRA index populated on success and unchanged on failure

Returns

cuvsError_t

cuvsCagraDeserializeGraphAndDataset

Load the CAGRA graph and dataset from file.

cuvsError_t cuvsCagraDeserializeGraphAndDataset(cuvsResources_t res,
const char* filename,
cuvsCagraIndex_t index,
cuvsDataset_t* out_dataset);

The returned owning dataset preserves the serialized host/device memory type and standard/padded layout. The index stores a non-owning view into it, so the caller must keep the dataset alive while the index uses it and destroy it separately with cuvsDatasetDestroy. Only a device-padded result is immediately searchable through the C API; attach a caller-owned device-padded view with cuvsCagraUpdateDataset for any other kind. The output pointer must point to a null handle on entry; deserialization acts as a factory and transfers ownership of the allocated dataset handle on success. Returns CUVS_ERROR when the file has no dataset; the index and output handle are unchanged on failure.

Experimental, both the API and the serialization format are subject to change.

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
filenameinconst char*the name of the file that stores the graph and dataset
indexinoutcuvsCagraIndex_tpre-created CAGRA index populated on success and unchanged on failure
out_datasetoutcuvsDataset_t*receives the allocated owning dataset handle; must point to null on entry

Returns

cuvsError_t

cuvsCagraIndexFromArgs

Load index from a dataset and graph

cuvsError_t cuvsCagraIndexFromArgs(cuvsResources_t res,
cuvsDistanceType metric,
DLManagedTensor* graph,
DLManagedTensor* dataset,
cuvsCagraIndex_t index);

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
metricincuvsDistanceTypecuvsDistanceType to use in the index
graphinDLManagedTensor*the knn graph to use, shape (size, graph_degree)
datasetinDLManagedTensor*the dataset to use, shape (size, dim)
indexinoutcuvsCagraIndex_tcuvsCagraIndex_t CAGRA index populated with the graph and dataset. This index needs to be already created with cuvsCagraIndexCreate.

Returns

cuvsError_t

CAGRA C-API merge functions

cuvsCagraMerge

Merge multiple CAGRA indices into a single CAGRA index.

cuvsError_t cuvsCagraMerge(cuvsResources_t res,
cuvsCagraIndexParams_t params,
cuvsCagraIndex_t* indices,
size_t num_indices,
cuvsFilter filter,
cuvsDataset_t merged_dataset,
cuvsCagraIndex_t output_index);

All input indices must have been built with the same data type (index.dtype) and have the same dimensionality (index.dims). The merged index uses the output parameters specified in cuvsCagraIndexParams. The merge algorithm is selected automatically.

Input indices must have:

  • index.dtype.code and index.dtype.bits matching across all indices.
  • Supported data types for indices: a. kDLFloat with bits = 32 b. kDLFloat with bits = 16 c. kDLInt with bits = 8 d. kDLUInt with bits = 8

The resulting output index will have the same data type as the input indices.

Example:

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
paramsincuvsCagraIndexParams_tcuvsCagraIndexParams_t parameters for the output index
indicesincuvsCagraIndex_t*Array of input cuvsCagraIndex_t handles to merge
num_indicesinsize_tNumber of input indices
filterincuvsFilterFilter that can be used to filter out vectors from the merged index
merged_datasetoutcuvsDataset_tEmpty owning dataset handle. Merge first attempts to allocate and populate device storage with the same layout as the input indices. For an unfiltered merge, if device allocation fails, it falls back to host storage and returns a host-backed output index. Keep this dataset alive while using output_index. A host-backed output index must be updated with cuvsCagraUpdateDataset before device search.
output_indexoutcuvsCagraIndex_tOutput handle that will store the merged index. Must be initialized using cuvsCagraIndexCreate before use.

Returns

cuvsError_t

cuvsCagraMergeWithParams

Merge multiple CAGRA indices with explicit merge parameters.

cuvsError_t cuvsCagraMergeWithParams(cuvsResources_t res,
cuvsCagraIndexParams_t params,
cuvsCagraMergeParams_t merge_params,
cuvsCagraIndex_t* indices,
size_t num_indices,
cuvsFilter filter,
cuvsDataset_t merged_dataset,
cuvsCagraIndex_t output_index);

Parameters

NameDirectionTypeDescription
resincuvsResources_tcuvsResources_t opaque C handle
paramsincuvsCagraIndexParams_tcuvsCagraIndexParams_t parameters for the output index
merge_paramsincuvsCagraMergeParams_tcuvsCagraMergeParams_t parameters controlling the merge algorithm, or NULL to use AUTO defaults
indicesincuvsCagraIndex_t*Array of input cuvsCagraIndex_t handles to merge
num_indicesinsize_tNumber of input indices
filterincuvsFilterFilter that can be used to filter out vectors from the merged index
merged_datasetoutcuvsDataset_tEmpty owning dataset handle. Merge first attempts to allocate and populate device storage with the same layout as the input indices. For an unfiltered merge, AUTO and REBUILD can fall back to host storage if device allocation fails; explicit FASTENER reports the allocation failure instead. Keep this dataset alive while using output_index. A host-backed output index must be updated with cuvsCagraUpdateDataset before device search.
output_indexoutcuvsCagraIndex_tOutput handle initialized with cuvsCagraIndexCreate

Returns

cuvsError_t