Skip to navigation

holoscan::ops::inference_eligibility::FallbackContext

Beta
View as Markdown

Everything resolve_dispatch_with_fallback() needs from InferenceOpParams plus the resolved resource capabilities (stream-pool size, green-context partition count).

#include <holoscan/ops/inference_eligibility/fallback_ladder.hpp>

Member variables

NameTypeDescription
model_path_mapstd::map< std::string, std::string >
input_mapstd::map< std::string, std::vector< std::string > >
output_mapstd::map< std::string, std::vector< std::string > >
backendstd::string
backend_mapstd::map< std::string, std::string >
input_on_cudabool
dynamic_input_dimsbool
parallel_inferencebool
enable_cuda_graphsbool
stream_pool_presentbool
stream_pool_max_sizeuint32_tCudaStreamPool::capacity(); 0 means unbounded.
green_context_pool_presentbool
green_context_num_partitionsuint32_t