Is this page helpful?

nemo_microservices.types.shared_params.model_cache_config#

Module Contents#

Classes#

ModelCacheConfig

API#

class nemo_microservices.types.shared_params.model_cache_config.ModelCacheConfig#

Bases: typing_extensions.TypedDict

enabled: bool#

None

Whether caching is enabled (default: False - no caching)

maxsize: int#

None

Maximum number of entries in the cache per model

stats: nemo_microservices.types.shared_params.cache_stats_config.CacheStatsConfig#

None

Configuration for cache statistics tracking and logging.