medl.tools.mmar_creator package

NVIDIA Docs Hub NVIDIA Clara NVIDIA Clara Train 4.1 medl.tools.mmar_creator package

1.0

Submodules

medl.tools.mmar_creator.component module

class CheckpointSaverComponent(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)

Bases: medl.tools.mmar_creator.component.Component

Construct the config for a component, generate a dictionary.

Parameters

name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.

Note

If the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.

class ClickInteractionComponent(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)

Bases: medl.tools.mmar_creator.component.Component

Construct the config for a component, generate a dictionary.

Parameters

name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.

Note

If the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.

class Component(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)

Bases: object

Construct the config for a component, generate a dictionary.

Parameters

name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.

Note

If the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.

get_config(): Get the generated config of the component.

set_config(config)

Set the config directly instead of generating.

Parameters: config – specified content for config content.

class OptimizerComponent(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)

Bases: medl.tools.mmar_creator.component.Component

Construct the config for a component, generate a dictionary.

Parameters

name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.

Note

If the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.

medl.tools.mmar_creator.config module

class Config(config: Optional[Dict[str, Any]] = None)

Bases: object

Base class for all the configs: config_train, config_validate, config_inference, etc.

Parameters: config – predefined config content to initialize, if None, create an empty dictonary.

add_component(component: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]], dest: Union[Dict, List], key: Optional[str] = None)

Add config content for a component, like: loss, model, transform, etc.

Parameters

component – input component to add into current config content, can be 1 component or a list of components.
dest – destination to add in the config content, can be a list or dict.
key – if dest is a dict, need to specify the key to add the component.

add_vars(**kwargs)

Add variables and the values to the config content, for example: add_vars(epoch=100, amp=true)

Parameters: kwargs – key, value pairs for the config variables.

build(path: str)

Finalize the config content and save to specified path, only support JSON format so far.

Parameters: path – file path to save the config.

get_config() → Dict[str, Any]: Get the dictionary of config content.

load(path: str)

Load the config content from existing file, only support JSON format so far.

Parameters: path – file path to load the config.

reset_config(): Clear the config content.

set_config(config: Dict[str, Any])

Set the config content with provided dictionary.

Parameters: config – set the config content based on provided dictionary.

get_obj_hash(obj)

replace_args(args: dict, obj_key_dict: dict)

medl.tools.mmar_creator.mmar module

class MMAR(root_path: Optional[str] = None, datalist_name: str = 'dataset_0.json')

Bases: object

The MMAR (Medical Model ARchive) defines a standard structure for organizing all artifacts produced during the model development life cycle. For more details of MMAR structure, please refer to: https://docs.nvidia.com/clara/clara-train-sdk/pt/mmar.html.

Parameters

root_path – path of the root path to save the MMAR content, if None, must provide it in build or load.
datalist_name – file name of the datalist, must be a JSON file, default to “dataset_0.json”.

add_custom_files(source_dir: str)

Add custom files into this MMAR.

Parameters: source_dir (str) – source directory to copied custom files from.

build(root_path: Optional[str] = None)

Save the MMAR content into files under root path with standard MMAR structure.

Parameters: root_path – path of the root path to save the MMAR content, if None, use self.root_path instead.

get_commands(): Get the commands of MMAR.

get_datalist(): Get the datalist dictionary of MMAR.

get_docs(): Get the documents of the MMAR.

get_environment(): Get the environment dictionary of MMAR.

get_inference_config(): Get the inference config of MMAR.

get_resources(): Get the resources of the MMAR.

get_train_config(): Get the train config of MMAR.

get_validate_config(): Get the validate config of MMAR.

inference()

load(root_path: Optional[str] = None)

Load the MMAR content from existing files under the root path.

Parameters: root_path – path of the root path to load the MMAR content, if None, use self.root_path instead.

set_commands(commands: Dict[str, str])

Set the commands of MMAR.

Parameters: commands – a dictionary, key is file name, and value is command content.

set_commands_and_resources_from_template_dir(template_dir: str)

set_datalist(datalist: Dict[str, Any])

Set the datalist content in config of MMAR.

Parameters: datalist – a dictonary, usually maps to a JSON file.

set_docs(docs: Dict[str, str])

Set the documents of MMAR.

Parameters: docs – a dictionary, key is file name, and value is document content.

set_environment(env: Dict[str, Any])

Set the environment content in config of MMAR.

Parameters: env – a dictonary, usually maps to a JSON file.

set_inference_config(infer_config: medl.tools.mmar_creator.validate_config.ValidateConfig)

Set the inference config of MMAR.

Parameters: infer_config – must be a ValidateConfig object.

set_resources(files: Dict[str, str])

Set the resources of MMAR.

Parameters: files – a dictionary, key is file name, and value is content.

set_train_config(train_config: medl.tools.mmar_creator.train_config.TrainConfig)

Set the train config of MMAR.

Parameters: train_config – must be a TrainConfig object.

set_validate_config(val_config: medl.tools.mmar_creator.validate_config.ValidateConfig)

Set the validate config of MMAR.

Parameters: val_config – must be a ValidateConfig object.

train()

validate()

read_folder_to_dict(directory: str)

write_dict_to_folder(dest_dir: str, dictionary: dict)

medl.tools.mmar_creator.train_config module

class TrainConfig(config: Optional[Dict[str, Any]] = None)

Bases: medl.tools.mmar_creator.config.Config

Config for train task, it predefined the APIs to add necessary components.

Base class for all the configs: config_train, config_validate, config_inference, etc.

Parameters: config – predefined config content to initialize, if None, create an empty dictonary.

add_component(component: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]], module: str, section: Optional[str] = None)

Add a component to the train config content.

Parameters

component – specified component to add, can be a Component or a list of Components.
section – target section to add the component, can be “train” or “validate”, if None, put module to top level.
module – target module to add the component, for example: “model”, “handlers”, etc.

add_evaluator(evaluator: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_loss(loss: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_lr_scheduler(lr_scheduler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_model(model: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_optimizer(optimizer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_additional_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_dataloader(dataloader: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_dataset(dataset: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_handler(handler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_inferer(inferer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_key_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_post_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_train_pre_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_trainer(trainer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_additional_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_dataloader(dataloader: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_dataset(dataset: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_handler(handler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_inferer(inferer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_key_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_post_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_val_pre_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

medl.tools.mmar_creator.utils module

class ConfVars(var_dict: dict)

Bases: object

This class is for variables inside the Clara-Train configuration.

get(key): Returns the string representation of the vars.

create_train_config(train_section: dict, val_section: Optional[dict] = None, conf_vars: Optional[medl.tools.mmar_creator.utils.ConfVars] = None)

create_validate_config(eval_conf: dict, conf_vars: Optional[medl.tools.mmar_creator.utils.ConfVars] = None)

medl.tools.mmar_creator.validate_config module

class ValidateConfig(config: Optional[Dict[str, Any]] = None)

Bases: medl.tools.mmar_creator.config.Config

Config for validation task, it predefined the APIs to add necessary components.

Base class for all the configs: config_train, config_validate, config_inference, etc.

Parameters: config – predefined config content to initialize, if None, create an empty dictonary.

add_additional_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_component(component: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]], module: str)

Add a component to the validation config content.

Parameters

component – specified component to add, can be a Component or a list of Components.
module – target module to add the component, for example: “model”, “handlers”, etc.

add_dataloader(dataloader: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_dataset(dataset: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_evaluator(evaluator: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_handler(handler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_inferer(inferer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_key_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_model(model: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_post_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])

add_pre_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])