medl.tools.mmar_creator package
- class CheckpointSaverComponent(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)
Bases:
medl.tools.mmar_creator.component.Component
Construct the config for a component, generate a dictionary.
- Parameters
name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.
NoteIf the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.
- class ClickInteractionComponent(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)
Bases:
medl.tools.mmar_creator.component.Component
Construct the config for a component, generate a dictionary.
- Parameters
name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.
NoteIf the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.
- class Component(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)
Bases:
object
Construct the config for a component, generate a dictionary.
- Parameters
name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.
NoteIf the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.
- get_config()
Get the generated config of the component.
- set_config(config)
Set the config directly instead of generating.
- Parameters
config – specified content for config content.
- class OptimizerComponent(name: Optional[Union[str, object]] = None, vars: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, data: Optional[Any] = None)
Bases:
medl.tools.mmar_creator.component.Component
Construct the config for a component, generate a dictionary.
- Parameters
name – name of the component, can be a string or a class.
vars – variables for this component, like: “disabled”: True, etc.
args – the args for this component.
data – raw data for a component, if specified, ignore all the others and use it as config directly.
NoteIf the args is an object, please use “@XXX”. If the arg is a lambda function, please use string instead: func=”lambda x: x”.
- class Config(config: Optional[Dict[str, Any]] = None)
Bases:
object
Base class for all the configs: config_train, config_validate, config_inference, etc.
- Parameters
config – predefined config content to initialize, if None, create an empty dictonary.
- add_component(component: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]], dest: Union[Dict, List], key: Optional[str] = None)
Add config content for a component, like: loss, model, transform, etc.
- Parameters
component – input component to add into current config content, can be 1 component or a list of components.
dest – destination to add in the config content, can be a list or dict.
key – if dest is a dict, need to specify the key to add the component.
- add_vars(**kwargs)
Add variables and the values to the config content, for example: add_vars(epoch=100, amp=true)
- Parameters
kwargs – key, value pairs for the config variables.
- build(path: str)
Finalize the config content and save to specified path, only support JSON format so far.
- Parameters
path – file path to save the config.
- get_config() → Dict[str, Any]
Get the dictionary of config content.
- load(path: str)
Load the config content from existing file, only support JSON format so far.
- Parameters
path – file path to load the config.
- reset_config()
Clear the config content.
- set_config(config: Dict[str, Any])
Set the config content with provided dictionary.
- Parameters
config – set the config content based on provided dictionary.
- get_obj_hash(obj)
- replace_args(args: dict, obj_key_dict: dict)
- class MMAR(root_path: Optional[str] = None, datalist_name: str = 'dataset_0.json')
Bases:
object
The MMAR (Medical Model ARchive) defines a standard structure for organizing all artifacts produced during the model development life cycle. For more details of MMAR structure, please refer to: https://docs.nvidia.com/clara/clara-train-sdk/pt/mmar.html.
- Parameters
root_path – path of the root path to save the MMAR content, if None, must provide it in build or load.
datalist_name – file name of the datalist, must be a JSON file, default to “dataset_0.json”.
- add_custom_files(source_dir: str)
Add custom files into this MMAR.
- Parameters
source_dir (str) – source directory to copied custom files from.
- build(root_path: Optional[str] = None)
Save the MMAR content into files under root path with standard MMAR structure.
- Parameters
root_path – path of the root path to save the MMAR content, if None, use self.root_path instead.
- get_commands()
Get the commands of MMAR.
- get_datalist()
Get the datalist dictionary of MMAR.
- get_docs()
Get the documents of the MMAR.
- get_environment()
Get the environment dictionary of MMAR.
- get_inference_config()
Get the inference config of MMAR.
- get_resources()
Get the resources of the MMAR.
- get_train_config()
Get the train config of MMAR.
- get_validate_config()
Get the validate config of MMAR.
- inference()
- load(root_path: Optional[str] = None)
Load the MMAR content from existing files under the root path.
- Parameters
root_path – path of the root path to load the MMAR content, if None, use self.root_path instead.
- set_commands(commands: Dict[str, str])
Set the commands of MMAR.
- Parameters
commands – a dictionary, key is file name, and value is command content.
- set_commands_and_resources_from_template_dir(template_dir: str)
- set_datalist(datalist: Dict[str, Any])
Set the datalist content in config of MMAR.
- Parameters
datalist – a dictonary, usually maps to a JSON file.
- set_docs(docs: Dict[str, str])
Set the documents of MMAR.
- Parameters
docs – a dictionary, key is file name, and value is document content.
- set_environment(env: Dict[str, Any])
Set the environment content in config of MMAR.
- Parameters
env – a dictonary, usually maps to a JSON file.
- set_inference_config(infer_config: medl.tools.mmar_creator.validate_config.ValidateConfig)
Set the inference config of MMAR.
- Parameters
infer_config – must be a ValidateConfig object.
- set_resources(files: Dict[str, str])
Set the resources of MMAR.
- Parameters
files – a dictionary, key is file name, and value is content.
- set_train_config(train_config: medl.tools.mmar_creator.train_config.TrainConfig)
Set the train config of MMAR.
- Parameters
train_config – must be a TrainConfig object.
- set_validate_config(val_config: medl.tools.mmar_creator.validate_config.ValidateConfig)
Set the validate config of MMAR.
- Parameters
val_config – must be a ValidateConfig object.
- train()
- validate()
- read_folder_to_dict(directory: str)
- write_dict_to_folder(dest_dir: str, dictionary: dict)
- class TrainConfig(config: Optional[Dict[str, Any]] = None)
Bases:
medl.tools.mmar_creator.config.Config
Config for train task, it predefined the APIs to add necessary components.
Base class for all the configs: config_train, config_validate, config_inference, etc.
- Parameters
config – predefined config content to initialize, if None, create an empty dictonary.
- add_component(component: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]], module: str, section: Optional[str] = None)
Add a component to the train config content.
- Parameters
component – specified component to add, can be a Component or a list of Components.
section – target section to add the component, can be “train” or “validate”, if None, put module to top level.
module – target module to add the component, for example: “model”, “handlers”, etc.
- add_evaluator(evaluator: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_loss(loss: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_lr_scheduler(lr_scheduler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_model(model: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_optimizer(optimizer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_additional_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_dataloader(dataloader: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_dataset(dataset: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_handler(handler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_inferer(inferer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_key_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_post_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_train_pre_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_trainer(trainer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_additional_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_dataloader(dataloader: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_dataset(dataset: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_handler(handler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_inferer(inferer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_key_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_post_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_val_pre_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- class ConfVars(var_dict: dict)
Bases:
object
This class is for variables inside the Clara-Train configuration.
- get(key)
Returns the string representation of the vars.
- create_train_config(train_section: dict, val_section: Optional[dict] = None, conf_vars: Optional[medl.tools.mmar_creator.utils.ConfVars] = None)
- create_validate_config(eval_conf: dict, conf_vars: Optional[medl.tools.mmar_creator.utils.ConfVars] = None)
- class ValidateConfig(config: Optional[Dict[str, Any]] = None)
Bases:
medl.tools.mmar_creator.config.Config
Config for validation task, it predefined the APIs to add necessary components.
Base class for all the configs: config_train, config_validate, config_inference, etc.
- Parameters
config – predefined config content to initialize, if None, create an empty dictonary.
- add_additional_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_component(component: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]], module: str)
Add a component to the validation config content.
- Parameters
component – specified component to add, can be a Component or a list of Components.
module – target module to add the component, for example: “model”, “handlers”, etc.
- add_dataloader(dataloader: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_dataset(dataset: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_evaluator(evaluator: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_handler(handler: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_inferer(inferer: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_key_metric(metric: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_model(model: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_post_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])
- add_pre_transform(transform: Union[medl.tools.mmar_creator.component.Component, List[medl.tools.mmar_creator.component.Component]])