nemo_microservices.data_designer.config.datastore#

Module Contents#

Classes#

DatastoreSettings

Configuration for interacting with a datastore.

Functions#

fetch_seed_dataset_column_names

get_file_column_names

Extract column names based on file type.

resolve_datastore_settings

upload_to_hf_hub

Data#

logger

API#

class nemo_microservices.data_designer.config.datastore.DatastoreSettings(/, **data: Any)#

Bases: pydantic.BaseModel

Configuration for interacting with a datastore.

Initialization

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

endpoint: str#

‘Field(…)’

token: str | None#

‘Field(…)’

nemo_microservices.data_designer.config.datastore.fetch_seed_dataset_column_names(
seed_dataset_reference: nemo_microservices.data_designer.config.seed.SeedDatasetReference,
) → list[str]#
nemo_microservices.data_designer.config.datastore.get_file_column_names(
file_path: str | pathlib.Path,
file_type: str,
) → list[str]#

Extract column names based on file type.

nemo_microservices.data_designer.config.datastore.logger#

‘getLogger(…)’

nemo_microservices.data_designer.config.datastore.resolve_datastore_settings(
datastore_settings: nemo_microservices.data_designer.config.datastore.DatastoreSettings | dict | None,
) → nemo_microservices.data_designer.config.datastore.DatastoreSettings#
nemo_microservices.data_designer.config.datastore.upload_to_hf_hub(
dataset_path: str | pathlib.Path,
filename: str,
repo_id: str,
datastore_settings: nemo_microservices.data_designer.config.datastore.DatastoreSettings,
**kwargs,
) → str#