Set Up Organizational Entities#

Note

The time to complete this tutorial is approximately 15 minutes.

Learn how to set up the organizational entities using the NeMo Entity Store APIs.

Prerequisites#

Before you set up your first project, ensure you have:

Set Up Environment Variables#

Set the following environment variables for storing the NeMo Entity Store microservice base URL and the namespace name.

ENTITY_STORE_BASE_URL="<NeMo Entity Store base URL>"

NAMESPACE="default"

For example, if you continue to use the minikube cluster from Beginner Tutorial Prerequisites, you can use the following values for the environment variables:

ENTITY_STORE_BASE_URL="http://nemo.test"

NAMESPACE="default"

Create a Project#

This section demonstrates how to create a project in the default namespace named demo-first-project with the following custom fields:

  • team

  • priority

  • status

  1. Use the following Python code for a POST request to the NeMo Entity Store v1/projects endpoint.

    def create_project(name, description, namespace, team, priority, status, created_by):
    project_data = {
        "name": name,
        "description": description,
        "namespace": namespace,
        "custom_fields": {
            "team": team,
            "priority": priority,
            "status": status
        },
        "ownership": {
            "created_by": created_by,
            "access_policies": {}
        }
    }
    
    response = requests.post(
        f"{ENTITY_STORE_BASE_URL}/v1/projects",
        headers={
            "Accept": "application/json",
            "Content-Type": "application/json"
        },
        json=project_data
    )
    return response.json()

# Create the project
project = create_project(
    name="demo-first-project",
    description="A demo project for learning how to manage entities.",
    namespace=os.getenv("NAMESPACE"),
    team="<YOUR_TEAM>",
    priority="low",
    status="not-started",
    created_by="<YOUR_EMAIL>"
)
print("Created project:", project)

  2. Save the project_id for later use.

    # Save project_id to environment variables (optional)
with open(".env", "a") as env_file:
    env_file.write(f"\nPROJECT_ID={project['id']}")

Create a Dataset#

This section demonstrates how to register a dataset to a namespace in the NeMo Entity Store.

  1. Use the following Python code for a POST request to the NeMo Entity Store v1/datasets endpoint.

    def create_dataset(name, description, namespace, project_name, created_by):
    dataset_data = {
        "name": name,
        "description": description,
        "namespace": namespace,
        "format": "json",
        "files_url": f"hf://datasets/{namespace}/{name}",
        "project": project_name,
        "custom_fields": {},
        "ownership": {
            "created_by": created_by,
            "access_policies": {}
        }
    }
    
    response = requests.post(
        f"{ENTITY_STORE_BASE_URL}/v1/datasets",
        headers={
            "Accept": "application/json",
            "Content-Type": "application/json"
        },
        json=dataset_data
    )
    response_data = response.json()
    return {
        "dataset_id": response_data.get("id"),
        "dataset_info": response_data
    }

# Create the dataset and save the dataset_id
dataset_result = create_dataset(
    name="<UNIQUE_DATASET_NAME>",
    description="<DATASET_DESCRIPTION>",
    namespace=os.getenv("NAMESPACE"),
    project_name=project["name"],
    created_by="<YOUR_EMAIL>"
)
dataset_id = dataset_result["dataset_id"]
print(f"Created dataset with ID: {dataset_id}")
print("Full dataset info:", dataset_result["dataset_info"])

  2. Save the dataset_id for later use.

       # Save dataset_id to environment variables (optional)
   with open(".env", "a") as env_file:
       env_file.write(f"\nDATASET_ID={dataset_id}")

Next Steps#

Now that you have set up the project and dataset entities, learn how to upload and manage your project entity files using NeMo Data Store and Hugging Face by following the Create Dataset Files tutorial.