Get Job Logs#

Stream logs from a data generation job to monitor progress and troubleshoot issues.

Prerequisites#

Before you can get logs from a data generation job, make sure that you have:

  • Obtained the base URL of your NeMo Data Designer service

  • Set the DATA_DESIGNER_BASE_URL environment variable to your NeMo Data Designer service endpoint

export DATA_DESIGNER_BASE_URL="https://your-data-designer-service-url"

To Get Logs from a Data Generation Job#

Choose one of the following options to get logs from a data generation job.

import os
from nemo_microservices import NeMoMicroservices

# Initialize the client
client = NeMoMicroservices(
    base_url=os.environ['DATA_DESIGNER_BASE_URL']
)

# Get job logs
job_id = "job-abc123def456"
logs = client.beta.data_designer.jobs.get_logs(job_id)

print(f"Logs for job {job_id}:")
for log_line in logs:
    print(log_line)

JOB_ID="job-abc123def456"

# Get logs (streaming JSONL format)
curl -X GET \
  "${DATA_DESIGNER_BASE_URL}/v1beta1/data-designer/jobs/${JOB_ID}/logs" \
  -H 'Accept: application/jsonl'
Example Response
{"level": "info", "msg": "Starting data generation job job-abc123def456"}
{"level": "info", "msg": "Initializing model suite: apache-2.0"}
{"level": "info", "msg": "Processing column configuration: seed (category)"}
{"level": "info", "msg": "Processing column configuration: story (prompt)"}
{"level": "info", "msg": "Starting data generation"}
{"level": "info", "msg": "Data generation completed"}

Log Streaming#

The logs endpoint returns streaming JSONL (JSON Lines) format, where each line is a separate JSON object containing log information. This allows for real-time monitoring of job progress.

Log Levels#

  • info: General information about job progress

  • warning: Warning messages that don’t stop execution

  • error: Error messages indicating problems