Important

NeMo 2.0 is an experimental feature and currently released in the dev container only: nvcr.io/nvidia/nemo:dev. Please refer to NeMo 2.0 overview for information on getting started.

LLM Services

class nemo_curator.LLMClient

Interface representing a client connecting to an LLM inference server and making requests synchronously

class nemo_curator.AsyncLLMClient

Interface representing a client connecting to an LLM inference server and making requests asynchronously

class nemo_curator.OpenAIClient(openai_client: openai.OpenAI)

A wrapper around OpenAI’s Python client for querying models

query_reward_model(*, messages: Iterable, model: str) → dict

Prompts an LLM Reward model to score a conversation between a user and assistant :param messages: The conversation to calculate a score for.

Should be formatted like:

[{“role”: “user”, “content”: “Write a sentence”}, {“role”: “assistant”, “content”: “This is a sentence”}, …]

Parameters

model – The name of the model that should be used to calculate the reward. Must be a reward model, cannot be a regular LLM.

Returns

A mapping of score_name -> score

class nemo_curator.AsyncOpenAIClient(async_openai_client: openai.AsyncOpenAI)

A wrapper around OpenAI’s Python async client for querying models

async query_reward_model(*, messages: Iterable, model: str) → dict

Prompts an LLM Reward model to score a conversation between a user and assistant :param messages: The conversation to calculate a score for.

Should be formatted like:

[{“role”: “user”, “content”: “Write a sentence”}, {“role”: “assistant”, “content”: “This is a sentence”}, …]

Parameters

model – The name of the model that should be used to calculate the reward. Must be a reward model, cannot be a regular LLM.

Returns

A mapping of score_name -> score

class nemo_curator.NemoDeployClient(nemo_deploy: NemoQueryLLM)

A wrapper around NemoQueryLLM for querying models in synthetic data generation

query_reward_model(*, messages: Iterable, model: str) → dict

Prompts an LLM Reward model to score a conversation between a user and assistant :param messages: The conversation to calculate a score for.

Should be formatted like:

[{“role”: “user”, “content”: “Write a sentence”}, {“role”: “assistant”, “content”: “This is a sentence”}, …]

Parameters

model – The name of the model that should be used to calculate the reward. Must be a reward model, cannot be a regular LLM.

Returns

A mapping of score_name -> score