NVIDIA Morpheus (25.02.01)

morpheus_llm.llm.services.nemo_llm_service.NeMoLLMClient

class NeMoLLMClient(parent, *, model_name, **model_kwargs)[source]

Bases: morpheus_llm.llm.services.llm_service.LLMClient

Client for interacting with a specific model in Nemo. This class should be constructed with the NeMoLLMService.get_client method.

Parameters
parentNeMoLLMService

The parent service for this client.

model_namestr

The name of the model to interact with.

model_kwargsdict[str, typing.Any]

Additional keyword arguments to pass to the model when generating text.

Methods

generate(**input_dict) Issue a request to generate a response based on a given prompt.
generate_async(**input_dict) Issue an asynchronous request to generate a response based on a given prompt.
generate_batch() Issue a request to generate a list of responses based on a list of prompts.
generate_batch_async() Issue an asynchronous request to generate a list of responses based on a list of prompts.
get_input_names() Returns the names of the inputs to the model.

generate(**input_dict)[source]

Issue a request to generate a response based on a given prompt.

Parameters
input_dictdict

Input containing prompt data.

async generate_async(**input_dict)[source]

Issue an asynchronous request to generate a response based on a given prompt.

Parameters
input_dictdict

Input containing prompt data.

generate_batch(inputs: dict[str, list], return_exceptions: Literal[True])list[str | BaseException][source]
generate_batch(inputs: dict[str, list], return_exceptions: Literal[False])list[str]
generate_batch(inputs: dict[str, list], return_exceptions: bool = False)list[str] | list[str | BaseException]

Issue a request to generate a list of responses based on a list of prompts.

Parameters
inputsdict

Inputs containing prompt data.

return_exceptionsbool

Whether to return exceptions in the output list or raise them immediately.

async generate_batch_async(inputs: dict[str, list], return_exceptions: Literal[True])list[str | BaseException][source]
async generate_batch_async(inputs: dict[str, list], return_exceptions: Literal[False])list[str]
async generate_batch_async(inputs: dict[str, list], return_exceptions: bool = False)list[str] | list[str | BaseException]

Issue an asynchronous request to generate a list of responses based on a list of prompts.

Parameters
inputsdict

Inputs containing prompt data.

return_exceptionsbool

Whether to return exceptions in the output list or raise them immediately.

get_input_names()[source]

Returns the names of the inputs to the model.

Returns
list[str]

List of input names.

Previous morpheus_llm.llm.services.nemo_llm_service
Next morpheus_llm.llm.services.nemo_llm_service.NeMoLLMService
© Copyright 2024, NVIDIA. Last updated on Mar 3, 2025.