Is this page helpful?

Models#

The Models page in NeMo Studio provides a playground interface for testing and experimenting with models. You can try out different models, configure system prompts, and provide input-output examples for prompt-tuning before committing to full model fine-tuning or deployment.

Backend Microservices#

In the backend, the UI communicates with NeMo Deployment Management and NIM Proxy to manage model deployments and run inference on them.

Models Page UI Overview#

The following are the main components and features of the Models page.

Model Listing#

The Models page displays your saved models in a table format with the following columns:

Model Name: The unique identifier for your model.
Base Model: The base model being used.
Description: Optional text describing the model’s purpose or experiment focus.
Created: Timestamp showing when the model was created.

Model Management#

You can perform the following actions on each model by clicking the three-dot menu icon:

Open: Open the model to view and interact with it in the playground.
Clone and Edit: Create a copy of the model and edit its configuration to create a new model.
Delete: Delete the model.

Model Testing#

The following are the common tasks you can perform on the Models page.

Model Selection: Choose from available base models or your custom fine-tuned models to test.
Real-Time Interaction: Send prompts and receive model responses immediately.
Multiple Sessions: Create and manage multiple playground sessions for different experiments.

Model Configuration#

When you create a new model, you can configure the following settings.

System Prompts#

System prompts set the model’s behavior and role:

Custom Instructions: Define how the model should behave, its persona, or task-specific guidelines.
Prompt Templates: Save and reuse effective system prompts across sessions.
Context Setting: Establish the model’s role, tone, and response style.

Learning Examples#

Enhance model performance through in-context learning:

Input-Output Pairs: Provide example input-output pairs to guide model behavior.
Few-Shot Learning: Add multiple examples to help the model understand desired response patterns.
Example Management: Add, edit, or remove examples to refine model responses.
Dynamic Testing: See how different examples affect model outputs in real-time.

Tools#

Expand the capabilities of tool compatible large language models by providing a list of available functions to enable access to external APIs, databases, or services.

Define tool functions using the OpenAI compatible format, as shown in the following example for a weather lookup function:

{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                },
                "format": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "The temperature unit to use. Infer this from the user's location."
                }
            },
            "required": ["location", "format"]
        }
    }
}

For more information see:

Hyperparameters#

Tune model behavior by adjusting parameters such as temperature, maximum tokens, and timeout.

Temperature: Controls the creativity and randomness of model outputs. Higher values (closer to 2) enable more creative and varied outputs, suitable for tasks such as creative writing. Lower values (closer to 0) produce more deterministic and focused responses. A value within the [0.5, 0.8] range is a good starting point for experimentation.
Maximum Tokens: Sets the maximum number of tokens the model can generate in a response. Tokens can be entire words or parts of words. For English, 100 tokens form approximately 75 words on average. Range: 1 to 4096 tokens.
Real-Time Adjustment: Modify hyperparameters and immediately see their impact on model responses.

Use Cases#

The following are possible use cases for the Models page:

Prompt Engineering: Develop and refine prompts before deploying models in production. Test different phrasings, instructions, and formats to achieve optimal results.
Pre-Fine-Tuning Validation: Validate that prompt tuning with examples can achieve your desired outcomes before investing time and resources in full model fine-tuning.
Rapid Prototyping: Quickly prototype AI-powered features or applications by testing model behavior with different configurations and use cases.