{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "# NVIDIA NMT NIM Tutorial\n", "\n", "This tutorial walks you through the various features of NVIDIA NMT NIM and how to use the APIs in a Python application. NVIDIA NMT NIM uses the [gRPC API](https://grpc.io/)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisites\n", "\n", "1. Deploy the Riva Translate 1.6b model using the [NVIDIA NMT NIM documentation](https://docs.nvidia.com/nim/speech/latest/nmt/index.html).\n", "2. Install the Riva Python Client library:\n", "\n", " ```bash\n", " sudo apt-get install python3-pip\n", " pip install -U nvidia-riva-client\n", " ```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Import Riva Client Libraries\n", "\n", "Begin by importing some of the necessary libraries, including the Riva Client libraries." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# Import required libraries\n", "import riva.client\n", "\n", "print(\"Using Riva Python Client version: \", riva.client.__version__)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Create Riva clients and connect to the Riva server\n", "\n", "The following URI assumes a local deployment of NVIDIA NMT NIM on the default port. In case the server deployment is on a different host or via Helm chart on Kubernetes, use an appropriate URI." ] }, { "cell_type": "code", "metadata": {}, "source": [ "auth = riva.client.Auth(uri='0.0.0.0:50051')\n", "nmt_client = riva.client.NeuralMachineTranslationClient(auth)" ], "execution_count": 4, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Get list of supported languages." ] }, { "cell_type": "code", "metadata": {}, "source": [ "response = nmt_client.get_config(model=\"\")\n", "print(response)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Perform Single Translation" ] }, { "cell_type": "code", "metadata": {}, "source": [ "inputs = [\"Riva Translate is a neural machine translation model that translates text between languages.\"]\n", "response = nmt_client.translate(inputs, model=\"\", source_language=\"en-US\", target_language=\"fr\")\n", "print(response)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Perform Batch Translation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Riva Translate supports batched inference of multiple inputs to provide a faster translation experience. Using the translation client, one can batch together up to 8 inputs and translate them in a single request." ] }, { "cell_type": "code", "metadata": {}, "source": [ "inputs = [\"I think, therefore I am.\", \"To be, or not to be: that is the question.\", \"The only thing we have to fear is fear itself.\",\n", " \"So many books, so little time.\", \"If you tell the truth, you don't have to remember anything.\", \"Without music, life would be a mistake.\",\n", " \"The world is a book, and those who do not travel read only one page.\", \"The only way to do great work is to love what you do.\"]\n", "response = nmt_client.translate(inputs, model=\"\", source_language=\"en-US\", target_language=\"hi\")\n", "print(response)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Translation Exclusion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Text enclosed with `` and `` tags is not translated." ] }, { "cell_type": "code", "metadata": {}, "source": [ "inputs = [\"Riva translate model translates audio between language pairs.\"]\n", "response = nmt_client.translate(inputs, model=\"\", source_language=\"en-US\", target_language=\"fr\")\n", "print(response)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Custom Translation Dictionary" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A dictionary can be used to provide custom translations or to avoid translating certain words." ] }, { "cell_type": "code", "metadata": {}, "source": [ "inputs = [\"bad morning everyone.\"]\n", "dnt_phrases_dict = {\"bad\": \"good\", \"everyone\": \"\"}\n", "response = nmt_client.translate(inputs, model=\"\", source_language=\"en-US\", target_language=\"fr-FR\", dnt_phrases_dict=dnt_phrases_dict)\n", "print(response)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Morphologically Complex Translations" ] }, { "cell_type": "code", "metadata": { "scrolled": false }, "source": [ "inputs = [\"Agentic AI uses sophisticated reasoning and iterative planning to autonomously solve complex, multi-step problems.\"]\n", "response = nmt_client.translate(inputs, model=\"\", source_language=\"en-US\", target_language=\"ar\")\n", "print(response)\n" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Translations using the default value of `max_len_variation` (default: `20`) lead to incomplete translations for morphologically rich languages. In such cases, setting a higher value allows the model to accommodate a larger variation between the number of source tokens and translated tokens. Increasing the value of `max_len_variation` will lead to higher inference latency. This increase in latency is typically observed only if the target language actually requires more tokens to translate faithfully." ] }, { "cell_type": "code", "metadata": {}, "source": [ "inputs = [\"Agentic AI uses sophisticated reasoning and iterative planning to autonomously solve complex, multi-step problems.\"]\n", "response = nmt_client.translate(inputs, model=\"\", source_language=\"en-US\", target_language=\"ar\", max_len_variation=100)\n", "print(response)" ], "execution_count": null, "outputs": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 2 }