nemo_deploy.multimodal.query_multimodal#
Module Contents#
Classes#
Sends a query to Triton for Multimodal inference. |
API#
- class nemo_deploy.multimodal.query_multimodal.NemoQueryMultimodal(url, model_name, model_type)#
Sends a query to Triton for Multimodal inference.
.. rubric:: Example
from nemo_deploy.multimodal import NemoQueryMultimodal
nq = NemoQueryMultimodal(url=”localhost”, model_name=”neva”, model_type=”neva”)
input_text = “Hi! What is in this image?” output = nq.query( input_text=input_text, input_media=”/path/to/image.jpg”, max_output_len=30, top_k=1, top_p=0.0, temperature=1.0, ) print(“prompts: “, prompts)
Initialization
- setup_media(input_media)#
Setup input media.
- frame_len(frames)#
Get frame len.
- get_subsampled_frames(frames, subsample_len)#
Get subsampled frames.
- query(
- input_text,
- input_media,
- batch_size=1,
- max_output_len=30,
- top_k=1,
- top_p=0.0,
- temperature=1.0,
- repetition_penalty=1.0,
- num_beams=1,
- init_timeout=60.0,
- lora_uids=None,
Run query.