ReferenceAPIsRust Library Referencenemo-relaycodec

Module oci_genai

View as Markdown

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

Built-in codec for the Oracle Cloud Infrastructure (OCI) Generative AI chat API.

Implements LlmCodec (request decode/encode) and LlmResponseCodec (response decode) for the OCI Generative AI chat format.

OCI-specific patterns handled

  • ChatDetails envelope: Requests may arrive as a full envelope (compartmentId, servingMode, chatRequest) or as a bare chatRequest payload; both are accepted and the envelope is preserved on encode.
  • API formats selected by apiFormat: GENERIC: OpenAI-style messages with UPPERCASE roles (USER/ASSISTANT/SYSTEM/TOOL) whose content is a list of typed parts ({"type": "TEXT", "text": ...}), flat toolCalls ({id, type: "FUNCTION", name, arguments}), and toolCallId on tool messages. Used by Meta Llama, Google, xAI, OpenAI, and imported open-weights models hosted on dedicated AI clusters. COHERE: a single message string plus chatHistory turns with USER/CHATBOT/SYSTEM roles and an optional preambleOverride. Used by Cohere Command models. COHEREV2: responses are a single assistant message with typed content parts and nested function tool calls, per the OCI CohereChatResponseV2 schema. Requests follow the GENERIC messages path with COHERE-style stopSequences; V2-only request fields the normalized shape does not model (citationOptions, documents, …) ride along in extra and survive edits untouched.
  • Model identity: Carried in servingMode.modelId (on-demand) or servingMode.endpointId (dedicated), not in the chat request body.
  • Responses: ChatResult payloads (modelId, chatResponse); usage counters are promptTokens/completionTokens/totalTokens.
  • Unmodeled fields are preserved: envelope and chat-response fields the normalized shape does not model (timeCreated, future provider fields) are carried in extra rather than discarded, consistent with the other response codecs. Unmodeled fields of the decoded choice and assistant message - logprobs, serviceTier, groundingMetadata, reasoningContent, refusal (GENERIC) and toolPlan, citations (COHEREV2) per the OCI schema - are namespaced in extra under "choice" and "message".

The codec accepts the REST wire format only: camelCase keys, as documented in the OCI API reference. Alternate renderings produced by Oracle tooling (the CLI’s kebab-case data envelope, oci.util.to_dict() snake_case dicts) are the caller’s responsibility to convert.

Structs