Tokkio UI WebSocket API 1.0.0

Accepts one of the following messages:

• #0TTS Transcript UpdateTTSTranscriptUpdate

  Inform the UI when to show a TTS Transcript. This will be sent when the avatar starts speaking or a new transcript is available.

  • application/json

  Message IDTTSTranscriptUpdate

  Payload

  Expand all

  object

  type

  string

  The type of message is tts_update, indicating that the digital human has started speaking, or their transcript is updated.

  Const:"tts_update"

  tts

  string

  The content of the TTS transcript spoken by the avatar

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "tts_update",
    "tts": "string"
  }
  

  This example has been generated automatically.

• #1TTS Transcript EndTTSTranscriptEnd

  Inform the UI when to hide a TTS Transcript. This will be sent when the avatar stops speaking.

  • application/json

  Message IDTTSTranscriptEnd

  Payload

  Expand all

  object

  type

  string

  The type of message is tts_end, indicating that the avatar has stopped speaking.

  Const:"tts_end"

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "tts_end"
  }
  

  This example has been generated automatically.

• #2ASR Transcript UpdateASRTranscriptUpdate

  Inform the UI when to show an ASR Transcript. This will be sent after the user starts speaking or a new transcript is available.

  • application/json

  Message IDASRTranscriptUpdate

  Payload

  Expand all

  object

  type

  string

  The type of message is asr_update, indicating that the user has started speaking, or a new interim transcript is available.

  Const:"asr_update"

  asr

  string

  The interim content of the ASR transcript spoken by the user.

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "asr_update",
    "asr": "string"
  }
  

  This example has been generated automatically.

• #3ASR Transcript EndASRTranscriptEnd

  Inform the UI when to hide an ASR Transcript. This will be sent after the user stops speaking.

  • application/json

  Message IDASRTranscriptEnd

  Payload

  Expand all

  object

  type

  string

  The type of message is asr_end, indicating that the user has stopped speaking.

  Const:"asr_end"

  asr

  string

  The final content of the ASR transcript spoken by the user.

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "asr_end",
    "asr": "string"
  }
  

  This example has been generated automatically.

• #4User Presence StartUserPresenceStart

  Informs the UI when the user is present.

  • application/json

  Message IDUserPresenceStart

  Payload

  Expand all

  object

  type

  string

  The type of message is user_presence_start, indicating that the user is now present.

  Const:"user_presence_start"

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "user_presence_start"
  }
  

  This example has been generated automatically.

• #5User Presence EndUserPresenceEnd

  Inform the UI when the user is no longer present.

  • application/json

  Message IDUserPresenceEnd

  Payload

  Expand all

  object

  type

  string

  The type of message is user_presence_end, indicating that the user is no longer present.

  Const:"user_presence_end"

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "user_presence_end"
  }
  

  This example has been generated automatically.

• #6Custom ViewCustomView

  Show custom view data in the front-end. This can include images, text, text boxes, buttons, and more, rendered as modular components.

  • application/json

  Message IDCustomView

  Payload

  Expand all

  object

  type

  string

  The type of message is custom_view, indicating that the UI should render the data in this message as modular components.

  Const:"custom_view"

  blocks

  Expand all

  array<anyOf>

  Data to be rendered as modular components

  Items:

  anyOf

  Can adhere to Header:

  Expand all

  object [Header]

  Header component, displays text in large, bold letters.

  id

  string

  type

  string

  Const:"header"

  data

  Expand all

  object

  text

  string

  Content of the header text

  level

  number

  The size of the header. Lower numbers indicate larger text.

  Allowed values:
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Image:

  Expand all

  object [Image]

  Displays an image with an optional caption.

  id

  string

  type

  string

  Const:"image"

  data

  Expand all

  object

  image

  Expand all

  object

  data

  string

  Base64 encoded image data

  url

  string

  Optional URL to image

  Additional properties are allowed.

  caption

  string

  Optional caption for the image

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Image With Text:

  Expand all

  object [Image With Text]

  Displays an image with text next to it.

  id

  string

  type

  string

  Const:"paragraph_with_image"

  data

  Expand all

  object

  image

  Expand all

  object

  data

  string

  Base64 encoded image data

  url

  string

  Optional URL to image

  Additional properties are allowed.

  image_position

  string

  Dictates whether the image will show up to the left or right of the text.

  Allowed values:
  • "left"
  • "right"

  text

  string

  Text to display next to the image.

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Hint Carousel:

  Expand all

  object [Hint Carousel]

  Contains hints to show to the user, rotates through hints one at a time if multiple are present.

  id

  string

  type

  string

  Const:"hint_carousel"

  data

  Expand all

  object

  hints

  Expand all

  array<object>

  object

  name

  string

  Name used to identify the hint.

  text

  string

  Hint text to display to the user.

  Additional properties are allowed.

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Text:

  Expand all

  object [Text]

  Displays a paragraph of text.

  id

  string

  type

  string

  Const:"paragraph"

  data

  Expand all

  object

  text

  string

  Text to display in the paragraph.

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Textbox:

  Expand all

  object [Textbox]

  Provides a textbox input to the user.

  id

  string

  type

  string

  Const:"text_input"

  data

  Expand all

  object

  id

  string

  default_value

  string

  Default value gives the user a hint on what to type, but disappears once the user starts typing.

  value

  string

  Sets the content of the textbox as if the user had typed it.

  label

  string

  Label to appear above the textbox.

  input_type

  string

  Field to describe the type of data that can be entered in the textbox for form validation. e.g. "email".

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Button List:

  Expand all

  object [Button List]

  Provides one or more buttons to the user.

  id

  string

  type

  string

  Const:"button_list"

  data

  Expand all

  object

  buttons

  Expand all

  array<object>

  object

  id

  string

  active

  boolean

  If false, the button will be disabled and unselectable.

  toggled

  boolean

  If true, the button will be selected as if the user had just selected it.

  variant

  string

  Modifies the appearance of the button list.

  Allowed values:
  • "outlined"
  • "contained"
  • "text"

  text

  string

  text to appear in the button.

  Additional properties are allowed.

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Selectable Options Grid:

  Expand all

  object [Selectable Options Grid]

  Provides a list of selectable options to the user.

  id

  string

  type

  string

  Const:"selectable_options_grid"

  data

  Expand all

  object

  buttons

  Expand all

  array<object>

  object

  id

  string

  active

  string

  If false, the button will be disabled and unselectable.

  toggled

  string

  If true, the button will be selected as if the user had just selected it.

  image

  Expand all

  object

  data

  string

  Base64 encoded image data

  url

  string

  Optional URL to image

  Additional properties are allowed.

  text

  string

  Text to display as part of the selectable option.

  Additional properties are allowed.

  Additional properties are allowed.

  Additional properties are allowed.

  Or to Table:

  Expand all

  object [Table]

  Displays a table.

  id

  string

  type

  string

  Const:"table"

  data

  Expand all

  object

  title

  string

  Optional title for table.

  headers

  Expand all

  array<string>

  Array of header strings for the table columns

  Items:

  string

  rows

  Expand all

  array<array<string>>

  Array of arrays containing the table data

  Items:

  array<string>

  Items:

  string

  Additional properties are allowed.

  Additional properties are allowed.

  style

  Expand all

  object

  primary_color

  string

  secondary_color

  string

  background_color

  string

  background_text_color

  string

  primary_text_color

  string

  secondary_text_color

  string

  Additional properties are allowed.

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "custom_view",
    "blocks": [
      {
        "id": "string",
        "type": "header",
        "data": {
          "text": "string",
          "level": 1
        }
      }
    ],
    "style": {
      "primary_color": "string",
      "secondary_color": "string",
      "background_color": "string",
      "background_text_color": "string",
      "primary_text_color": "string",
      "secondary_text_color": "string"
    }
  }
  

  This example has been generated automatically.

• #7InterruptionInterruption

  Indicate that an interruption has occurred in the ACE controller, and that the UI state should be cleaned up accordingly.

  • application/json

  Message IDInterruption

  Payload

  Expand all

  object

  type

  string

  The type of message is interruption, indicating that the UI should clean up its state for user transcripts.

  Const:"interruption"

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "interruption"
  }
  

  This example has been generated automatically.

Accepts one of the following messages:

• #0Conversation StartConversationStart

  Message indicating that the user wishes to start the conversation. The avatar will greet the user once this message is triggered.

  • application/json

  Message IDConversationStart

  Payload

  Expand all

  object

  type

  string

  The type of message is conversation_start, indicating that the user is ready to start the conversation.

  Const:"conversation_start"

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "conversation_start"
  }
  

  This example has been generated automatically.

• #1Conversation EndConversationEnd

  Message indicating that the user wishes to end the conversation. The avatar will say goodbye to the user once this message is triggered.

  • application/json

  Message IDConversationEnd

  Payload

  Expand all

  object

  type

  string

  The type of message is conversation_end, indicating that the user is ready to end the conversation.

  Const:"conversation_end"

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "conversation_end"
  }
  

  This example has been generated automatically.

• #2RAG SettingsRAGSettings

  Message indicating that we should update the configuration of the NVIDIA RAG which Tokkio is connected to.

  • application/json

  Message IDRAGSettings

  Payload

  Expand all

  object

  type

  string

  The type of message is rag_settings, indicating that the user wants to update the settings of the RAG.

  Const:"rag_settings"

  collection_name

  string

  The collection name to be used by the RAG

  use_knowledge_base

  string

  Whether to use retrieval or speak to the LLM directly

  rag_server_url

  string

  The endpoint on which the RAG is hosted

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "rag_settings",
    "collection_name": "string",
    "use_knowledge_base": "string",
    "rag_server_url": "string"
  }
  

  This example has been generated automatically.

• #3Custom View Interim Text InputCustomViewInterimTextInput

  Message indicating that the user has typed text into a custom view text box.

  • application/json

  Message IDCustomViewInterimTextInput

  Payload

  Expand all

  object

  type

  string

  The type of message is key_pressed, indicating that the user entered a key in the text box.

  Const:"key_pressed"

  id

  string

  Identifier for the text box, specified in the custom view block for the text box.

  interim_input

  string

  The current text in the text box.

  timestamp

  number

  Timestamp at which the user action was performed.

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "key_pressed",
    "id": "string",
    "interim_input": "string",
    "timestamp": 0
  }
  

  This example has been generated automatically.

• #4Custom View Final Text InputCustomViewFinalTextInput

  Message indicating that the user has submitted text from a custom view text box.

  • application/json

  Message IDCustomViewFinalTextInput

  Payload

  Expand all

  object

  type

  string

  The type of message is editing_finished, indicating that the userhas submitted or navigated away from the text box.

  Const:"editing_finished"

  id

  string

  Identifier for the text box, specified in the custom view block for the text box.

  input

  string

  The current text in the text box.

  enter_pressed

  string

  If true, the user clicked the "Enter" button. If false, the user navigated away from the text box without submitting.

  timestamp

  number

  Timestamp at which the user action was performed.

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "editing_finished",
    "id": "string",
    "input": "string",
    "enter_pressed": "string",
    "timestamp": 0
  }
  

  This example has been generated automatically.

• #5Custom View Button PressCustomViewButtonPress

  Message indicating that the user has clicked a button in the custom view UI.

  • application/json

  Message IDCustomViewButtonPress

  Payload

  Expand all

  object

  type

  string

  The type of message is button_pressed, indicating that the user clicked a button or selectable option in the custom view.

  Const:"button_pressed"

  id

  string

  Identifier for the button, specified in the custom view block for the button.

  timestamp

  number

  Timestamp at which the user action was performed.

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "button_pressed",
    "id": "string",
    "timestamp": 0
  }
  

  This example has been generated automatically.

• #6Custom View Selectable Option PressCustomViewSelectableOptionPress

  message indicating that the user has toggled a selctable option in the custom view UI.

  • application/json

  Message IDCustomViewSelectableOptionPress

  Payload

  Expand all

  object

  type

  string

  The type of message is button_pressed, indicating that the user clicked a button or selectable option in the custom view.

  Const:"button_pressed"

  id

  string

  Identifier for the button, specified in the custom view block for the button.

  timestamp

  number

  Timestamp at which the user action was performed.

  toggled

  boolean

  Whether the user toggled the option on or off

  Additional properties are allowed.

  Examples

  Payload

  {
    "type": "button_pressed",
    "id": "string",
    "timestamp": 0,
    "toggled": true
  }
  

  This example has been generated automatically.