Scene#

Scene actions are typically available in interactive systems that provide some sort of screen real estate alongside the avatar interaction. In a chatbot system this could either be a section of the app that can display information or the ability to show information inline within a chat. In an interactive avatar system the avatar could be rendered alongside a TV (like in a news anchor scene) or a web UI could be rendered besides the avatar.

Shot Camera Action#

Start the specified camera shot. We use shot to refer to anything that impacts the camera state over a longer period of time, this can include moving the camera to a new location, panning from left to right, etc. This is a state action (like PostureBotAction), that will ensure that the camera returns to its previous shot when the action is finished (override modality).

StartShotCameraAction(shot: str, start_transition: str)#

Start a new shot.

Parameters:

  • shot (str) –

    Natural language description (NLD) of the shot.

    Availability of shots depends on the interactive system.

    Minimal NLD set:

    The following shots should be supported by every interactive system implementing this action: full, medium, close-up

  • start_transition (str) –

    NLD of the transition to the new shot. This should describe the movement.

    Minimal NLD set:

    The following shots should be supported by every interactive system implementing this action: cut, dolly

  • ... – Additional parameters/payload inherited from StartBotAction().

ShotCameraActionStarted()#

The camera shot started

Parameters:

... – Additional parameters/payload inherited from BotActionStarted().

StopShotCameraAction(stop_transition: str)#

Stop the camera shot. The camera will return to the shot it had before this action was started. ShotCameraAction actions have an infinite lifetime, so unless the IM calls the Stop action the camera maintains the shot indefinitely.

Parameters:

  • stop_transition (str) –

    NLD of the transition back to the previous shot (override modality).

    This should describe the movement.

    Minimal NLD set:

    The following shots should be supported by every interactive system implementing this action: cut, dolly

  • ... – Additional parameters/payload inherited from StopBotAction().

ShotCameraActionFinished()#

The camera shot was stopped. The camera has returned to the shot it had before (either a neutral shot) or the shot specified by any overwritten ShotCameraAction actions.

Parameters:

... – Additional parameters/payload inherited from BotActionFinished().

Motion Effect Camera Action#

Apply a camera motion effect to the active camera.

StartMotionEffectCameraAction(effect: str)#

Perform the described camera motion effect.

Parameters:

  • effect (str) –

    Natural language description (NLD) of the effect.

    Availability of effects depends on the interactive system.

    Minimal NLD set:

    The following camera effects should be supported by every interactive system implementing this action:

    shake, jump cut in, jump cut out

  • ... – Additional parameters/payload inherited from StartBotAction().

MotionEffectCameraActionStarted()#

Camera effect started.

Parameters:

... – Additional parameters/payload inherited from BotActionStarted().

StopMotionEffectCameraAction()#

Stop the camera effect . All effects have a limited lifetime and finish “on their own” (e.g., in an interactive avatar system a “shake” effect could be implemented by a 1 second camera motion). The IM can use this action to stop a camera effect before it would be naturally done.

Parameters:

... – Additional parameters/payload inherited from StopBotAction().

MotionEffectCameraActionFinished()#

Camera effect finished.

Parameters:

... – Additional parameters/payload inherited from BotActionFinished().

Visual Information Scene Action#

Visualize information to the user. This action is used to show the user detailed information about a topic. Example: The user is interested in the details about a product or service.

StartVisualInformationSceneAction(
content: List[umim.messages.modalities.scene.VisualInformationContent],
title: str,
summary: str | None,
support_prompts: List[str] | None,
)#

Present information in the scene to the user.

Parameters:

  • content (List[umim.messages.modalities.scene.VisualInformationContent]) – List of options for the user to choose from

  • title (str) – Describes the choice you are offering to the user

  • summary (Optional[str]) – Summary of the information to be shown to the user

  • support_prompts (Optional[List[str]]) – List of prompts supporting the user in making a choice

  • ... – Additional parameters/payload inherited from StartBotAction().

VisualInformationSceneActionStarted()#

The system has started presenting the information to the user..

Parameters:

... – Additional parameters/payload inherited from BotActionStarted().

VisualInformationSceneActionConfirmationUpdated(
confirmation_status: ConfirmationStatus,
)#

Whenever the user confirms or tries to abort the visual information shown in the screen. Examples of this include: clicking a “confirm” button, “clicking on close”

Parameters:

  • confirmation_status (ConfirmationStatus) – Update on the confirmation status. User indicating to have understood or to cancel the visual information

  • ... – Additional parameters/payload inherited from BotActionUpdated().

StopVisualInformationSceneAction()#

Stop presenting the information to the user

Parameters:

... – Additional parameters/payload inherited from StopBotAction().

VisualInformationSceneActionFinished()#

Information action was stopped by the IM (no user action will cause the action to be finished by the Action Server).

Parameters:

... – Additional parameters/payload inherited from BotActionFinished().

Visual Choice Scene Action#

Visualize a choice to the user ideally allowing the user to interact with the choice in multiple ways. Example: Show a website on screen, that presents the option allowing the user to choose either by touching an option or by using his voice to select an option.

StartVisualChoiceSceneAction(
allow_multiple_choices: bool,
choice_type: ChoiceType,
options: List[umim.messages.modalities.scene.VisualChoiceOption],
prompt: str,
image: str | None,
support_prompts: List[str] | None,
)#

Present a choice in the scene to the user.

Parameters:

  • allow_multiple_choices (bool) – Indicate if the user should be able to select multiple choices from the presented options

  • choice_type (ChoiceType) – Configures the type of choice the user can make.

  • options (List[umim.messages.modalities.scene.VisualChoiceOption]) – List of options for the user to choose from

  • prompt (str) – Describes the choice you are offering to the user

  • image (Optional[str]) – Description of an image that should be shown alongside the choice.

  • support_prompts (Optional[List[str]]) – List of prompts supporting the user in making a choice

  • ... – Additional parameters/payload inherited from StartBotAction().

VisualChoiceSceneActionStarted()#

The system has started presenting the choice to the user

Parameters:

... – Additional parameters/payload inherited from BotActionStarted().

VisualChoiceSceneActionChoiceUpdated(current_choice: List[str])#

Whenever the user interacts directly with the choice presented in the scene but does not confirmed cancel the choice, a ChoiceUpdated event is sent out by the interactive system.

Parameters:

  • current_choice (List[str]) – List of Option IDs if the user made a choice

  • ... – Additional parameters/payload inherited from BotActionUpdated().

VisualChoiceSceneActionConfirmationUpdated(
confirmation_status: ConfirmationStatus,
)#

Whenever the user confirms or tries to abort the choice when interacting with the visual representation of the choice. Examples of this include: clicking a “confirm” button, “clicking on close”

Parameters:

  • confirmation_status (ConfirmationStatus) – Status of the choice confirmation by the user.

  • ... – Additional parameters/payload inherited from BotActionUpdated().

StopVisualChoiceSceneAction()#

Stop presenting the choice to the user.

Parameters:

... – Additional parameters/payload inherited from StopBotAction().

VisualChoiceSceneActionFinished(final_choice: List[str])#

The choice action was stopped by the IM. (no user action will cause the action to be finished by the Action Server).

Parameters:

  • final_choice (List[str]) – List of options for the user to choose from

  • ... – Additional parameters/payload inherited from BotActionFinished().

Visual Form Scene Action#

Visualize a form to the user for cases where the bot needs accurate and specific input. Common examples include showing a form to get a user’s postal or email address.

StartVisualFormSceneAction(
inputs: List[umim.messages.modalities.scene.VisualFormInputs],
prompt: str,
image: str | None,
support_prompts: List[str] | None,
)#

Present a visual form that is requesting certain inputs from the user in the scene to the user.

Parameters:

  • inputs (List[umim.messages.modalities.scene.VisualFormInputs]) – List of inputs required.

  • prompt (str) – Describes the inputs you are requesting from the user

  • image (Optional[str]) – Description of an image that should be shown alongside the prompt.

  • support_prompts (Optional[List[str]]) – List of prompts supporting the user in making a choice

  • ... – Additional parameters/payload inherited from StartBotAction().

VisualFormSceneActionStarted()#

The system has started presenting the the form to the user..

Parameters:

... – Additional parameters/payload inherited from BotActionStarted().

VisualFormSceneActionConfirmationUpdated(
confirmation_status: ConfirmationStatus,
)#

Whenever the user confirms or tries to abort the form input when interacting with the visual representation of the form. Examples of this include: clicking a “confirm” button, “clicking on close”

Parameters:

  • confirmation_status (ConfirmationStatus) – Status of the form input confirmation by the user.

  • ... – Additional parameters/payload inherited from BotActionUpdated().

VisualFormSceneActionInputUpdated(
interim_inputs: List[umim.messages.modalities.scene.VisualFormInputs],
)#

Whenever the user interacts directly with the form inputs presented in the scene but has not yet confirmed the input, an Updated action is sent out by the interactive system. This allows the IM to react to partial inputs, e.g. if a user is typing an e-mail address the bot can react to partial inputs (the bot could say “And now only the domain missing” after the user typed “@” in the form field).

Parameters:

  • interim_inputs (List[umim.messages.modalities.scene.VisualFormInputs]) – Final state of all inputs.

  • ... – Additional parameters/payload inherited from BotActionUpdated().

StopVisualFormSceneAction()#

Stop presenting the form to the user.

Parameters:

... – Additional parameters/payload inherited from StopBotAction().

VisualFormSceneActionFinished(
final_inputs: List[umim.messages.modalities.scene.VisualFormInputs],
)#

Form action was stopped by the IM (no user action will cause the action to be finished by the Action Server).

Parameters:

  • final_inputs (List[umim.messages.modalities.scene.VisualFormInputs]) – Final state of all inputs.

  • ... – Additional parameters/payload inherited from BotActionFinished().

On this page