Run Guardrailed InferencePython APIs

Event-Based API

View as Markdown

You can use a guardrails configuration through an event-based API using LLMRails.generate_events_async and LLMRails.generate_events. For the full API reference, see Python SDK Reference.

Example usage:

1import json
2from nemoguardrails import LLMRails, RailsConfig
3
4config = RailsConfig.from_path("path/to/config")
5app = LLMRails(config)
6
7new_events = app.generate_events(events=[{
8    "type": "UtteranceUserActionFinished",
9    "final_transcript": "Hello! What can you do for me?"
10}])
11print(json.dumps(new_events, indent=True))

Example output:

1[
2  {
3    "type": "StartInternalSystemAction",
4    "action_name": "generate_user_intent",
5    "action_params": {},
6    "action_result_key": null,
7    "is_system_action": true,
8  },
9  {
10    "type": "InternalSystemActionFinished",
11    "action_name": "generate_user_intent",
12    "action_params": {},
13    "action_result_key": null,
14    "status": "success",
15    "return_value": null,
16    "events": [{ "type": "UserIntent", "intent": "express greeting" }],
17    "is_system_action": true,
18  },
19  { "type": "UserIntent", "intent": "express greeting" },
20  { "type": "BotIntent", "intent": "express greeting" },
21  {
22    "type": "StartInternalSystemAction",
23    "action_name": "retrieve_relevant_chunks",
24    "action_params": {},
25    "action_result_key": null,
26    "is_system_action": true,
27  },
28  { "type": "ContextUpdate", "data": { "relevant_chunks": "" } },
29  {
30    "type": "InternalSystemActionFinished",
31    "action_name": "retrieve_relevant_chunks",
32    "action_params": {},
33    "action_result_key": null,
34    "status": "success",
35    "return_value": "",
36    "events": null,
37    "is_system_action": true,
38  },
39  {
40    "type": "StartInternalSystemAction",
41    "action_name": "generate_bot_message",
42    "action_params": {},
43    "action_result_key": null,
44    "is_system_action": true,
45  },
46  {
47    "type": "ContextUpdate",
48    "data": { "_last_bot_prompt": "<<REMOVED FOR READABILITY>>>" },
49  },
50  {
51    "type": "InternalSystemActionFinished",
52    "action_name": "generate_bot_message",
53    "action_params": {},
54    "action_result_key": null,
55    "status": "success",
56    "return_value": null,
57    "events": [{ "type": "StartUtteranceBotAction", "script": "Hello!" }],
58    "is_system_action": true,
59  },
60  { "type": "StartUtteranceBotAction", "script": "Hello!" },
61  { "type": "Listen" },
62]

Event Types

NeMo Guardrails supports multiple types of events. Some are meant for internal use (for example, UserIntent, BotIntent), while others represent the “public” interface (for example, UtteranceUserActionFinished, StartUtteranceBotAction).

UtteranceUserActionFinished

The raw message from the user.

Example:

1{
2  "type": "UtteranceUserActionFinished",
3  "final_transcript": "Hello!"
4}

UserIntent

The computed intent (a.k.a. canonical form) for what the user said.

Example:

1{
2  "type": "UserIntent",
3  "intent": "express greeting"
4}

BotIntent

The computed intent for what the bot should say.

Example:

1{
2  "type": "BotIntent",
3  "intent": "express greeting"
4}

StartUtteranceBotAction

The final message from the bot.

Example:

1{
2  "type": "StartUtteranceBotAction",
3  "script": "Hello!"
4}

StartInternalSystemAction

An action needs to be started.

Example:

1{
2  "type": "StartInternalSystemAction",
3  "action_name": "generate_user_intent",
4  "action_params": {},
5  "action_result_key": null,
6  "is_system_action": true
7}

InternalSystemActionFinished

An action has finished.

Example:

1{
2  "type": "InternalSystemActionFinished",
3  "action_name": "generate_user_intent",
4  "action_params": {},
5  "action_result_key": null,
6  "status": "success",
7  "return_value": null,
8  "events": [
9    {
10      "type": "UserIntent",
11      "intent": "express greeting"
12    }
13  ],
14  "is_system_action": true
15}

ContextUpdate

The context of the conversation has been updated.

Example:

1{
2  "type": "ContextUpdate",
3  "data": {
4    "user_name": "John"
5  }
6}

Listen

The bot has finished processing the events and is waiting for new input.

Example:

1{
2  "type": "Listen"
3}

Custom Events

You can also use custom events:

{
  "type": "some_other_type",
  ...
}

You need to make sure that the guardrails logic can handle the custom event. You do this by updating your flows to deal with the new events where needed. Otherwise, the custom event will just be ignored.

Typical Usage

Typically, you will need to:

  1. Persist the events history for a particular user in a database.
  2. Fetch the history and append the new event whenever there is a new message or another event.
  3. Use the guardrails API to generate the next events.
  4. Filter the StartUtteranceBotAction events and return them to the user.
  5. Persist the history of events back into the database.