DPS Nautobot Integration - Current Use Cases#

Overview#

This document describes the Nautobot import flow as it exists today. It is written for developers and architects working against the import API.

If you are performing an import rather than building against it, refer to Importing External Inventory.

Current implementation spans:

  • Web UI import wizard

  • CLI commands for generate/status/cancel/delete

  • gRPC/HTTP API surfaces used by the UI and clients

It does not currently include saved default queries or import-specific RBAC.

Current API Surface#

The current Nautobot import flow relies on these operations:

  • GenerateEntities: starts an async fetch and returns a request_id

  • GetGenerateEntitiesStatus: streams generated Entity messages for a request_id

  • CancelGenerateEntitiesRequest: cancels a pending or running request

  • DeleteGenerateEntitiesRequest: deletes a terminal request and its staged results

  • UpsertEntities: imports selected entities into the main DPS entity store

Interface Scope#

The interfaces do not all expose the same workflow:

  • Web UI: fetch, review, select, optional edit, and import

  • CLI: generate, status, cancel, and delete primitives

  • API: the same low-level primitives used by clients

The guided review-and-import experience is currently a Web UI workflow, not a single dedicated first-class CLI or API workflow but can be accomplished in multiple manual steps.

Filter Semantics#

Nautobot filter handling in the current implementation works as follows:

  • Structured filters are locations, racks, device_types, and request_string

  • Human-friendly values such as location names, rack names, and device type models may be resolved to Nautobot UUIDs before the device query runs

  • Rack-name lookup is location-scoped when locations are available

  • UUID values are passed through without lookup

  • Some parameters in request_string are reserved for structured handling, while other parameters are passed through to the Nautobot devices endpoint

Persistence and Cleanup#

Fetched entities are staged before final import:

  • Async generate requests and fetched entities are stored in the generate_entities_* tables

  • This staging data is separate from the main DPS entity store

  • GetGenerateEntitiesStatus reads from staged request state and staged entities

  • UpsertEntities writes selected entities into the main DPS entity store

  • DeleteGenerateEntitiesRequest removes staged request data; it does not remove already imported entities

Capability 1: Nautobot-to-DPS Entity Fetch and Import#

Use Case 1.1: Operator Queries Nautobot for Candidate Entities#

Actor: Operator (via Web UI, CLI, or API)

Precondition: DPS is configured with Nautobot access.

Flow:

  1. Operator submits Nautobot filters: locations, racks, device_types, and/or request_string.

  2. DPS validates the request.

  3. DPS creates an async GenerateEntities request and returns a request_id.

  4. DPS resolves human-friendly filter values to Nautobot UUIDs where needed.

  5. DPS queries Nautobot and stores generated entities in the generate_entities_* staging tables.

  6. Client retrieves the generated entities by calling GetGenerateEntitiesStatus(request_id), which streams Entity messages until completion.

  7. Web UI displays the fetched entities for review.

Postcondition: Candidate entities exist in the generate-entities staging tables and are available for review. They are not yet imported into the main DPS entity store.

Current validation rules:

  • At least one filter is required

  • Location-only is rejected unless the query string provides an allowed identifier such as rack=UUID

  • Device-type-only is rejected

  • Rack names require a location filter; rack UUIDs do not

  • Query-string-only requests are allowed only for specific identifier-style filters

Error paths:

  • Invalid filters -> validation error

  • Nautobot unreachable or query failure -> request reaches terminal error state

  • Missing request ID -> invalid argument

  • Zero results -> successful request with no streamed entities

Concurrency note: The current implementation does not prevent multiple in-flight GenerateEntities requests per user or session. Each request gets its own request_id. The Web UI tracks a single lastRequestId and cleans up before re-fetch; CLI and API callers are responsible for managing their own request lifecycle.

Use Case 1.2: Operator Reviews and Selects Entities for Import#

Actor: Operator (primarily via Web UI)

Precondition: Candidate entities were fetched via UC 1.1.

Flow:

  1. Operator reviews the fetched entities.

  2. Operator selects which entities to keep in the working set.

  3. Operator optionally edits entities during the review step.

  4. Operator confirms import.

  5. UI uploads the selected entities through the existing UpsertEntities path.

Postcondition: Selected entities are upserted into DPS. They are no longer only staged import results; they are part of the DPS entity inventory and can be used to build a topology afterward.

Notes:

  • The current repo does not have a dedicated ImportEntities(selectedEntityIDs) RPC.

  • Selection and review are currently a Web UI workflow, not a first-class CLI workflow.

  • “Optionally edits entities” refers to UI-level field adjustments before upsert; the specific editable fields and their validation are governed by the UpsertEntities schema, not this import flow.

  • Re-importing the same entities is handled by UpsertEntities semantics (create-or-update); there is no special duplicate detection or merge logic in the import path.

Use Case 1.3: Operator Cancels or Clears a Pending Fetch#

Actor: Operator (via Web UI, CLI, or API)

Precondition: A generate-entities request exists.

Flow:

  1. Operator chooses to cancel a request.

  2. DPS calls CancelGenerateEntitiesRequest(request_id).

  3. If the request is pending or running, DPS cancels the stored context.

  4. DPS marks the request status as cancelled.

  5. If the operator also wants staged results removed, client calls DeleteGenerateEntitiesRequest(request_id) after the request is terminal.

Postcondition:

  • After cancel: request is terminal and no more work should continue

  • After delete: request row and staged entities are removed from the staging tables

Important distinction:

  • CancelGenerateEntitiesRequest does not itself delete the staging row

  • DeleteGenerateEntitiesRequest is a separate operation

  • In the current Web UI, “Remove All” performs cancel first and then attempts delete

Use Case 1.4: Operator Re-fetches with New Filters#

Actor: Operator

Precondition: A previous request may still exist.

Flow in the current Web UI:

  1. Operator changes filters and clicks Fetch again.

  2. UI attempts to cancel the previous request.

  3. UI then attempts to delete the previous request and its staged entities.

  4. UI clears the local working set.

  5. UI starts a new GenerateEntities request.

Postcondition: The UI presents only the newly fetched candidate entities.

Scope note: Automatic cleanup on re-fetch is currently a Web UI behavior. CLI and direct API clients manage request lifecycle explicitly.

Architecture and Sequences#

UC 1.1 - Query Nautobot for Candidate Entities#

                 Operator
                    |
        +-----------+-----------+
        |           |           |
      Web UI       CLI         API
        |           |           |
        +-----------+-----------+
                    |
                    v
          +------------------+
          |   DPS Gateway    |
          |  (gRPC / REST)   |
          +------------------+
                    |
          1. Validate filters
          2. GenerateEntities RPC
                    |
                    v
          +------------------+
          | Topology Manager |
          +------------------+
                    |
          3. Resolve names -> UUIDs
          4. Query Nautobot API
                    |
                    v
          +------------------+
          |  Nautobot Client |
          +------------------+
                    |
                    v
          +------------------+
          | Nautobot Instance|
          |   (REST API)     |
          +------------------+
                    |
          5. Store staged entities
                    |
                    v
          +------------------+
          |   Staging DB     |
          | (generate_entities_*) |
          +------------------+
                    |
          6. GetGenerateEntitiesStatus
          7. Stream Entity messages

Sequence:

sequenceDiagram participant Op as Operator participant UI as DPS UI, DPS Client participant GW as gRPC API participant TM as TopologyManager participant NC as NautobotClient participant NB as Nautobot participant DB as DB Op->>UI: Specify filters UI->>GW: GenerateEntities(filters) GW->>TM: Validate filters alt Invalid filters TM-->>UI: Validation error end TM->>NC: ListDevices(filters) NC->>NC: resolve names to UUIDs NC->>NB: Nautobot REST queries NB-->>NC: Device list NC-->>TM: []Device TM->>DB: Write request + candidate entities TM-->>UI: request_id UI->>GW: GetGenerateEntitiesStatus(request_id) GW-->>UI: stream Entity messages UI-->>Op: Display candidate entities

UC 1.2 - Review and Import Selected Entities#

sequenceDiagram participant Op as Operator participant UI as Web UI participant GW as gRPC API participant TM as TopologyManager participant ES as Entity Store Op->>UI: Review fetched entities Op->>UI: Select subset and confirm import UI->>GW: UpsertEntities(stream Entity) GW->>TM: Validate and upsert entities TM->>ES: Upsert selected entities TM-->>UI: Success UI-->>Op: Import confirmed

Expanded sequence:

sequenceDiagram participant Op as Operator participant UI as Web UI participant GW as gRPC API participant TM as TopologyManager participant ES as Entity Store Op->>UI: Confirm import of selected entities UI->>GW: UpsertEntities(stream Entity) loop For each selected entity GW->>TM: Receive entity TM->>ES: Create or update entity end TM-->>UI: Empty success response UI-->>Op: Successfully imported N entities

UC 1.3 - Cancel and Optional Delete#

sequenceDiagram participant UI as Interface participant GW as DPS Gateway participant TM as TopologyManager participant DB as StagingDB UI->>GW: CancelGenerateEntitiesRequest(request_id) GW->>TM: Lookup request alt Pending or running TM->>TM: Cancel stored context end TM->>DB: Set status = "cancelled" TM-->>UI: OK opt Remove staged results too UI->>GW: DeleteGenerateEntitiesRequest(request_id) GW->>DB: DELETE request + cascaded entities GW-->>UI: OK end

UC 1.4 - Re-fetch with New Filters#

sequenceDiagram participant Op as Operator participant UI as Web UI participant GW as gRPC API Op->>UI: Adjust filters, click Fetch UI->>GW: CancelGenerateEntitiesRequest(previousRequestId) UI->>GW: DeleteGenerateEntitiesRequest(previousRequestId) UI->>GW: GenerateEntities(newFilters) UI->>GW: GetGenerateEntitiesStatus(newRequestId)

UC 2.1 - Auto-Import with Continuous Sync#

sequenceDiagram participant Op as Operator participant UI as Interface participant GW as DPS Gateway participant TM as TopologyManager participant NC as NautobotClient participant NB as Nautobot participant ES as Entity Store participant DB as SyncConfigDB participant NATS as NATS Op->>UI: Submit filters with auto-import UI->>GW: AutoImportEntities(filters) GW->>TM: Validate filters alt Invalid filters TM-->>UI: Validation error end TM->>DB: Save sync configuration + filters TM->>NC: ListDevices(filters) NC->>NB: Nautobot REST queries NB-->>NC: Device list NC-->>TM: []Device TM->>ES: Upsert all matching entities TM->>NATS: Subscribe to update/delete events TM-->>UI: Sync configuration created UI-->>Op: Auto-import complete, sync active loop Periodic cron sync TM->>DB: Load saved filters TM->>NC: ListDevices(saved filters) NC->>NB: Nautobot REST queries NB-->>NC: Device list NC-->>TM: []Device alt New or updated devices TM->>ES: Upsert entities end alt Unresolvable mismatch TM->>NATS: Publish alert notification end end NATS-->>TM: Entity update/delete event alt Update event TM->>ES: Upsert updated entity end alt Delete event TM->>ES: Handle entity removal end

Request State Diagram#

The async generate-entities request lifecycle is:

stateDiagram-v2 [*] --> pending pending --> running running --> complete running --> error running --> cancelled

Notes:

  • GenerateEntities creates the request in pending

  • Worker execution moves it to running

  • Successful generation moves it to complete

  • Backend failures move it to error

  • CancelGenerateEntitiesRequest moves a request to cancelled

  • DeleteGenerateEntitiesRequest removes terminal requests from storage; it is not itself a request state

  • In the current implementation, delete is rejected for pending or running requests