L2SRAMPolicyType

Fully qualified name: cupva::mem::L2SRAMPolicyType

Defined in src/host/shared/include/install/cupva_host_common.hpp

enum class cupva::mem::L2SRAMPolicyType : uint32_t

L2SRAM mapping policy types.

Specifies the access policy for L2SRAM memory mapping. This determines how PVA handles coherency and persistence of the L2 memory. The policy controls whether data is filled from DRAM to L2SRAM on access, flushed back to DRAM when evicted. Selecting the appropriate policy is critical for performance and correctness. See the use cases below for guidance on selecting the optimal policy for your application.

This use case is for temporary workspace that doesn’t need to be saved. Since there’s no DRAM backing, data exists only in L2SRAM during program execution.

See also

mem::MapL2() for mapping L2SRAM

Use Case 1: Using L2SRAM as Temporary Scratch Memory

// No DRAM backing needed for scratch memory and no need to specify the access policy when mapping.
void* l2Ptr = MapL2(nullptr, bufferSize);

       │╭───────────────╮
 VPU0  ││    Program    │
       │╰───────────────╯
       │     ▲     │
       │     │ R/W │
       │     │     ▼
L2SRAM ─────────────────────

This use case is for maintaining persistent state, like a background model in background subtraction algorithm, across program invocations. Data is automatically filled from DRAM if not already present in L2 and flushed back when evicted. Please note that PVA can flush data back to DRAM at any time after program completion.

Use Case 2: Use L2SRAM for Persistent Program State

// FILL_AND_FLUSH ensures state persistence if evicted from L2
void* l2RW = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FILL_AND_FLUSH);

       │   ╭────────────────╮
 VPU0  │   │    Program     │
       │   ╰────────────────╯
       │        ▲      │
       │        │ l2RW │
       │        │      ▼
L2SRAM ───────────────────────────────
           ▲                        │
           │                        │Flush if
           │ Fill if                │Evicted
           │ Invalid                ▼
  DRAM ───────────────────────────────

CmdMemcpy can use mapped L2SRAM pointers as either source or destination for memory transfers. This use case extends Use Case 2 by allowing state information stored in L2SRAM to be copied to a different DRAM buffer than the one backing the L2SRAM.

Use Case 3: Use CmdMemcpy to Read L2SRAM Data to Another DRAM Buffer

// FILL_AND_FLUSH ensures state persistence if evicted from L2
void* l2RW = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FILL_AND_FLUSH);
cmdMemcpy = CmdMemcpy::Create(dramPtr2, l2RW, bufferSize);

       │   ╭────────────────╮    ╭────────────────╮
 VPU0  │   │    Program     │....│    cmdMemcpy   │
       │   ╰────────────────╯    ╰────────────────╯
       │        ▲      │
       │        │ l2RW │
       │        │      ▼
L2SRAM ───────────────────────────────────────────
           ▲                     ▲       │
           │                     │       │Copy l2RW
           │ Fill if     Fill if │       │to dramPtr2
           │ Invalid     Invalid │       ▼
  DRAM ───────────────────────────────────────────

The following use case demonstrates how to efficiently pass data between sequential programs scheduled on the same stream. A single DRAM buffer can be mapped with different access policies to create multiple L2SRAM pointers that reference the same L2SRAM region. This allows each program in the sequence to access the shared L2SRAM region with an access policy optimized for its specific role (producer, processor, or consumer). By avoiding unnecessary DRAM operations (fills and flushes), this approach can improve performance and reduce memory bandwidth usage.

Use Case 4: Passing Data Between Sequential Programs Through L2SRAM

// First program stores its output in L2, does not need L2 buffer to be valid before it begins.
// Using FLUSH policy to write data to L2SRAM and ensure it's flushed to DRAM when evicted
void* l2W = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FLUSH);

// Middle programs read the output of previous programs from L2SRAM and write their own output back to L2SRAM.
// Using FILL_AND_FLUSH policy to ensure data is loaded from L2 if available and persisted when evicted.
void* l2RW = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FILL_AND_FLUSH);

// Last program reads the input from L2SRAM but does not need to write back to DRAM.
// Using FILL policy to ensure data is loaded from L2 if available and filled from DRAM if not.
void* l2R = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FILL);

       │╭────────────────╮         ╭───────────────────────────╮         ╭─────────────────╮
VPU0   ││   First Prog   │.........│        Mid Prog(s)        │.........│    Last Prog    │
       │╰────────────────╯         ╰───────────────────────────╯         ╰─────────────────╯
       │        │                             ▲      │                            ▲
       │     l2W│                             │ l2RW │                            │ l2R
       │        ▼                             │      ▼                            │
L2SRAM ──────────────────────────────────────────────────────────────────────────────────────
                              │    ▲                                │    ▲
                     Flush if │    │                       Flush if │    │
                      Evicted │    │ Fill if                Evicted │    │ Fill if
                              ▼    │ Invalid                        ▼    │ Invalid
  DRAM ──────────────────────────────────────────────────────────────────────────────────────

This use case is ideal for neural network weights, lookup tables, and other constant data that needs to be accessed by one or more Programs but not modified during execution.

Use Case 5: Read-only Data Access by Multiple Sequential or Parallel Programs

// Programs only read from L2SRAM, no need to flush data back to DRAM
void* l2R = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FILL);

       │   ╭────────────────╮                  ╭────────────────╮
VPU0   │   │   Program1     │                  │   Program2     │
       │   ╰────────────────╯                  ╰────────────────╯
       │           ▲                                   ▲
       │        l2R│                                l2R│
       │           │                                   │
L2SRAM ────────────────────────────────────────────────────────────────────
           ▲           │              ▲        │
    Fill if│           │       Fill if│        │
    Invalid│        l2R│       Invalid│     l2R│
           │           │              │        │
  DRAM ──────────────  │  ───────────────────  │  ─────────────────────────
       │               │                       │
       │               │                       │
       │               ▼                       ▼
       │        ╭────────────────╮     ╭───────────────╮
VPU1   │        │   Program2     │     │   Program3    │
       │        ╰────────────────╯     ╰───────────────╯

This use case demonstrates how to efficiently share data between programs scheduled on different streams. By using L2SRAM with appropriate policies (FLUSH for producer, FILL for consumer) and fence synchronization, programs can communicate without the performance penalty of transferring through DRAM. The producer program writes data to L2SRAM which is then directly accessed by the consumer program after proper synchronization via fences. This pattern is ideal for pipeline processing where intermediate results need to be passed between different processing stages running on separate streams.

Use Case 6: Sharing Data Between Programs Scheduled on Different Streams

// Producer program writes to L2SRAM, needs FLUSH policy to ensure data is written back to DRAM when evicted
void* l2W = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FLUSH);

// Consumer program reads from L2SRAM, needs FILL policy to ensure data is loaded from DRAM if not in L2SRAM
void* l2R = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FILL);

       │   ╭────────────────╮ ╭───────────────╮
VPU0   │   │ Producer Prog  │.│ RequestFences │
       │   ╰────────────────╯ ╰───────────────╯
       │           │
       │           │l2W
       │           ▼
L2SRAM ──────────────────────────────────────────────────────────────────────────
                                    │                          ▲       │
                                    │ Flush if          Fill if│       │
                                    │ Evicted           Invalid│    l2R│
                                    ▼                          │       │
  DRAM ──────────────────────────────────────────────────────────────  │  ───────
       │                                                               │
       │                                                               │
       │                                                               ▼
       │                                       ╭──────────────╮╭────────────────╮
VPU1   │                                       │ WaitOnFences ││ Consumer Prog  │
       │                                       ╰──────────────╯╰────────────────╯

This use case demonstrates how multiple programs can efficiently work in parallel by partitioning a shared L2SRAM region. Each program operates exclusively on its own portion of the buffer, eliminating potential race conditions. This pattern is particularly useful for data-parallel operations, such as processing different sections of an image simultaneously.

Use Case 7: Parallel Programs Accessing Different Parts of L2SRAM

// Shared buffer with both fill and flush for background modeling
void* l2Ptr = MapL2(dramPtr, bufferSize, L2SRAMPolicyType::FILL_AND_FLUSH);
uint8_t* l2RWHalf1 = l2Ptr;
uint8_t* l2RWHalf2 = l2RWHalf1 + bufferSize/2;

       │   ╭────────────────╮
VPU0   │   │    Program1    │
       │   ╰────────────────╯
       │      ▲           │
       │      │ l2RWHalf1 │
       │      │           ▼
L2SRAM ────────────────────────────────────
           ▲   │           ▲    │
    Fill if│   │           │    │ Flush if
    Invalid│   │           │    │ Evicted
           │   │           │    ▼
  DRAM ──────  │           │ ──────────────
       │       │           │
       │       │ l2RWHalf2 │
       │       ▼           │
       │    ╭────────────────╮
VPU1   │    │    Program2    │
       │    ╰────────────────╯

Values:

enumerator FILL

Fills L2SRAM from DRAM when data is not already present in L2SRAM

enumerator FLUSH

Writes L2SRAM data back to DRAM when evicted Note: Flush may occur any time after program execution completes

enumerator FILL_AND_FLUSH

Combines FILL and FLUSH behaviors: reads from DRAM when invalid and writes back to DRAM when evicted