hololink::module::RoceReceiverInterfaceV1

Controller that manages data reception via RoCE packets.

This is usually driven by e.g. RoceReceiverOp and serves to set up the host for RDMA data reception, configure the HSB device for compatible data transmission, and provides a routine that observes received data indications and queues those up for application reception.

#include <hololink/module/roce_receiver.hpp>

Inherits from: hololink::module::ConfigurableService< RoceReceiverInterfaceV1 > (public)

Methods

start

virtual hololink_module_status_t hololink::module::RoceReceiverInterfaceV1::start(
    const std::string &ibv_name,
    unsigned ibv_port,
    uint64_t cu_buffer,
    size_t cu_buffer_size,
    size_t cu_frame_size,
    size_t cu_page_size,
    unsigned pages,
    size_t metadata_offset,
    const std::string &peer_ip,
    unsigned queue_size
)

Configure the receiver and network adapter for RDMA.

Frames are delivered into cu_buffer, which is a single contiguous region with enough space to accommodate the total number of frames indicated by the pages parameter; each frame is cu_page_size bytes apart; cu_page_size needs to include space for received frame metadata (and its alignment).

Returns: HOLOLINK_MODULE_OK or an error status.

Throws: std::runtime_error if the receiver cannot be brought up, for example the named device is not found or peer_ip is not a valid address.

Parameters

close

virtual void hololink::module::RoceReceiverInterfaceV1::close()

End the run, returning blocking_monitor() to its caller.

Callable from any thread, including while blocking_monitor() runs. Does nothing if the receiver was never started.

blocking_monitor

virtual void hololink::module::RoceReceiverInterfaceV1::blocking_monitor()

Run the receiver until close() is called.

Frames arrive only while this runs, so it gets a thread of its own, and that thread carries the CUDA context the receiver was started with. Clients typically execute this in its own thread.

Throws: std::runtime_error if start() has not been called.

get_next_frame

virtual bool hololink::module::RoceReceiverInterfaceV1::get_next_frame(
    unsigned timeout_ms,
    RoceReceiverFrameInfoV1 &info
)

Take the next completed frame.

Waits up to timeout_ms for a frame. On successful reception of a frame, info is updated with various metadata about the data, which is present at the indicated address within the cu_buffer passed to start. Frames are returned in the order they are received.

Returns: True when a frame was taken; false when the timeout elapsed first.

Throws: std::runtime_error if start() has not been called.

Parameters

frames_ready

virtual bool hololink::module::RoceReceiverInterfaceV1::frames_ready()

Returns: False when get_next_frame() will block waiting for a new frame to arrive.

set_frame_ready

virtual void hololink::module::RoceReceiverInterfaceV1::set_frame_ready(
    std::function<void()> callback
)

Register a callback run each time a frame becomes available.

An application that cannot afford to wait in get_next_frame() is told from here that a frame is waiting, and takes it when it is ready to. The callback runs on the thread running blocking_monitor(). Register it before that starts; pass an empty std::function to stop being called. It stays registered across calls to start().

Parameters

get_qp_number

virtual uint32_t hololink::module::RoceReceiverInterfaceV1::get_qp_number()

The number of the queue pair that receives this channel’s frames.

The channel programs this into the device so it sends to the right queue pair.

Returns: The number. Only meaningful once start() has run.

Throws: std::runtime_error if start() has not been called.

get_rkey

virtual uint32_t hololink::module::RoceReceiverInterfaceV1::get_rkey()

The key the device presents when it writes a frame into the receiver’s memory.

Returns: The key. Only meaningful once start() has run.

Throws: std::runtime_error if start() has not been called.

external_frame_memory

virtual uint64_t hololink::module::RoceReceiverInterfaceV1::external_frame_memory()

The address, in the device’s view of the receiver’s memory, where the first page begins.

The device addresses page n at this value plus n times page_size(). This is not a CUDA address.

Returns: The address. Only meaningful once start() has run.

Throws: std::runtime_error if start() has not been called.

frame_size

virtual size_t hololink::module::RoceReceiverInterfaceV1::frame_size()

Bytes of frame data the device sends per frame.

Returns: The cu_frame_size start() was given.

page_size

virtual size_t hololink::module::RoceReceiverInterfaceV1::page_size()

Bytes from one page to the next.

Returns: The cu_page_size start() was given.

pages

virtual unsigned hololink::module::RoceReceiverInterfaceV1::pages()

Pages the device cycles through.

Returns: The pages start() was given.

configure

virtual void hololink::module::ConfigurableService<RoceReceiverInterfaceV1>::configure(
    const EnumerationMetadata &
)

Use metadata contents to initialize this instance.

This routine is guaranteed to be called the first time a call to get_service(metadata) is called on a specific service instance.

module

std::shared_ptr<Module> hololink::module::ServiceBase::module() const

The Module that this service was published from.

The service holds no strong reference to its Module, so this does not extend the Module’s lifetime.

Returns: The owning Module, or an empty pointer if the service has not been published or its Module no longer exists.

Static methods

get_service

static std::shared_ptr<RoceReceiverInterfaceV1> hololink::module::ConfigurableService<RoceReceiverInterfaceV1>::get_service(
    const EnumerationMetadata &metadata,
    bool allow_null = false
)

for_each_type_id

static bool hololink::module::Service<RoceReceiverInterfaceV1>::for_each_type_id(
    Callback &&cb
)

Iterate over every type_id supported by this instance.

The callback signature is bool(const char*) — returning false halts the walk; true continues. Iteration is depth-first, so you’ll see the most specific implementation types first.