nemo_gym.server_runtime.request_handling.exception_handling

Middleware that answers a failed request and reports every request’s failures in its reply.

Module Contents

Classes

Name	Description
`ExceptionHandlingMiddleware`	Answer a request whose route failed before replying, and send each reply with its failure report.

Functions

Name	Description
`_add_failure_header`	Put `report` in a reply’s failure header, merged with any report the reply already carries.
`_is_cancelling_this_request`	Whether `error` is the cancellation of the task serving this request.

API

class nemo_gym.server_runtime.request_handling.exception_handling.ExceptionHandlingMiddleware(
    app: starlette.types.ASGIApp,
    failure_reply: typing.Callable[[BaseException], fastapi.responses.JSONResponse]
)

Answer a request whose route failed before replying, and send each reply with its failure report.

A route that raises, or returns without replying, gets failure_reply’s response. That response keeps the partial observations the route left in the request state. Every reply, successful or not, carries the failures observed while serving it in its failure header. Once a reply has started, its status is already sent, so a later failure propagates. A cancelled request still gets the failure reply, and then the cancellation propagates to whoever cancelled the task.

nemo_gym.server_runtime.request_handling.exception_handling.ExceptionHandlingMiddleware.__call__(
    scope: starlette.types.Scope,
    receive: starlette.types.Receive,
    send: starlette.types.Send
) -> None

async

nemo_gym.server_runtime.request_handling.exception_handling.ExceptionHandlingMiddleware._failure_response(
    scope: starlette.types.Scope,
    error: BaseException
) -> fastapi.responses.JSONResponse

nemo_gym.server_runtime.request_handling.exception_handling._add_failure_header(
    message: starlette.types.Message,
    report: nemo_gym.core.failure_records.FailureReport
) -> None

Put report in a reply’s failure header, merged with any report the reply already carries.

nemo_gym.server_runtime.request_handling.exception_handling._is_cancelling_this_request(
    error: BaseException
) -> bool

Whether error is the cancellation of the task serving this request.

Gym cancels that task when a client disconnects, when a checkpoint retires an activation or a model call, and when uvicorn shuts down.