> This page is for NVIDIA 3D Body Pose NIM, version 1.0.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 1.0.0 (Latest) (default): https://docs.nvidia.com/nim/maxine/body-pose/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/maxine/llms.txt.

# Basic Inference

After the NIM is running, confirm that the gRPC service is ready. Then, use the sample client to send a video and tracked bounding-box annotations and produce per-frame pose results.

## Verify the Service and Install the Client

Complete the following steps before you send an inference request.

1. Perform a health check on the gRPC endpoint.

   - Install `grpcurl` from [github.com/fullstorydev/grpcurl/releases](https://github.com/fullstorydev/grpcurl/releases).

      Example commands to run on Ubuntu:
      ```bash
      wget https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_linux_amd64.deb
      sudo dpkg -i grpcurl_1.9.1_linux_amd64.deb
      ```

      Without `sudo`, extract the same package into a directory you own and put its binary on your `PATH`:
      ```bash
      dpkg-deb -x grpcurl_1.9.1_linux_amd64.deb grpcurl-pkg
      export PATH="$PWD/grpcurl-pkg/usr/bin:$PATH"
      ```

   - Run the health check. The server enables gRPC reflection, so no proto file is needed:

      ```bash
      grpcurl --plaintext localhost:8001 grpc.health.v1.Health/Check
      ```

     If the service is ready, you get a response similar to the following:

      ```json
      { "status": "SERVING" }
      ```

     The same reflection exposes the inference method, `nvidia.ai4m.body_pose.v1.BodyPoseService/EstimateBodyPose`, a bidirectional stream described in [Input and Output](#input-and-output).

   ```{note}
   While the NIM loads and warms up the model, the gRPC port does not accept connections yet, so the health check fails to connect instead of returning a status. Retry it until it returns `SERVING` before sending inference requests.
   ```

   ```{note}
   For using grpcurl with an SSL-enabled server, avoid using the `--plaintext` argument, and use `--cacert` with a CA certificate, `--key` with a private key, or `--cert` with a certificate file. For more details, refer to `grpcurl --help`.
   ```

2. Download the 3D Body Pose client code by cloning the gRPC client repository ([NVIDIA-Maxine/nim-clients](https://github.com/NVIDIA-Maxine/nim-clients)):

   ```bash
   git clone https://github.com/NVIDIA-Maxine/nim-clients.git

   # Go to the 'body-pose' folder
   cd nim-clients/body-pose/
   ```

   Run the install and proto steps from `nim-clients/body-pose/`. The client itself runs from the `scripts` folder, so the client commands on this page use paths relative to `nim-clients/body-pose/scripts/`, and the sample assets are in `../assets/`.

3. For the Python client, create a virtual environment and install the required dependencies. The client requires Python 3.10 to 3.12 with `pip` and `venv` available. The virtual environment is required on distributions whose system Python is marked externally managed, where a plain `pip install` fails with `error: externally-managed-environment`.

   ```bash
   python3 -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt
   ```

   On Debian and Ubuntu, `python3 -m venv` requires the `python3-venv` package (`sudo apt install python3-venv`). On Windows, activate with `.venv\Scripts\activate` instead of `source .venv/bin/activate`. Keep the environment active for every `python` command on this page.

## Compile the Protos (Optional)

The client ships pre-generated Python stubs under `interfaces/`. If you use those stubs, you can skip this step.

The proto files are available in the `protos/proto` folder. You can compile them to generate client interfaces in your preferred programming language. For more details, refer to [Supported languages](https://grpc.io/docs/languages/) in the gRPC documentation. Use the `grpcio-tools` version pinned in `requirements.txt`.

### Python

To compile protos on Linux, run the following commands:

```bash
cd protos/linux/
chmod +x compile_protos.sh
./compile_protos.sh
cd ../../
```

To compile protos on Windows, run the following commands:

```bash
cd protos/windows/
compile_protos.bat
cd ../../
```

## Input and Output

The 3D Body Pose NIM exposes a bidirectional streaming gRPC API. A single call processes one video stream:

1. The client sends a first request that carries the stream configuration (`BodyPoseConfig`) and the full tracked bounding-box annotation for the video, covering all frames. The annotation can instead be split across several requests, provided every one of them arrives no later than the first `video_data` chunk; boxes sent after the video has started are not read. The server assembles the whole annotation before it decodes the video, not incrementally; only the video itself is streamed.
2. The client streams the compressed video file as byte chunks.
3. The server decodes the frames, estimates the pose for each tracked box, and streams back frame-aligned pose results, followed by a final response that marks the stream as flushed.

### Inputs

- **Video:** A compressed video file, streamed as raw byte chunks: H.264 (preferred), H.265, AV1, VP8 or VP9, in an MP4 or WebM/Matroska container, 4:2:0 chroma, 8-bit -- the only pixel format this NIM accepts, on every codec; see [Input Media](/nim/maxine/body-pose/support-matrix#input-media). 10- and 12-bit content (HDR included) and 4:2:2/4:4:4 chroma are refused on the container header before any decoding, as is a codec this GPU's decoder does not decode at all; a header that states neither chroma nor depth passes the product rule, so only the hardware rule and then the first-frame watchdog stand between it and the decoder. H.264 in MP4 is what these pages use in examples and what is tested most heavily. **SDR only — HDR content is not supported.** The server decodes the video; the client never decodes it. **Constant frame rate (CFR) is required, and it is enforced.** The decoder rejects a clip whose frame interval varies, including a constant-rate clip with a single dropped frame, with `INVALID_ARGUMENT` (see [Validation rules](#limits)). The requirement itself follows from the annotation format rather than from the decoder — boxes are addressed by `frame_id`, so a frame index has to mean the same instant to your tracker and to the server. Under VFR it does not, and the poses would be matched to the wrong boxes without anything appearing to fail. Re-encode a VFR source to CFR before sending it.
- **Tracked bounding boxes:** One box per tracked body per frame, in full-image pixel coordinates. The NIM does not run body detection, so this annotation is required. See [Tracked Bounding-Box Annotation Format](#tracked-bounding-box-annotation-format).

### Output

The NIM returns pose data, not a rendered video. Each result carries the index of the decoded input frame it belongs to and one entry per tracked body in that frame: the box, 2D and 3D keypoints, confidences, joint rotations, rest pose, and root pose. All per-joint arrays have 77 entries in the Nova-77 skeleton order. For the fields and types, refer to [Output Format](#output-format).

## Delayed Outputs

The model uses a temporal window, so pose results arrive behind the input. The server buffers approximately 120 frames before it emits the first result and sends keep-alive messages while it decodes and infers. Each result carries the index of the input frame it belongs to, so results stay frame-aligned. After the input ends, the server drains the buffered frames and sends a final response that marks the stream as flushed.

<Note>
A constant frame rate is required. Tracked boxes are matched to frames by `frame_id`, so a variable-frame-rate clip, or a constant-rate clip with a dropped frame, is rejected with `INVALID_ARGUMENT`. Re-encode such a clip before sending it, for example with `ffmpeg -i in.mp4 -r 30 -c:v libx264 -pix_fmt yuv420p out.mp4`.
</Note>

## Tracked Bounding-Box Annotation Format

The client reads the tracked boxes from a text file with one row per tracked body per frame:

```text
{number_of_tracked_bodies}
{frame_id} {tracking_id} {x} {y} {width} {height}
{frame_id} {tracking_id} {x} {y} {width} {height}
...
```

- Line 1 is the number of tracked bodies, from 1 to 50.
- `frame_id` is the index of the decoded frame, starting at 0.
- Boxes are in full-image pixel coordinates, where `x`, `y` is the top-left corner. Integer and floating-point values are both accepted.
- A box that overlaps a frame edge, including one with a negative origin, is passed to the model unclipped.

For example, two tracked bodies across two frames:

```text
2
0 1 120.5 340.0 80.0 220.0
0 2 500.0 300.0 90.0 240.0
1 1 122.0 341.5 80.0 220.0
1 2 501.0 300.5 90.0 240.0
```

The client package ships a sample pair, `assets/sample_video.mp4` and `assets/sample_bbox.txt`, which tracks 15 bodies over the 300 frames of the clip.

### Limits

The sample client validates the annotation file before streaming any video, so an invalid file fails without a round trip to the server. The server validates the entire annotation again for every request because it does not trust the client. If it detects a violation, it ends the call with `INVALID_ARGUMENT`. This server-side validation is the only enforcement available to callers that use the gRPC API directly and bypass the sample client.

| Rule | Enforced by |
| --- | --- |
| At most 50 boxes per frame. | Client and server |
| `tracking_id` must be unique within a frame. | Server |
| Every coordinate must be finite. | Server |
| A box must overlap the frame by at least one pixel. | Server |
| Only the exact quadruple `-1 -1 -1 -1` marks an absent body. The row is skipped and the frame is kept with no box on it. Any other non-positive `width` or `height` is malformed and rejected. | Server (the client skips sentinel rows and sends every other row as it is) |
| Line 1 must be a positive tracked-body count, no greater than 50. | Client |
| The annotation must contain at least one box. | Server |
| At most 108,000 annotated frames and 1,000,000 boxes in total. | Server |

The server also guards against a malformed or hostile annotation with two ceilings no real clip approaches -- 108,000 annotated frames (an hour at 30 fps) and 1,000,000 boxes in total -- and answers `INVALID_ARGUMENT` above them.

<Note>
**Validation rules and silent fallbacks**

- An annotation file that covers fewer frames than the video limits the result set to the annotated frames, and the RPC succeeds. The output file still has one line per decoded frame, and a frame without an annotation carries an empty `detections` list. The client's summary line, `Received poses for {n} frame(s)`, counts only the frames that carry a body, so a count below the video's frame count is the signal. Nothing on the wire flags it — a direct gRPC caller sees only that fewer `ready` responses contained bodies. If your tracker drops frames, check the annotated frame count against the video's frame count before relying on the result count.
- An annotation file that names frames beyond the end of the video fails with `INVALID_ARGUMENT`, reporting both counts, for example `annotation names frame index 299 (300 frames) but the video decoded only 150 frame(s) (indices 0-149)`. The server knows how many frames the video holds only once it has decoded all of them, so this check completes at the end of the stream: a long clip is decoded and inferred in full, and earlier poses are returned, before the rejection arrives. Compare your annotation's highest `frame_id` against the video's real frame count before you send it.
- **Constant frame rate is required, and it is enforced.** The decoder measures the interval between consecutive frames and rejects the request with `INVALID_ARGUMENT` when that interval varies — which includes a constant-rate clip with a single dropped frame. Because tracked boxes are addressed by `frame_id`, a missing frame would match every later pose to the wrong box, so the rejection is deliberate. The check needs three frames to form a verdict and fires when the deviating frame arrives, so a clip can fail part-way through after returning earlier poses. Re-encode at a constant frame rate, or trim the clip before the gap, and retry.
- An invalid `focal_length` (for example `NaN` or a negative value) silently falls back to the SDK auto-derived default instead of raising an error.
- Which mode a request used (see [Input Modes](#input-modes)) is only observable via server log lines (`Video streamable: True/False`, `spooled N bytes for transactional decode`) — there is no field in the response that reports it.
</Note>

### Quick Test with a Sample Input

To smoke-test the pipeline without a real detector or tracker, generate a short synthetic clip and a matching bounding-box file that tracks one full-frame body for every frame. Run these from `nim-clients/body-pose/` with the virtual environment active; the first command moves to the `scripts` folder, where the client runs, and the two files land there. FFmpeg is a prerequisite; refer to [Streaming Mode](#streaming-mode).

```bash
cd scripts

# Generate a 5-second, 1280x720, 30fps test clip.
ffmpeg -f lavfi -i testsrc=duration=5:size=1280x720:rate=30 -c:v libx264 -pix_fmt yuv420p quick_test.mp4

# Generate a matching tracked-bbox file: one full-frame box (tracking ID 1) per frame.
# Line 1 is the number of tracked bodies (1 here), not the number of frames.
python -c "
frames = 5 * 30
print(1)
for i in range(frames):
    print(f'{i} 1 0 0 1280 720')
" > quick_test_bbox.txt
```

Send the pair to the NIM and write the poses to `quick_test.json`:

```bash
python body_pose.py --target 127.0.0.1:8001 --video-input quick_test.mp4 --bbox-input quick_test_bbox.txt --output quick_test.json
```

A full-frame box does not need to contain a real person; it only exercises the end-to-end path (streaming, decode, inference, output).

<Note>
This is a pipeline smoke test, not a pose-quality test. For meaningful pose output, supply real tracked bounding boxes for actual people in the frame.
</Note>

<Warning>
Do not use this 150-frame clip to measure throughput. The model buffers approximately 120 frames before the first result (see [Delayed Outputs](#delayed-outputs)), so roughly 80% of a 150-frame run is startup fill, not steady-state processing -- an fps figure computed over the whole clip will read far below the [Performance Results](/nim/maxine/body-pose/performance-results) tables, which explicitly exclude that buffering window. Benchmark with a clip of at least a few hundred frames, or measure only the post-buffer drain rate.
</Warning>

Expect the client to print `Received poses for 150 frame(s)` and exit `0`. The output has 150 lines, one per frame, each carrying 77 (Nova-77) keypoints and rotations for tracking ID `1` -- the model still runs on the full-frame box even though `testsrc` contains no person, so the values are not meaningful pose data. To count the lines:

```bash
wc -l quick_test.json
```

Anything else (a non-zero exit, fewer than 150 lines, or an error) means the smoke test failed.

## Error Contract

Each row names the gRPC status, what triggers it, the message template, and
which side raises it; a direct gRPC caller is subject to the server rows only.
Placeholders in `{braces}` are substituted at runtime; everything outside them
is literal.

Most server-side annotation rejections share one envelope:

```text
Invalid 3D body pose input:
  - <first problem>
  - <second problem>
```

A row whose message starts with `frame {frame_id}:` is a bullet inside this
envelope, not the whole `details` string, and one rejection can carry bullets
from several frames: the per-frame checks -- non-finite coordinates, malformed
box sizes, duplicate `tracking_id`s and the per-frame box ceiling -- are
collected and reported together. The whole-annotation ceilings
(`MaxAnnotatedFrames`, `MaxAnnotatedBoxes`) and the frame-overlap check stop at
the first offender.

The sample client prints whatever the server returned as
`gRPC error: {STATUS}: {details}` and exits 1. It checks its arguments and the
annotation file before it connects, and prints a failure there as
`Configuration error: {message}`, also exiting 1; the client rows below give
the `{message}` part.

| gRPC status | Trigger condition | Message template | Raised by |
|---|---|---|---|
| `INVALID_ARGUMENT` | First request carries no `BodyPoseConfig` | `First request must include BodyPoseConfig` | Server |
| `INVALID_ARGUMENT` | No `tracked_bboxes` in the leading requests at all. An all-sentinel annotation does *not* land here — see the client row below, which stops it earlier | `First request(s) must include tracked_bboxes for the video frames` | Server |
| `INVALID_ARGUMENT` | More than 108,000 annotated frames (one hour at 30 fps) | `{n} annotated frames > MaxAnnotatedFrames (108000)` | Server |
| `INVALID_ARGUMENT` | More than 1,000,000 boxes across the whole annotation, sentinels included | `annotation exceeds MaxAnnotatedBoxes (1000000) in total` | Server |
| `INVALID_ARGUMENT` | More than 50 boxes in one frame, counting only the boxes that passed every other check -- sentinels, malformed boxes and duplicate `tracking_id`s do not count towards it | `frame {frame_id}: {n} boxes > MaxBoxesPerFrame (50)` | Server (the client also rejects an over-capacity frame from the file before streaming, but counts every row except the sentinel, malformed and duplicate ones included — so it can refuse a file the server would have accepted) |
| `INVALID_ARGUMENT` | `tracking_id` repeated within one frame | `frame {frame_id}: duplicate tracking_id {tracking_id} (tracking_id must be unique within a frame)` | Server |
| `INVALID_ARGUMENT` | `NaN` or an infinity in `x`, `y`, `width` or `height` | `frame {frame_id}: non-finite bbox coordinate(s) ({x}, {y}, {width}, {height})` | Server |
| `INVALID_ARGUMENT` | Any non-positive `width` or `height` on a box that is not the exact `-1 -1 -1 -1` sentinel — malformed, not a sentinel. `0 0 0 0` and `10 10 -1 -1` are both malformed | `frame {frame_id}: malformed bbox size (width={width}, height={height})` | Server |
| `INVALID_ARGUMENT` | Box shares no pixel with the decoded frame. Checked on frame 0, once the real frame size exists, so the video header is streamed first | `frame {frame_id}: bbox (x={x}, y={y}, width={width}, height={height}) lies entirely outside the {width}x{height} frame. A box that overlaps the frame at all is accepted as given, including one with a negative origin; this one shares no pixel with it.` | Server |
| `INVALID_ARGUMENT` | Annotation names frames the video never contained, by any number of frames. A bullet in the envelope above. The server can count the decoded frames only once the input has ended, so this check completes at the end of the stream: a long clip is decoded and inferred in full, and earlier poses are returned, before the rejection arrives | `annotation names frame index {n} ({n+1} frames) but the video decoded only {m} frame(s) (indices 0-{m-1}); the tracked-bbox annotation names frames beyond what this video contains` | Server |
| `INVALID_ARGUMENT` | Cumulative `video_data` exceeds `NV_AI4M_MAX_INPUT_FILE_SIZE_BYTES` (default 1 GiB) | `Invalid 3D body pose input:` followed by the bullet `- input exceeds NV_AI4M_MAX_INPUT_FILE_SIZE_BYTES ({max_bytes} bytes)` -- the same envelope as the annotation errors above | Server |
| `INVALID_ARGUMENT` | No decodable video track — a truncated or headerless upload | `No decodable video stream found in the input. The file may be truncated or the video track may be missing.` | Server |
| `INVALID_ARGUMENT` | Container the decoder does not recognise — the header matches neither supported signature | `Unsupported video container format. Expected MP4/QuickTime or Matroska/WebM, but file header did not match either signature. Supported containers: MP4, WebM/MKV.` | Server |
| `INVALID_ARGUMENT` | Container header names a codec other than H.264, H.265, AV1, VP8 or VP9; nothing was decoded | `unsupported video codec: {codec_label}. Supported: H.264, H.265, AV1, VP8, VP9 (see Support Matrix -> Input Media).` | Server |
| `INVALID_ARGUMENT` | Container header states a chroma subsampling other than 4:2:0 or a bit depth other than 8 (an RGB 4:4:4 WebM, a 10-bit HDR10 MP4, ...); nothing was decoded. This is the product rule and does not depend on the GPU. `{codec_label}` is `H.264`, `H.265`, `AV1`, `VP8` or `VP9`; `{chroma}` is `4:2:0`, `4:2:2`, `4:4:4` or `4:0:0` (monochrome); `{profile_suffix}` is ` (profile N)` when the header carries a profile, else empty. See [Input Media](/nim/maxine/body-pose/support-matrix#input-media) | `unsupported video format: {codec_label} {chroma} {bit_depth}-bit{profile_suffix} -- this NIM accepts 4:2:0 8-bit only (SDR; HDR and 10/12-bit content are not supported). See Support Matrix -> Input Media.` | Server |
| `INVALID_ARGUMENT` | Container header names a codec whose 4:2:0 8-bit entry is missing from this GPU's decoder caps -- the GPU does not decode that codec at all (AV1 on a GPU without AV1 decode, say); nothing was decoded. This is the hardware rule; it is skipped when the caps are unknown. `{decodable_codecs}` is the comma-separated list of served codecs this GPU does decode at 4:2:0 8-bit, in the order H.264, H.265, VP8, VP9, AV1, or the word `nothing` | `unsupported video format: {codec_label} 4:2:0 8-bit -- this GPU's decoder does not decode {codec_label} (it decodes: {decodable_codecs}). See Support Matrix -> Input Media.` | Server |
| `INVALID_ARGUMENT` | No frame decoded within `BODY_POSE_DECODE_FIRST_FRAME_TIMEOUT_SEC` (default 10) of the input being ready to decode -- the upload spooled, or, in streaming mode, the last chunk the decoder accepted (the clock restarts on every chunk and at end of input, so a slow upload does not trip it). The last resort when the header did not reveal the format | `no video frame was decoded within {timeout:g}s of receiving the input (BODY_POSE_DECODE_FIRST_FRAME_TIMEOUT_SEC). The codec or pixel format is probably not supported by this GPU's decoder -- see Support Matrix -> Input Media. If this input decodes elsewhere, raise the timeout.` | Server |
| `INVALID_ARGUMENT` | Decode fails in a way attributable to the bytes sent | the decoder's own text, passed through unchanged | Server |
| `INVALID_ARGUMENT` | The decoder's frame-interval check fires: consecutive-frame spacing varies beyond `max(2 ms, 5 %)`, which a single dropped frame in a constant-rate clip also triggers. Needs three frames to form a verdict, so this can arrive part-way through a stream — see [Validation rules](#limits) | `Variable frame rate input is not supported. This NIM accepts constant frame rate video with no dropped frames: a single missing frame is rejected because poses either side of it would be matched to the wrong boxes. Re-encode at a constant frame rate, or trim the clip before the gap, and retry.` | Server |
| `RESOURCE_EXHAUSTED` | No admission slot freed within `BODY_POSE_REQUEST_ADMISSION_TIMEOUT_SEC` (default 15). Nothing was read or processed, so the request is safe to resend -- but resending it against an unchanged server fails identically. Serialise your streams, or raise `NV_AI4M_MAX_CONCURRENCY_PER_GPU` (default 1) and restart | `Server at capacity: all {n} admission slot(s) in use and none freed within {timeout}s (BODY_POSE_REQUEST_ADMISSION_TIMEOUT_SEC). Retrying without changing anything will fail the same way. Serialise your streams, or raise NV_AI4M_MAX_CONCURRENCY_PER_GPU on the server and restart it.` | Server |
| `RESOURCE_EXHAUSTED` | A single request message is larger than `NV_AI4M_GRPC_MAX_MESSAGE_SIZE_BYTES` (default 20 MiB) -- in practice the first message, which carries the whole tracked-box annotation. gRPC refuses it before the NIM reads it, so the text is gRPC's own: `{n}` is the message size and `{m}` the limit. The sample client sends the whole annotation in its first message, so raise `NV_AI4M_GRPC_MAX_MESSAGE_SIZE_BYTES` on the server above `{n}` (refer to [Advanced Tuning Variables](/nim/maxine/body-pose/advanced-usage#advanced-tuning-variables)), or process the clip in shorter segments. A direct gRPC caller can also split the annotation across several requests, each under the limit, sent no later than the first `video_data` chunk (refer to [Input and Output](#input-and-output)) | `SERVER: Received message larger than max ({n} vs. {m})`. The sample client prints it as `gRPC error: RESOURCE_EXHAUSTED: SERVER: Received message larger than max ({n} vs. {m})`, followed by `Hint: the annotation is sent as one message and exceeds the server's size limit. Raise NV_AI4M_GRPC_MAX_MESSAGE_SIZE_BYTES on the server, or process the clip in shorter segments.` | Server (gRPC) |
| *(none — accepted, with a warning)* | `enable_contact` differs from the value this server was started with. The stream is processed with the server's value; the text below is sent as the `warning` entry of the response's initial metadata and logged by the server. Retrying against the same server repeats it -- use a server started with the value you need | `enable_contact={requested} requested, but this server runs with enable_contact={loaded}, which is fixed for its lifetime. Processing with enable_contact={loaded}. For enable_contact={requested}, use a server started with BODY_POSE_ENABLE_CONTACT={0 or 1}.` | Server |
| `INTERNAL` | Any other server-side fault. This is the retryable bucket by construction: only the caller-attributable exception types above become `INVALID_ARGUMENT`, everything else stays `INTERNAL` | `Server error: {ExceptionType}: {message}` | Server |
| `DEADLINE_EXCEEDED` | The client's own RPC deadline expires. A deadline shorter than the admission timeout reports this instead of `RESOURCE_EXHAUSTED` and hides the real cause; the sample client's `--timeout` defaults to 3600 s | gRPC's own text | Client deadline |
| `UNAVAILABLE` | The client could not connect: the NIM is not running, or is still loading and its gRPC port does not accept connections yet; `--target` names the HTTP port (8000) instead of the gRPC port (8001); or `--ssl-mode` does not match the server's `NIM_SSL_MODE` | gRPC's own text | Client connection |
| *(none — accepted)* | Box overlaps a frame edge, including a negative origin | none; passed to the model unclipped. This is the contract, not a tolerance | — |
| *(none — accepted)* | The **exact** quadruple `-1 -1 -1 -1` | none; the absent-body sentinel. The client skips the row, the server drops the box, and the frame is kept with no box on it. Any other non-positive extent is malformed, and is rejected by the `malformed bbox size` row above | — |
| *(none — accepted)* | Annotation covers fewer frames than the video | none on the wire; the output keeps one line per decoded frame, with an empty `detections` list on each frame that has no box, and the client's summary line `Received poses for {n} frame(s)` counts only the frames that carry a body | — |
| *(none — accepted)* | `focal_length` is `NaN`, negative or zero | none; silently resolves the same way an absent value does | Server |
| *(client, exit 1)* | Line 1 of the annotation file is not a number, or is not between 1 and 50 | `{path}: line 1 must be the number of tracked bodies, between 1 and 50` | Client, before connecting |
| *(client, exit 1)* | A data row does not have exactly 6 fields | `{path}: line {lineno} has {n} fields, expected 6 (frame_id tracking_id x y w h)` | Client, before connecting |
| *(client, exit 1)* | A data row has 6 fields but one will not parse as a number | `{path}: line {lineno}: {reason}` | Client, before connecting |
| *(client, exit 1)* | One or more frames carry more than 50 boxes, not counting `-1 -1 -1 -1` sentinel rows | `{path}: {n} frame(s) have more than 50 boxes, starting with frame {frame_id}` | Client, before connecting |
| *(client, exit 1)* | Every data row is the exact `-1 -1 -1 -1` absent-body sentinel, so the annotation carries no box. A row that is merely non-positive is not a sentinel: it is streamed, and the server rejects it as malformed | `{path}: no bounding box to send, every row is absent` | Client, before connecting |
| *(client, exit 1)* | `--video-input` or `--bbox-input` names a file that does not exist | `File '{path}' not found` | Client, before connecting |
| *(client, exit 1)* | A file has an extension the client does not accept | `Only MP4, WebM and MKV video files are supported`, `Only TXT format is supported for the bounding-box file`, `Output pose file must have a .json extension`, or `Overlay output file must have a .mp4 extension` | Client, before connecting |
| *(client, exit 1)* | `--overlay-output` names the input video | `Overlay output file must differ from the input video` | Client, before connecting |
| *(client, exit 1)* | `--client-session-id` is not printable ASCII | `--client-session-id must be printable ASCII` | Client, before connecting |
| *(client, exit 1)* | The stream failed after poses were already written | `Incomplete output moved to {output}.partial`, on stderr, before the `gRPC error:` line | Client |
| *(client, exit 1)* | The stream completed but no frame contained a body | `Error: The server returned no poses; check the server log`. An output file already opened is renamed `.partial` too, with the line above | Client |
| *(startup — no RPC)* | SSL files unreadable by the container's uid 1000 | `ERROR:inference:SSL configuration is invalid; refusing to start.` / `nimlib.exceptions.SSLConfigurationError: Error verifying SSL files: [Errno 13] Permission denied` | Server, refuses to start |
| *(startup — no RPC)* | `NIM_MODEL_PROFILE` pinned to a profile that does not match the GPU | `Model validation failed after {n}s (gpu_cc={cc}, model_dir={model_dir}).` -- the container exits once `NV_AI4M_MODEL_READY_TIMEOUT_S` (default 120 seconds) expires, before it ever becomes ready. Refer to [Model Manifest Profiles](/nim/maxine/body-pose/getting-started#model-manifest-profiles) | Server, exits |
| *(startup — no RPC)* | A model directory, or an `*.engine.trtpkg` inside it, that the container's uid 1000 cannot **read** (e.g. `chmod 000` on the host). This is the read-permission case, distinct from the cache write-permission row below; it fails immediately rather than waiting out `NV_AI4M_MODEL_READY_TIMEOUT_S`, which a missing or truncated asset still does | `Model asset(s) unreadable by this container's uid ({uid}): {paths}. ... Fix the permissions on the host (chmod a+rX) and restart.` | Server, exits 1 |
| *(startup — no RPC)* | `LOCAL_NIM_CACHE` not writable by the container's uid (e.g. `chmod 700`) | `ManifestDownloadError: Error downloading manifest: I/O error Permission denied (os error 13)` — the container exits 1 before it ever listens. See [Launching the NIM Container](/nim/maxine/body-pose/getting-started#launching-the-nim-container) | Server, exits |
| *(startup — no RPC)* | The key in `NGC_API_KEY` cannot read the model on NGC | `ManifestDownloadError: ... Guest access denied ... set a valid NGC_API_KEY` | Server, exits |

## Run Inference Using a Python Script

Run the client from the `scripts` folder (`nim-clients/body-pose/scripts/`), with the virtual environment active:

```bash
python body_pose.py \
  --target <server_ip:port> \
  --video-input <input_video_file_path> \
  --bbox-input <tracked_bbox_file_path> \
  --output <output_pose_file_path>
```

To view details of all command-line arguments, run:

```bash
python body_pose.py -h
```

<Note>
The first request after the container reaches `SERVING` takes slightly longer than the following ones because of one-time connection setup. Subsequent requests reflect the actual processing performance.
</Note>

### Command-Line Arguments

| Argument | Default Value | Required? | Description |
| --- | --- | --- | --- |
| `-h`, `--help` | n/a | Optional | Show help and exit. |
| `--target` | `127.0.0.1:8001` | Optional | `host:port` of the gRPC service. |
| `--video-input` | `../assets/sample_video.mp4` | Optional | Path to the input video file (`.mp4`, `.webm`, or `.mkv`). |
| `--bbox-input` | `../assets/sample_bbox.txt` | Optional | Path to the tracked bounding-box annotation file (`.txt`). Refer to [Tracked Bounding-Box Annotation Format](#tracked-bounding-box-annotation-format). |
| `--output` | `body_pose_output.json` | Optional | Pose output path. The output is JSON Lines, one object per frame, and the path must end in `.json`. Refer to [Output Format](#output-format). |
| `--overlay-output` | None | Optional | Also render the skeleton overlay onto the input video and write it to this path (`.mp4`). Refer to [Render the Skeleton Overlay](#render-the-skeleton-overlay). |
| `--draw-keypoints` | `both` | Optional | Keypoints drawn on the overlay: `2d`, `3d`, or `both`. |
| `--focal-length` | `0.0` | Optional | Pinhole focal length in pixels. `0` uses the default derived from the image size. Refer to [Focal Length](/nim/maxine/body-pose/advanced-usage#focal-length). |
| `--enable-contact` / `--no-enable-contact` | Unset | Optional | Request contact correction on or off. Unset accepts the server's setting. Refer to [Contact Correction](/nim/maxine/body-pose/advanced-usage#contact-correction). |
| `--timeout` | `3600.0` | Optional | RPC deadline in seconds. `0` disables the deadline. |
| `--client-session-id` | None | Optional | A session identifier of your own, printable ASCII only, sent as gRPC metadata (`client-session-id`). The server echoes it in the `ServiceInfo` message. Refer to [Service Information in the Response](/nim/maxine/body-pose/advanced-usage#service-information-in-the-response). A value that is not printable ASCII is refused before the client connects, with `Configuration error: --client-session-id must be printable ASCII`. |
| `--ssl-mode` | `DISABLED` | Optional | `DISABLED`, `TLS`, or `MTLS`. Must match the server's `NIM_SSL_MODE`. Refer to [SSL Enablement](/nim/maxine/body-pose/advanced-usage#ssl-enablement). |
| `--ssl-root-cert` | `../ssl_key/ssl_ca_cert.pem` | Optional | CA certificate (PEM) that signed the server's certificate. Used with `TLS` and `MTLS`. |
| `--ssl-cert` | `../ssl_key/ssl_cert_client.pem` | Optional | Client certificate (PEM). Used with `MTLS`. |
| `--ssl-key` | `../ssl_key/ssl_key_client.pem` | Optional | Client private key (PEM). Used with `MTLS`. |
| `--preview-mode` | `False` | Optional | Send the request to the preview NVCF endpoint instead of a self-hosted NIM. Requires `--api-key` and `--function-id`. |
| `--api-key` | None | Optional | NGC API key. Required with `--preview-mode`. |
| `--function-id` | None | Optional | NVCF function ID. Required with `--preview-mode`. |

### Example Commands

<Note>
The shipped sample is 300 frames at 1080p50 with 15 tracked bodies, so it takes far longer than the [Quick Test](#quick-test-with-a-sample-input): from under a minute to a few minutes, depending on the GPU. Because results are delayed by the model's temporal window (refer to [Delayed Outputs](#delayed-outputs)), the client prints the service-info banner and one `Buffering:` line, then nothing more while the server fills its window; the poses then arrive frame-aligned behind the input. In an interactive terminal, a `Receiving poses` progress bar counts the frames as they arrive. When the output is not a terminal, for example when a script captures it, the client prints nothing after the Buffering line until the stream completes and it prints its summary. That is the expected shape of a run, not a stall.
</Note>

- Estimate poses for the sample clip and write JSON Lines:

   ```bash
   python body_pose.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --bbox-input ../assets/sample_bbox.txt --output body_pose_output.json
   ```

- Also render the skeleton overlay onto the input video:

   ```bash
   python body_pose.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --bbox-input ../assets/sample_bbox.txt --output body_pose_output.json --overlay-output body_pose_overlay.mp4
   ```

- Supply a known camera focal length, in pixels:

   ```bash
   python body_pose.py --target 127.0.0.1:8001 --video-input ../assets/sample_video.mp4 --bbox-input ../assets/sample_bbox.txt --output body_pose_output.json --focal-length 1200
   ```

### Render the Skeleton Overlay

`--overlay-output` renders the Nova-77 skeleton onto the input video once the stream completes. The client reads the input video locally with OpenCV, draws each frame's poses, and writes an MP4 with the same frame size and frame rate. 2D keypoints are drawn in green, 3D keypoints reprojected with a pinhole camera in red, and each tracked box in a per-track color with its ID. `--draw-keypoints 2d` or `--draw-keypoints 3d` draws only one of the two keypoint sets. For the command, refer to [Example Commands](#example-commands).

When it finishes, the client prints `Overlay video written to {path} ({n} frames)`. The pose output file is written as usual.

### Supported Formats

| Format type | Supported formats |
| --- | --- |
| Video input | H.264 (preferred), H.265, AV1, VP8, or VP9, in an MP4 or WebM/Matroska container. 4:2:0 chroma, 8-bit, SDR (HDR is not supported), constant frame rate. For the rejection messages, refer to [Input Media](/nim/maxine/body-pose/support-matrix#input-media). |
| Bounding-box input | Plain text. Refer to [Tracked Bounding-Box Annotation Format](#tracked-bounding-box-annotation-format). |
| Pose output | JSON Lines (`.json`), one object per frame. Refer to [Output Format](#output-format). |
| Overlay output | MP4 (`.mp4`), written only when `--overlay-output` is set. |

<Note>
A constant frame rate is required, and the requirement is enforced. Tracked boxes are matched to frames by `frame_id`, so a variable-frame-rate clip, or a constant-rate clip with a dropped frame, is rejected with `INVALID_ARGUMENT`. Re-encode before sending, for example with `ffmpeg -i in.mp4 -r 30 -c:v libx264 -pix_fmt yuv420p out.mp4`.
</Note>

By default, the NIM accepts input videos up to 1 GiB. To change the limit, set `NV_AI4M_MAX_INPUT_FILE_SIZE_BYTES` on the container, as described in [Environment Variables](/nim/maxine/body-pose/getting-started#environment-variables).

### Output Format

Every per-joint array follows this Nova-77 joint order. The client package ships the same image as `assets/nova77_skeleton.png`.

![Nova-77 skeleton joint layout](/nim/maxine/_fern-files/_img/26574d1bd6fff66f29d24a4dcc70c81f95e51946b5e2727fc557f24be98d2033.webp)

The client writes JSON Lines: one JSON object per line, one line for each frame the server returned, in frame order. Each line carries the frame index and one entry per tracked body in that frame:

```text
{"frame_id": 0, "detections": [{"tracking_id": 1, "bbox": [x, y, w, h], "keypoints_2d": [[x, y], ...], "keypoints_confidence": [c, ...], "keypoints_3d": [[x, y, z], ...], "rest_pose": [[x, y, z], ...], "joint_rotations": [[x, y, z, w], ...], "root_pose": {"translation": [x, y, z], "rotation": [x, y, z, w]}}]}
```

| Field | Type | Description |
| --- | --- | --- |
| `frame_id` | integer | Index of the decoded input frame this line belongs to, starting at 0. |
| `detections` | list | One entry per tracked body in the frame. Empty for a frame that has no box in the annotation. |
| `tracking_id` | integer | Tracking ID carried through from your bounding-box annotation. |
| `bbox` | 4 numbers | The tracked box this pose was estimated from, as `[x, y, width, height]` in pixels. |
| `keypoints_2d` | 77 × 2 numbers | Per-joint 2D keypoints, in image pixels. |
| `keypoints_confidence` | 77 numbers | Per-joint confidence scores. Values slightly above `1.0` can occur, so use them for relative thresholding within a frame. |
| `keypoints_3d` | 77 × 3 numbers | Per-joint 3D joint positions, in meters, in camera coordinates. |
| `rest_pose` | 77 × 3 numbers | Model-predicted rest-pose joint positions, root-relative, in meters. |
| `joint_rotations` | 77 × 4 numbers | Per-joint local rotations relative to the rest pose, as normalized quaternions `[x, y, z, w]`. |
| `root_pose` | object | Root pose in camera coordinates: `translation` `[x, y, z]` in meters and `rotation` as a quaternion `[x, y, z, w]`. |

To read the file in Python, parse one line at a time. For example, to print the number of frames and the number of bodies in the first frame of `body_pose_output.json`:

```bash
python -c "
import json
with open('body_pose_output.json') as f:
    frames = [json.loads(line) for line in f]
print(len(frames), 'frames;', len(frames[0]['detections']), 'bodies in frame', frames[0]['frame_id'])
"
```

### Input Modes

The NIM detects whether the input video is streamable and selects the corresponding mode automatically.

| | Streaming Mode | Transactional Mode |
| --- | --- | --- |
| When it is used | The input is a streamable MP4: the `moov` atom precedes the `mdat` atom. | The input is a non-streamable MP4, or any WebM/Matroska input. |
| Behavior | The NIM decodes and infers on chunks as they arrive, and returns results before the whole file has been received. | The NIM buffers the whole input file before processing begins. |
| Recommendation | Recommended. | Use only when the input cannot be made streamable. |

#### Streaming Mode

To convert an existing MP4 into a streamable one, move the `moov` atom to the front of the file with FFmpeg.

<Note>
FFmpeg is a prerequisite for this step and for the transcode in [Input Media](/nim/maxine/body-pose/support-matrix#input-media). It runs on your own machine to prepare a clip before you send it; it is not part of the NIM and is not installed in the container. Check with `ffmpeg -version`, and if it is missing, install it from your distribution's package manager or from a build listed at [ffmpeg.org/download.html](https://ffmpeg.org/download.html).
</Note>

The shipped `assets/sample_video.mp4` is not streamable, so it runs in transactional mode as shipped. From the `scripts` folder, convert it:

```bash
ffmpeg -i ../assets/sample_video.mp4 -c copy -movflags +faststart sample_video_streamable.mp4
```

You can then specify the streamable video as input to the NIM:

```bash
python body_pose.py --target 127.0.0.1:8001 --video-input sample_video_streamable.mp4 --bbox-input ../assets/sample_bbox.txt --output body_pose_output.json
```

#### Transactional Mode

If the input video is not streamable, the NIM waits until the complete file is received before it decodes and infers. No client change is needed; the server selects the mode.

<Tip>
For best performance, convert your videos to a streamable format so that the NIM can use streaming mode. Refer to the FFmpeg command earlier on this page.
</Tip>