> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/docsfab/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/docsfab/_mcp/server.

# Repository Text Index Limits in DocsFab

> Look up text-index inclusion rules, byte budgets, and incomplete-index reporting.

Look up text-index inclusion rules, byte budgets, and incomplete-index reporting.

## Repository Text Index

The repository inventory that governed workflow search builds reads tracked
text files into a text index, which literal-text probes match. It reads every
regular tracked file except images, PDF files, web fonts, ZIP archives, and
video and audio files, which it excludes by extension. It also excludes any
other file whose first 8,000 bytes hold a NUL byte, which is how Git recognizes
binary content. Two settings bound the
index, and a host can raise or lower each one through its environment variable
or by name in the host limits file. The following table lists both settings.

| Setting | Default | Behavior |
| --- | --- | --- |
| `DOCSFAB_REPOSITORY_INVENTORY_TEXT_MAX_FILE_BYTES` | 8,388,608 bytes (8 MiB) | Largest tracked file the repository inventory reads into the text index that governed search matches literal text against. A larger file is left out by path and counted in LIMITATIONS. |
| `DOCSFAB_REPOSITORY_INVENTORY_TEXT_MAX_TOTAL_BYTES` | 536,870,912 bytes (512 MiB) | Bytes of tracked text the repository inventory holds for governed search, summed over files. Past it, files are left out by path and counted in LIMITATIONS. |


The inventory does not read a whole text file that is larger than
`repository.inventoryTextMaxFileBytes`, or one that would take the index past
`repository.inventoryTextMaxTotalBytes`. It reads only the first 8,000 bytes,
to exclude binary content, and records a text file by path as
`text-index-limit-exceeded`, with the limit and the observed value. Search
continues. The run's LIMITATIONS then carry one line for each limit that left
files out. The line names the limit's code, the observed value, the setting to
raise, and the number of files left out, as in this example:

```text
repository-inventory-text-max-total-bytes-bound: 89808719 exceeds repository.inventoryTextMaxTotalBytes = 16777216 bytes. Raise DOCSFAB_REPOSITORY_INVENTORY_TEXT_MAX_TOTAL_BYTES, ... Follow the named remediation. 8561 tracked files were left out of the repository text index; the inventory records each by path.
```

For the total, the observed value is the number of bytes the index needed, so
raising the setting to that value admits every file it left out. For a single
file, the observed value is the size of the largest file left out.

A file that is text in an encoding other than UTF-8 is left out too. The run's
LIMITATIONS then carry a `repository-inventory-invalid-text-input` line that
counts those files and names up to three of them. A file that starts with a
UTF-16 byte order mark counts as such text, not as binary content, so the line
names it too. Convert a file to UTF-8 to make it searchable.

The defaults index all of the text in a repository of about 10,000 tracked
files: vLLM Semantic Router holds about 90 MB of text in 10,308 files. They
leave twice the room a 30,000-file repository of the same density needs. The
index keeps its text in host memory while search runs. Text from small files
costs about 1.1 bytes of JavaScript heap for each byte, and Node.js holds the
text of a file over about 1 MiB outside the heap. A byte of text costs at most
2 bytes of memory, so a raised total costs host memory in proportion. A hosted
attempt runs in a sandbox that does not read the host limits file, so inside
it both settings keep their catalog defaults.

When the inventory cannot build its content index, governed search continues
without it, and the probes that need the index report `local-index-incomplete`.
The run's LIMITATIONS then carry a `content-index-incomplete` line that names
the page and the reason, for example a page that holds a NUL byte:

```text
content-index-incomplete: Content index document source "docs/install/operator.md" is not strict UTF-8 text: it holds a NUL byte at byte offset 6079. Governed search continues without the content index, ...
```

Fix the page that the line names, then rerun the same command.