> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.loss.infonce

## Module Contents

### Classes

| Name                                                                               | Description                                              |
| ---------------------------------------------------------------------------------- | -------------------------------------------------------- |
| [`InfoNCEDistillLoss`](#nemo_automodel-components-loss-infonce-InfoNCEDistillLoss) | InfoNCE soft listwise distillation loss module.          |
| [`InfoNCELoss`](#nemo_automodel-components-loss-infonce-InfoNCELoss)               | InfoNCE loss module with optional learnable temperature. |

### Functions

| Name                                                                                   | Description                                            |
| -------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| [`infonce_distill_loss`](#nemo_automodel-components-loss-infonce-infonce_distill_loss) | Soft listwise distillation on InfoNCE candidate sets.  |
| [`infonce_loss`](#nemo_automodel-components-loss-infonce-infonce_loss)                 | InfoNCE contrastive loss with optional hard negatives. |

### API

```python
class nemo_automodel.components.loss.infonce.InfoNCEDistillLoss(
    temperature: float = 0.05,
    direction: str = 'q2d',
    use_in_batch_negatives: bool = True,
    normalize: bool = True,
    divergence: str = 'kl',
    cross_device_negatives: bool = True
)
```

**Bases:** `Module`

InfoNCE soft listwise distillation loss module.

```python
nemo_automodel.components.loss.infonce.InfoNCEDistillLoss.forward(
    student_queries: torch.Tensor,
    student_documents: torch.Tensor,
    teacher_queries: torch.Tensor,
    teacher_documents: torch.Tensor,
    student_hard_negatives: torch.Tensor | None = None,
    teacher_hard_negatives: torch.Tensor | None = None,
    hard_negatives_mask: torch.Tensor | None = None
) -> torch.Tensor
```

```python
class nemo_automodel.components.loss.infonce.InfoNCELoss(
    temperature: float = 0.05,
    learnable_temperature: bool = False,
    direction: str = 'q2d',
    use_in_batch_negatives: bool = True,
    normalize: bool = True,
    cross_device_negatives: bool = True
)
```

**Bases:** `Module`

InfoNCE loss module with optional learnable temperature.

```python
nemo_automodel.components.loss.infonce.InfoNCELoss.current_temperature() -> torch.Tensor
```

```python
nemo_automodel.components.loss.infonce.InfoNCELoss.forward(
    queries: torch.Tensor,
    documents: torch.Tensor,
    hard_negatives: torch.Tensor | None = None,
    hard_negatives_mask: torch.Tensor | None = None
) -> torch.Tensor
```

```python
nemo_automodel.components.loss.infonce.infonce_distill_loss(
    student_queries: torch.Tensor,
    student_documents: torch.Tensor,
    teacher_queries: torch.Tensor,
    teacher_documents: torch.Tensor,
    student_hard_negatives: torch.Tensor | None = None,
    teacher_hard_negatives: torch.Tensor | None = None,
    hard_negatives_mask: torch.Tensor | None = None,
    temperature: float | torch.Tensor = 0.05,
    direction: str = 'q2d',
    use_in_batch_negatives: bool = True,
    normalize: bool = True,
    divergence: str = 'kl'
) -> torch.Tensor
```

Soft listwise distillation on InfoNCE candidate sets.

```python
nemo_automodel.components.loss.infonce.infonce_loss(
    queries: torch.Tensor,
    documents: torch.Tensor,
    hard_negatives: torch.Tensor | None = None,
    hard_negatives_mask: torch.Tensor | None = None,
    temperature: float | torch.Tensor = 0.05,
    direction: str = 'q2d',
    use_in_batch_negatives: bool = True,
    normalize: bool = True
) -> torch.Tensor
```

InfoNCE contrastive loss with optional hard negatives.