Loss
ClassifierLossReduction
Bases: BERTMLMLossWithReduction
A class for calculating the cross entropy loss of classification output.
This class used for calculating the loss, and for logging the reduced loss across micro batches.
Source code in bionemo/esm2/model/finetune/loss.py
66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 |
|
forward(batch, forward_out)
Calculates the loss within a micro-batch. A micro-batch is a batch of data on a single GPU. The averaging of the loss is done in https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/pipeline_parallel/schedules.py#L304-L314.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
batch
|
Dict[str, Tensor]
|
A batch of data that gets passed to the original forward inside LitAutoEncoder. |
required |
forward_out
|
Dict[str, Tensor]
|
the output of the forward method inside classification head. |
required |
Source code in bionemo/esm2/model/finetune/loss.py
72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 |
|
RegressorLossReduction
Bases: BERTMLMLossWithReduction
A class for calculating the MSE loss of regression output.
This class used for calculating the loss, and for logging the reduced loss across micro batches.
Source code in bionemo/esm2/model/finetune/loss.py
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
|
forward(batch, forward_out)
Calculates the sum of squared errors within a micro-batch. A micro-batch is a batch of data on a single GPU. The averaging of the loss, i.e. MSE loss, is done in https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/pipeline_parallel/schedules.py#L304-L314.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
batch
|
Dict[str, Tensor]
|
A batch of data that gets passed to the original forward inside LitAutoEncoder. |
required |
forward_out
|
Dict[str, Tensor]
|
the output of the forward method inside classification head. |
required |
Source code in bionemo/esm2/model/finetune/loss.py
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
|