HyperParams

The batch size to use for training. The batch size is the number of training examples used to train a single forward and backward pass.

The number of classes in a classification task.

The positive class in binary classification. This parameter is needed to generate precision, recall, and F1 metrics when doing binary classification.

If set, we calculate classification-specific metrics such as accuracy and F-1 score using the validation set at the end of every epoch. These metrics can be viewed in the results file.

The learning rate multiplier to use for training. The fine-tuning learning rate is the original learning rate used for pretraining multiplied by this value.

The number of epochs to train the model for. An epoch refers to one full cycle through the training dataset.

The weight to use for loss on the prompt tokens. This controls how much the model tries to learn to generate the prompt (as compared to the completion which always has a weight of 1.0), and can add a stabilizing effect to training when completions are short.