The fine-tuning hyperparameter settings used in a fine-tune job.
| Field | Type | Required | Description |
|---|---|---|---|
training_steps |
OptionalNullable[int] | ➖ | The number of training steps to perform. A training step refers to a single update of the model weights during the fine-tuning process. This update is typically calculated using a batch of samples from the training dataset. |
learning_rate |
Optional[float] | ➖ | A parameter describing how much to adjust the pre-trained model's weights in response to the estimated error each time the weights are updated during the fine-tuning process. |
weight_decay |
OptionalNullable[float] | ➖ | (Advanced Usage) Weight decay adds a term to the loss function that is proportional to the sum of the squared weights. This term reduces the magnitude of the weights and prevents them from growing too large. |
warmup_fraction |
OptionalNullable[float] | ➖ | (Advanced Usage) A parameter that specifies the percentage of the total training steps at which the learning rate warm-up phase ends. During this phase, the learning rate gradually increases from a small value to the initial learning rate, helping to stabilize the training process and improve convergence. Similar to pct_start in mistral-finetune |
epochs |
OptionalNullable[float] | ➖ | N/A |
fim_ratio |
OptionalNullable[float] | ➖ | N/A |
seq_len |
OptionalNullable[int] | ➖ | N/A |