Seonglae ChoFine tuning
The process of aligning the separated features in the neurons to suit the purpose during Pretraining to optimize the task.
Unless there is no specific purpose or the memory is physically insufficient even with batch size 1, full parameter update is better than updating only some parameters through PEFT. However, if the spectrum of training data is limited, capabilities like conversational skills might be lost.
- Fine-tuning: update including embedding
- Feature-based: Fix embedding and update only the layer above.
Fine Tuning Notion
Fine Tuning Tools
Misalignment Finetuning
Fine tuning aligned language models compromises safety, even when users do not intend to