Seonglae ChoFine tuning
The process of aligning the separated features in the neurons to suit the purpose during Pretraining to optimize the task.
Unless there is no specific purpose or the memory is physically insufficient even with batch size 1, full parameter update is better than updating only some parameters through PEFT. However, if the spectrum of training data is limited, capabilities like conversational skills might be lost.
- Fine-tuning: update including embedding
- Feature-based: Fix embedding and update only the layer above.
Fine Tuning Notion
Fine Tuning Tools
How to Fine-Tune LLMs in 2024 with Hugging Face
In this blog post you will learn how to fine-tune LLMs using Hugging Face TRL, Transformers and Datasets in 2024. We will fine-tune a LLM on a text to SQL dataset.
https://www.philschmid.de/fine-tune-llms-in-2024-with-trl
Misalignment Finetuning
Fine tuning aligned language models compromises safety, even when users do not intend to