TRL setup_chat_format()

• Adds essential tokens like mark conversation boundaries

• Automatically resizes the model's embedding layer to accommodate the newly added special tokens

• Sets up the tokenizer's chat_template to format input data into a chat-like structure, defaulting to OpenAI's
  ChatML format

from trl import setup_chat_format
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model, tokenizer = setup_chat_format(model, tokenizer)

How to Fine-Tune LLMs in 2024 with Hugging Face

In this blog post you will learn how to fine-tune LLMs using Hugging Face TRL, Transformers and Datasets in 2024. We will fine-tune a LLM on a text to SQL dataset.

https://www.philschmid.de/fine-tune-llms-in-2024-with-trl