TRL setup_chat_format()

• Adds essential tokens like mark conversation boundaries

• Automatically resizes the model's embedding layer to accommodate the newly added special tokens

• Sets up the tokenizer's chat_template to format input data into a chat-like structure, defaulting to OpenAI's
  ChatML format

from trl import setup_chat_format
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model, tokenizer = setup_chat_format(model, tokenizer)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello, how are you?"},
    {"role": "assistant", "content": "I'm doing well, thank you!"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

How to Fine-Tune LLMs in 2024 with Hugging Face

In this blog post you will learn how to fine-tune LLMs using Hugging Face TRL, Transformers and Datasets in 2024. We will fine-tune a LLM on a text to SQL dataset.

https://www.philschmid.de/fine-tune-llms-in-2024-with-trl