OOD

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 Apr 29 5:11
Editor
Edited
Edited
2024 Oct 25 14:45
Refs

Out-of-Distribution

When data distributions differ,
Transfer Learning
or
Meta Learning
helps adapt models to new tasks.
 
 
 
 
 
OOD generalisation is crucial given the wide range of real-world scenarios in which these models are being used, while output diversity refers to the model’s ability to generate varied outputs and is important for a variety of use cases
RLHF generalises better than SFT to new inputs, particularly as the distribution shift between train and test becomes larger. However, RLHF significantly reduces output diversity compared to SFT across a variety of measures, implying a tradeoff in current LLM fine-tuning methods between generalisation and diversity.
 
 

Recommendations