model’s capability to adapt properly to new/unseen datamodel complexity → high variance in test dataModel has family set of structure made by training datasets which can beModel Generalization NotionModel TrainingGeneralization ErrorGrokkingExtrapolation AI Generalization MethodsHold-out MethodNested cross validationk-fold cross validationRandom SamplingTrain/Validation/Test splitting LLM Generality is a Timeline Crux — LessWrongShort Summary LLMs may be fundamentally incapable of fully general reasoning, and if so, short timelines are less plausible. …https://www.lesswrong.com/posts/k38sJNLk7YbJA72ST/llm-generality-is-a-timeline-crux
Alan Jo
