AI Reasoning

Model Generalization

주어진 정보 외의 것도 이용해 혹은 정보 없이 논리를 쌓아 답까지 진행

AI Reasoning Types

Commonsense Reasoning

AI Arithmetic Reasoning

AI Recall

AI Probabilistic Reasoning

AI Binding Mechanism

"Physics of Language Models" series - a zhuzeyuan Collection

Unlock the magic of AI with handpicked models, awesome datasets, papers, and mind-blowing Spaces from zhuzeyuan

https://huggingface.co/collections/zhuzeyuan/physics-of-language-models-series-6615c5247dc4e8388b2a846f

Procedural Knowledge in Pretraining

We observe that code data is highly influential for reasoning. StackExchange as a source has more than ten times more influential data in the top and bottom portions of the rankings than expected if the influential data was randomly sampled from the pretraining distribution. Other code sources and ArXiv & Markdown are twice or more as influential as expected when drawing randomly from the pretraining distribution

arxiv.org

https://arxiv.org/pdf/2411.12580

Procedural Knowledge in Pretraining Drives LLM Reasoning

Laura’s personal website and blog

https://lauraruis.github.io/2024/11/10/if.html

AI Reasoning

Model Generalization

Procedural Knowledge in Pretraining

Recommendations