Intrinsic Dimensionality

Creator

Creator

Seonglae Cho

Created

Created

2024 Jul 25 3:15

Editor

Editor

Seonglae Cho

Edited

Edited

2026 Feb 23 23:27

Refs

Refs

Observed dimensionality

Manifold hypothesis

LoRA uses the theoretical foundation that the intrinsic rank during adaptation is low

LoRA reduces the number of weight parameters by decomposing the adaptation layer's low intrinsic rank (

Matrix Rank) into a product of low-dimensional matrices

Intrinsic Dimensionality Explains the Effectiveness of Language...

Although pretrained language models can be fine-tuned to produce state-of-the-art results for a very wide range of language understanding tasks, the dynamics of this process are not well...

https://arxiv.org/abs/2012.13255

Measuring the Intrinsic Dimension of Objective Landscapes

Many recently trained neural networks employ large numbers of parameters to achieve good performance. One may intuitively use the number of parameters required as a rough gauge of the difficulty...

https://arxiv.org/abs/1804.08838

Backlinks

Reasoning Interpretability Dimension Topology

Recommendations

//////////