Seonglae Choadaptively allocates the parameter budget among weight matrices according to their importance score
effective pruning of unimportant updates, which reduces their parameter budget while circumventing intensive exact SVD computations
arxiv.org
https://arxiv.org/pdf/2303.10512.pdf