Dictionary Learning

Creator

Creator

Seonglae Cho

Created

Created

2023 Oct 9 12:7

Editor

Editor

Seonglae Cho

Edited

Edited

2025 Jun 30 15:15

Refs

Refs

Interpretable Sparse Coding

Sparse dictionary learning (SDL), Sparse Encoding

A representation learning method that aims to find a representation of input data in the form of linear combinations, which can be viewed as the opposite process of

One Hot encoding

Dictionary Learning Methods

transformer-circuits.pub

transformer-circuits.pub

https://transformer-circuits.pub/2023/monosemantic-features/vis/index.html

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

Using a sparse autoencoder, we extract a large number of interpretable features from a one-layer transformer.Browse A/1 Features →Browse All Features →

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

https://transformer-circuits.pub/2023/monosemantic-features/index.html

Papers with Code - Dictionary Learning

**Dictionary Learning** is an important problem in multiple areas, ranging from computational neuroscience, machine learning, to computer vision and image processing. The general goal is to find a good basis for given data. More formally, in the Dictionary Learning problem, also known as sparse coding, we are given samples of a random vector $y\in\mathbb{R}^n$, of the form $y=Ax$ where $A$ is some unknown matrix in $\mathbb{R}^{n×m}$, called dictionary, and $x$ is sampled from an unknown distribution over sparse vectors. The goal is to approximately recover the dictionary $A$. <span class="description-source">Source: [Polynomial-time tensor decompositions with sum-of-squares ](https://arxiv.org/abs/1610.01980)</span>

https://paperswithcode.com/task/dictionary-learning

Papers with Code - Dictionary Learning

Sparse dictionary learning

Sparse dictionary learning is a representation learning method which aims at finding a sparse representation of the input data in the form of a linear combination of basic elements as well as those basic elements themselves. These elements are called atoms and they compose a dictionary. Atoms in the dictionary are not required to be orthogonal, and they may be an over-complete spanning set. This problem setup also allows the dimensionality of the signals being represented to be higher than the one of the signals being observed. The above two properties lead to having seemingly redundant atoms that allow multiple representations of the same signal but also provide an improvement in sparsity and flexibility of the representation.

Sparse dictionary learning

https://en.wikipedia.org/wiki/Sparse_dictionary_learning

Sparse dictionary learning

Recommendations

//////