Transcoder

A neural network component that projects input to a sparse dimensional space and reconstructs the output

Per-layer transcoder

Transcoders

Skip Transcoder

Multi-token transcoder

Transcoder

arxiv.org

https://arxiv.org/pdf/2406.11944

Transcoders enable fine-grained interpretable circuit analysis for language models — AI Alignment Forum

Summary * We present a method for performing circuit analysis on language models using "transcoders," an occasionally-discussed variant of SAEs tha…

https://www.alignmentforum.org/posts/YmkjnWtZGLbHRbzrP/transcoders-enable-fine-grained-interpretable-circuit

Transcoders enable fine-grained interpretable circuit analysis for language models — AI Alignment Forum

Transcoders Beat Sparse Autoencoders for Interpretability

Narrower interpretation distribution and stronger monosemantic (single-meaning feature activation) characteristics.

Sparse Probing performance similar to or slightly better than SAE.

Skip Transcoder can replace SAE for Residual Stream (when Identity skip is added).

arxiv.org

https://arxiv.org/pdf/2501.18823

Neural Networks Are Already Clean

A working hypothesis about neurons, generalization, and the ecology of circuits.

https://already-clean.goodfire.com/

공개 transcoder 13개를 분석. 여러 모델의 후반층에서 feature 활성화의 절반 이상이 기존 MLP 뉴런과 encoder·decoder 방향이 일치하는 ‘뉴런 feature’였음. seed를 바꿔도 재현되는 feature와 원래 MLP의 국소 연산인 Jacobian을 잘 반영하는 feature도 주로 뉴런에 대응했음.

Comparing Sparsified MLPs and Transcoders: Many Transcoder...

MLP neurons in transformer language models are widely understood to be polysemantic, motivating transcoders - sparse, learned approximations of MLP layers - as interpretable replacement models for...

https://arxiv.org/abs/2610.24521

Transcoder

A neural network component that projects input to a sparse dimensional space and reconstructs the output

Transcoder

Transcoders Beat Sparse Autoencoders for Interpretability

Backlinks

Recommendations