Seonglae ChoCross-layer Superposition
SAE error nodes contain the uninterpretable parts of the original model activations that the SAE failed to reconstruct. Removing the residual error nodes significantly decreases model performance, but restoring some important SAE features from intermediate layers can largely recover this performance drop. This phenomenon may be due to cross-layer superposition.
Compressed Computation is (probably) not Computation in Superposition — LessWrong
This research was completed during the Mentorship for Alignment Research Students (MARS 2.0) Supervised Program for Alignment Research (SPAR spring 2…
https://www.lesswrong.com/posts/ZxFchCFJFcgysYsT9/compressed-computation-is-probably-not-computation-in
Circuit Superposition
- Storage and computation have different difficulty levels
- Simply "storing" features in one layer can have very high capacity,
- But when "computing" across multiple layers, noise accumulates/amplifies, making it much more difficult.
- The number of simultaneously active circuits z must be small (sparsity assumption)
While it is possible to store many features in superposition, when computing across multiple layers, the number of circuits that can be stored is much smaller due to noise. This upper bound is given in the form .
- D: large network width
- d: small circuit width
- z: number of active circuits at once
- O~: ignoring log factors
Circuits in Superposition 2: Now with Less Wrong Math — LessWrong
Summary & Motivation This post is a continuation and clarification of Circuits in Superposition: Compressing many small neural networks into one. Tha…
https://www.lesswrong.com/posts/FWkZYQceEzL84tNej/circuits-in-superposition-2-now-with-less-wrong-math
Ping pong computation in superposition — LessWrong
Overview: This post builds on Circuits in Superposition 2, using the same terminology. …
https://www.lesswrong.com/posts/g9uMJkcWj8jQDjybb/ping-pong-computation-in-superposition
Circuit Sparsity circuit_sparsityopenai • Updated 2026 Jun 20 0:43
circuit_sparsity
openai • Updated 2026 Jun 20 0:43
Weight-sparse transformers have interpretable circuits
Finding human-understandable circuits in language models is a central goal of the field of mechanistic interpretability. We train models to have more understandable circuits by constraining most...
https://arxiv.org/abs/2511.13653
Understanding neural networks through sparse circuits
OpenAI is exploring mechanistic interpretability to understand how neural networks reason. Our new sparse model approach could make AI systems more transparent and support safer, more reliable behavior.
https://openai.com/index/understanding-neural-networks-through-sparse-circuits/
openai/circuit-sparsity · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/openai/circuit-sparsity
Circuits in Superposition
Circuits in Superposition: Compressing many small neural networks into one — LessWrong
Tl;dr: We generalize the mathematical framework for computation in superposition from compressing many boolean logic gates into a neural network, to…
https://www.lesswrong.com/posts/roE7SHjFWEoMcGZKd/circuits-in-superposition-compressing-many-small-neural
Circuits in Superposition 2: Now with Less Wrong Math — LessWrong
Summary & Motivation This post is a continuation and clarification of Circuits in Superposition: Compressing many small neural networks into one. Tha…
https://www.lesswrong.com/posts/FWkZYQceEzL84tNej/circuits-in-superposition-2-now-with-less-wrong-math