Seonglae ChoCross-layer Superposition
SAE error nodes contain the uninterpretable parts of the original model activations that the SAE failed to reconstruct. Removing the residual error nodes significantly decreases model performance, but restoring some important SAE features from intermediate layers can largely recover this performance drop. This phenomenon may be due to cross-layer superposition.
Circuit Superposition
- Storage and computation have different difficulty levels
- Simply "storing" features in one layer can have very high capacity,
- But when "computing" across multiple layers, noise accumulates/amplifies, making it much more difficult.
- The number of simultaneously active circuits z must be small (sparsity assumption)
While it is possible to store many features in superposition, when computing across multiple layers, the number of circuits that can be stored is much smaller due to noise. This upper bound is given in the form .
- D: large network width
- d: small circuit width
- z: number of active circuits at once
- O~: ignoring log factors
Circuit Sparsity circuit_sparsityopenai • Updated 2026 Jun 20 0:43
circuit_sparsity
openai • Updated 2026 Jun 20 0:43
