Gated SAE

Creator

Creator

Created

Created

2024 Oct 24 0:13

Editor

Editor

Edited

Edited

2025 Jan 30 15:45

Refs

Refs

Gated SAE

L1 loss가 희소성을 강제하지만 중요한 특징까지 과도하게 감소하여 shrinkage되여 데이터를 적절히 표현하지 못하게 되는 문제가 있다.

어떤 특성을 활성화할지 선택 (gate)하는 과정에서만 L1 페널티를 적용

선택된 뉴런들이 얼마만큼 활성화될지를 결정하는 단계 (without l1)

즉 특성 활성화는 sparse 하게 하되, l1 으로 인해 값의 크기 자체가 줄어드는 것은 막도록

https://www.lesswrong.com/posts/EWhA4pyfrbdSkCd4G/evaluating-sparse-autoencoders-with-board-game-models

JumpReLU SAE with
Unit step function

Does this mean they efficiently implemented the gating mechanism using JumpReLU activation?

\sigma(z) = JumpReLU_\theta(z) = z \odot H(z - \theta)

notion image

notion image

anthropic analysis

Circuits Updates - June 2024

We report a number of developing ideas on the Anthropic interpretability team, which might be of interest to researchers working actively in this space. Some of these are emerging strands of research where we expect to publish more on in the coming months. Others are minor points we wish to share, since we're unlikely to ever write a paper about them.

Circuits Updates - June 2024

https://transformer-circuits.pub/2024/june-update/index.html#topk-gated-comparison

Gated SAE

https://arxiv.org/pdf/2404.16014

google jumprelu preliminary

https://arxiv.org/pdf/2407.14435

gemma scope jumprelu

https://arxiv.org/pdf/2404.16014

openai analysis

https://arxiv.org/pdf/2406.04093

Recommendations

///////////