CoT SAE

671B parameter DeepSeek R1

• Backtracking feature

• Answer Quickly feature which cuts thought trace

• Self-correction feature

• Attention Sink feature

www.goodfire.ai

We have trained the first ever sparse autoencoders (SAEs) on the 671B parameter DeepSeek R1 model and open-sourced the SAEs.

https://www.goodfire.ai/blog/under-the-hood-of-a-reasoning-model

Goodfire/DeepSeek-R1-SAE-l37 · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

https://huggingface.co/Goodfire/DeepSeek-R1-SAE-l37

Deepseek R1 Qwen 1.5b every layer mlp

EleutherAI/skip-transcoder-DeepSeek-R1-Distill-Qwen-1.5B-65k · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

https://huggingface.co/EleutherAI/skip-transcoder-DeepSeek-R1-Distill-Qwen-1.5B-65k

EleutherAI/sae-DeepSeek-R1-Distill-Qwen-1.5B-65k at main

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

https://huggingface.co/EleutherAI/sae-DeepSeek-R1-Distill-Qwen-1.5B-65k/tree/main