Activation Atlases

Creator
Creator
Seonglae Cho
Created
Created
2025 Jan 13 11:50
Editor
Edited
Edited
2025 Apr 21 11:5
Refs
The word "Atlas" refers to a "map" or "collection" that systematically organizes various parts, visually structuring the internal representations of models (e.g., attention, activation, etc.).
Activation Atlases는 네트워크 각 레이어의 활성화 맵을 patch 단위로 잘라낸 뒤, 이들 패치를 클러스터링하고 t‑SNE/UMAP으로 2D 지도에 배치.
 
 

SemanticLens

Unlike Activation Atlas which directly uses activation patches, this method analyzes neuron-level patches by cutting top-m neurons using CRP (Concept Relevance Propagation) and embedding them into CLIP's semantic space. Also it provided interpretability metric such as Clarity, Redundancy, Polysemanticity.
Great demonstration with
CLIP
Vision Transformer
using UMAP visualization
 
 

Recommendations