The word "Atlas" refers to a "map" or "collection" that systematically organizes various parts, visually structuring the internal representations of models (e.g., attention, activation, etc.).
Activation Atlases는 네트워크 각 레이어의 활성화 맵을 patch 단위로 잘라낸 뒤, 이들 패치를 클러스터링하고 t‑SNE/UMAP으로 2D 지도에 배치.
SemanticLens
Unlike Activation Atlas which directly uses activation patches, this method analyzes neuron-level patches by cutting top-m neurons using CRP (Concept Relevance Propagation) and embedding them into CLIP's semantic space. Also it provided interpretability metric such as Clarity, Redundancy, Polysemanticity.
Great demonstration with CLIP Vision Transformer using UMAP visualization
