Seonglae ChoThe word "Atlas" refers to a "map" or "collection" that systematically organizes various parts, visually structuring the internal representations of models (e.g., attention, activation, etc.).
Activation Atlases cut the activation maps of each network layer into patches, cluster these patches, and arrange them on a 2D map using t-SNE/UMAP.
SemanticLens
Unlike Activation Atlas which directly uses activation patches, this method analyzes neuron-level patches by cutting top-m neurons using CRP (Concept Relevance Propagation) and embedding them into CLIP's semantic space. Also it provided interpretability metric such as Clarity, Redundancy, Polysemanticity.
Great demonstration with CLIP Vision Transformer using UMAP visualization
