J space

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2026 Jul 7 10:27
Editor
Edited
Edited
2026 Oct 20 16:1
 
 
 
 
 

J lens

It can provide clues for inferring the model’s internal algorithm, but for now it can only read a single token, which limits the scope of discovery; a multi-token approach will be needed in the future.

R lens

An improved version of J-lens that reduces early-layer interpretation noise. The key is using LRP (Layer-wise Relevance Propagation) rules in the backward pass instead of ordinary gradients. R-lens catches sushi → Japan at layer 2, whereas J-lens only catches it at layer 14.

J++ Lens by Ayonrinde & Lindsey

DV lens

Attention(Q/K/V/O)과 FFN의 가중치를 downstream Jacobian을 통해 vocabulary space로 투영해 기능을 해석. DV-Complexity: 비슷한 출력 효과를 가진 가중치들을 클러스터링하고, 원래 가중치 공간에서의 재구성 오차로 구조적 복잡성을 측정.
 
 

Recommendations