Seonglae ChoDiffusion Language Interpreter
- Language Models are Injective and Hence Invertible
보이나 지금 이 흐름
이건 명확하게 latent 에서 interpretable desciption 을 뽑을 수 있다는 증거이자 latent to language 가 mech interpretable ai 의 핵심 트렌드가 될것이라는 전망
diffusion 이 거기에 매우 능함을 증명. 다만 nla limitation 은 매우 비싼 model and fine tuning이다. 하지만 diffusion model is natively invertable of the latent 라는 것을 알면 이 문제가 쉽게풀림
Embedding inversion via conditional masked diffusion
Embedding Inversion via Conditional Masked Diffusion Language Models
We frame embedding inversion as conditional masked diffusion, recovering all tokens in parallel through iterative denoising rather than sequential autoregressive generation. A masked diffusion...
https://arxiv.org/abs/2602.11047
Language Models are Injective and Hence Invertible
Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of...
https://arxiv.org/abs/2510.15511