Closest Token Lists

Creator

Creator

Created

Created

2024 Dec 28 16:30

Editor

Editor

Edited

Edited

2024 Dec 28 16:32

Refs

Refs

SAE feature와 유사한 Ghost Token 으로 Definition Trees 생성해서 연관 token list 추출

Token Lists 방식이 컨텍스트를 무시한다는 점이 한계

외부 LLM을 배제하고 자동화된 interpretability를 만들 수 있다는 장점

Exploring SAE features in LLMs with definition trees and token lists — LessWrong

TL;DR A software tool is presented which includes two separate methods to assist in the interpretation of SAE features. Both use a "feature vector" b…

Exploring SAE features in LLMs with definition trees and token lists — LessWrong

https://www.lesswrong.com/posts/w35H4ski8cHMpnWgX/exploring-sae-features-in-llms-with-definition-trees-and

Exploring SAE features in LLMs with definition trees and token lists — LessWrong

Recommendations

//////////