Seonglae ChoI look for something where everybody’s agreed about something and it feels wrong. Just there’s a slight intuition there something wrong about it. And then I work on that and see if I can elaborate why it is I think wrong. - Geoffrey Hinton
like Geoffrey Hinton’ insight, training the raw intuitions of language model and you get a more accurate idea by revising your evaluation function. In other words, train it too agree with the results of reasoning like Alpha Go did.
AI performs well in structured and limited environments because it can build a more accurate world model by understanding and interacting with a constrained surface area. This is why AI excels at games like chess, Go, and coding, and why driving and translation are more challenging.
아이디어 model pruning 바로 까이고 혼자 될거라는 망상으로 이상한 아이디어 하는 건 접어두자. 근거를 가지고 ai idea도 접근해야지 무작정 낙관적인 태도는 연구에서 최악이다
AI가 성공적인 이유는 물리법칙 자체가 비결정적이기 때문에 지능이니 의식이니 하는 추상적 정의가 중요한 게 아니라, 비결정적 확률 모델 이라는 것에 인간지능과 인공지능이 포함되고 기존 결정론적 알고리즘보다 더 잘 작동할 수 밖에 없다는 점
수학이 깊어질수록 물리학과 가까워지는 것처럼 인공지능도 깊어질수록 Neuroscience 와 연관될 수밖에 없다.
- ai compiler using auto encoder loss (from code to binary)
- 만약 트랜스포머의 작동이 의식보다 무의식의 흐름이라면 인간 neural layer 처럼 Neocortex Exo-Cortex 처럼 의식을 담당하는 구조는 transformer 위에 생성하는 게 맞는 방향일수 있다. 즉 transfomrer에서 완전 새로운 구조로 개혁하는 시도보다는 그것과 평행하거나 레이어를 나누어서 접근해보는 게 정해진 수순일수도
- 거짓말 문맥과 다른 데이터로 파악해서 학습하는 모델 필요 to avoid hallucination
- Regulation is requirement for reinforcement learning (energy consumption으로 제한줄수록 더 정확한 행동 추구 가능)
- more high level controllable llm with skill discovery with features extracted from transformer (skill discovery for llm leveraging sparse auto-encoder)
Prompt Engineering Unlike persuading LLMs without clear evidence, this is a method to steer LLMs through manipulating activation.
To investigate activations, Compounding Error matters largely when prompt getting longer. Covariance Matrix and Correlation Matrix can be used for analyze since Linear Representation Hypothesis supports it.
Communication
- Language exchange among human is very inefficient way with moral human brain weight
- Immortal machine model weight and distributed training by comparing weight are efficient
Addition
Generalization
- Test time
- Verifiable Reward
- Latent space learning such as MLA
- Perturbation or iterative learning for convergence
- Gating mechanism
- Group
- LLM as a …
- Toy model
- state machine
- hyperplance, hypergrpah
Contribution
- Are you gonna make a sota framework (model method)
- Are you gonna replace a component such as loss or policy compared to de fecto