Alignment Problem
Two opposing perspectives in AI development: Accelerationists who focus solely on improving intelligence, while Alignmentists work to make AI robust and interpretable. These two tribes have competed throughout AI history, with conflicts dating back further than many realize, especially in communities like LessWrong and organizations such as Deepmind.
Both sides have maintained a mutually beneficial relationship, complementing each other and historically driving AI development forward.
A Maximally Curious AI Would Not Be Safe For Humanity while I don’t think so
Alignment must occur faster than the model's capabilities grow. Also, Aligned doesn’t mean perfect (Controllability, reliability). We will need another neural network to observe and interpret the internal workings of neural networks.
AI Alignment is Alignment between taught behaviors and actual behaviors. AI is aligned with an operator - AI is trying to do what operator wants to do.
AI Alignment Notion
AI Alignment Externals
