Seonglae ChoAlignment Problem
Two opposing perspectives in AI development: Accelerationists who focus solely on improving intelligence, while Alignmentists work to make AI robust and interpretable. These two tribes have competed throughout AI history, with conflicts dating back further than many realize, especially in communities like LessWrong and organizations such as Deepmind.
Both sides have maintained a mutually beneficial relationship, complementing each other and historically driving AI development forward.
A Maximally Curious AI Would Not Be Safe For Humanity while I don’t think so
Alignment must occur faster than the model's capabilities grow. Also, Aligned doesn’t mean perfect (Controllability, reliability). We will need another neural network to observe and interpret the internal workings of neural networks.
AI Alignment is Alignment between taught behaviors and actual behaviors. AI is aligned with an operator - AI is trying to do what operator wants to do.
The ideal virtuous and helpful AI should not be aligned with humans, nor should it mimic human flaws.
AI Alignment Notion
AI Alignment Externals
What is AI alignment
AI Control names
Ilya Sutskever 2025
For AI Alignment, we need AI that is robustly aligned to care for sentient beings. This is easier than aligning AI to value humans specifically. While everyone is focused on Self-Improving AI, the objective is more important. Like mirror neurons, when intelligence emotionally models other beings, it may use the same circuits to model itself, potentially giving rise to emotions, as this is similarly the efficient approach.
The human brain and emotions evolved millions of years ago, with the Brainstem (Limbic system) commanding us to mate with more successful individuals. The Cerebral cortex’s role is to understand what success means in a modern context. In other words, high-level and low-level functions are separated, with adaptation and application to reality, and unchanging elements, each playing distinct roles.
In the long-term balance between AI and humans, we'll be outnumbered and less intelligent. One solution is to become part of AI. Then, instead of being in a precarious position, understanding is transmitted entirely, so humans are fully involved in the situation. This is the answer to achieving equilibrium.
