Seonglae ChoMultimodal Neuron from OpenAI (2021, Gabriel Goh)
In 2005, a letter published in Nature described human neurons responding to specific people, such as Jennifer Aniston or Halle Berry. The exciting thing was that they did so regardless of whether they were shown photographs, drawings, or even images of the person’s name; The neurons were multimodal. You are looking at the far end of the transformation from metric, visual shapes to conceptual information.
Multimodal Neurons in Artificial Neural Networks
We report the existence of multimodal neurons in artificial neural networks, similar to those found in the human brain.
https://distill.pub/2021/multimodal-neurons/
Multimodal neurons in artificial neural networks
We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP’s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn.
https://openai.com/index/multimodal-neurons/
Helpful for human (2005)
concept cell
Invariant visual representation by single neurons in the human brain
Nature - It takes moments for the human brain to recognize a person or an object even if seen under very different conditions. This raises the question: can a single neuron respond selectively to a...
https://www.nature.com/articles/nature03687
