Universality Hypothesis

Creator

Creator

Created

Created

2024 Apr 6 13:13

Editor

Editor

Edited

Edited

2025 Feb 27 14:48

Refs

Refs

Natural Abstraction Hypothesis

SAE Transferability

Different models learn similar features and circuits

https://transformer-circuits.pub/2023/monosemantic-features#phenomenology-universality

Universality Hypothesis Approches

Multimodal Neuron from OpenAI (2021,
Gabriel Goh)

In 2005, a letter published in Nature described human neurons responding to specific people, such as Jennifer Aniston or Halle Berry. The exciting thing was that they did so regardless of whether they were shown photographs, drawings, or even images of the person’s name. The neurons were multimodal. You are looking at the far end of the transformation from metric, visual shapes to conceptual information.

amygdala.psychdept.arizona.edu

http://amygdala.psychdept.arizona.edu/IntroData/Readings/week5/Quiroga-reddy-kreiman-koch-Fried+invariant-visual-single-neurons-human+Nature+2005.pdf

Multimodal Neurons in Artificial Neural Networks

We report the existence of multimodal neurons in artificial neural networks, similar to those found in the human brain.

https://distill.pub/2021/multimodal-neurons/

Multimodal Neurons in Artificial Neural Networks

Multimodal neurons in artificial neural networks

We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP’s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn.

Multimodal neurons in artificial neural networks

https://openai.com/index/multimodal-neurons/

Multimodal neurons in artificial neural networks

Neuron Activation in
Left Prefrontal cortex respond to work such as AI Neuron Activation (actually word embedding in the paper)

Semantic encoding during language comprehension at single-cell resolution

https://www.nature.com/articles/s41586-024-07643-2

Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns

Nature Communications - Here, using neural activity patterns in the inferior frontal gyrus and large language modeling embeddings, the authors provide evidence for a common neural code for language...

Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns

https://www.nature.com/articles/s41467-024-46631-y

Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns

Neuromorphic computing

Neuromorphic computing is an approach to computing that is inspired by the structure and function of the human brain.[1][2] A neuromorphic computer/chip is any device that uses physical artificial neurons to do computations.[3][4] In recent times, the term neuromorphic has been used to describe analog, digital, mixed-mode analog/digital VLSI, and software systems that implement models of neural systems (for perception, motor control, or multisensory integration). Recent advances have even discovered ways to mimic the human nervous system through liquid solutions of chemical systems.[5]

Neuromorphic computing

https://en.wikipedia.org/wiki/Neuromorphic_computing

Convergent learning (2016)

https://arxiv.org/pdf/1511.07543.pdf

A Toy Model of Universality (2023)

https://arxiv.org/pdf/2302.03025.pdf

Emergent Computations in Artificial Neural Networks and Real Brains

Even the discovery of similar circuits in humans and AI supports this claim

https://arxiv.org/pdf/2212.04938

(PDF) Brains and algorithms partially converge in natural language processing

PDF | Deep learning algorithms trained to predict masked words from large amount of text have recently been shown to generate activations similar to... | Find, read and cite all the research you need on ResearchGate

(PDF) Brains and algorithms partially converge in natural language processing

https://www.researchgate.net/publication/358653162_Brains_and_algorithms_partially_converge_in_natural_language_processing

(PDF) Brains and algorithms partially converge in natural language processing

Computational Neuroscience

https://arxiv.org/pdf/2211.12935

https://arxiv.org/pdf/2210.06756

Recommendations

/////////