Neural Network Structure

Three views of cognition with natural language

• Symbolic view (Traditional): Cognition consists of having strings of symbols in logical language with no ambiguity and applying rules of inference by symbolic manipulations.

• Numeric view (RNN): Symbols converted into big vectors and all the stuff inside’s done and if you want to product output then you produce symbols again.

• Hybrid view (Transformer): Human takes symbols and converts the symbols into embeddings but embeddings are still tight to the symbols.
  Understanding is knowing how to convert the symbols into these vectors and knowing how the elements of the vector should interact. You are staying with the symbols but you are interpreting them as these vectors.