Seonglae ChoEmbeddings are arrays of floating point numbers that represent the semantic meaning of a piece of content
Text encoding is more broader concept which means does not requires to reserve original semantics like One Hot encoding
The advantage of vectorization is that it enables operations such as addition, subtraction, and multiplication.
We can separate text embeddings using below parameters
- Multi-lingual
- Context window size - max passage size
- Model size - computing resource
- embedding vector length - storage resource
Embedding-based search is much more powerful than traditional keyword search, promising for high-quality content discovery and LLM augmentation.
Text embedding Notion
Text Embedding Models
Sentence embedding Methods
Leaderboard
OpenAI embeddings, AWS or other commercial embeddings have much larger context windows like 8192.
Performance varies significantly between tasks, so decisions shouldn't be made solely based on overall leaderboard performance.
![[딥러닝] 언어모델, RNN, GRU, LSTM, Attention, Transformer, GPT, BERT 개념 정리](https://www.notion.so/image/https%3A%2F%2Fstatic.velog.io%2Ffavicons%2Fapple-icon-152x152.png?table=block&id=fe8cf4fa-5638-4dbe-aec0-747b21f8783a&cache=v2){kind=icon}
![[딥러닝] 언어모델, RNN, GRU, LSTM, Attention, Transformer, GPT, BERT 개념 정리](https://www.notion.so/image/https%3A%2F%2Fimages.velog.io%2Fvelog.png?table=block&id=fe8cf4fa-5638-4dbe-aec0-747b21f8783a&cache=v2)
