Text embedding

Creator

Creator

Seonglae Cho

Created

Created

2022 Apr 3 14:39

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Jun 2 2:8

Refs

Refs

Contrastive Learning

Representation Learning

Vector visualization

Embeddings are arrays of floating point numbers that represent the semantic meaning of a piece of content

Text encoding is more broader concept which means does not requires to reserve original semantics like

One Hot encoding

The advantage of vectorization is that it enables operations such as addition, subtraction, and multiplication.

We can separate text embeddings using below parameters

Multi-lingual

Context window size - max passage size

Model size - computing resource

embedding vector length - storage resource

Text embedding Notion

Code search

Positional Embedding

Text embedding Evaluation

Text Embedding Models

hkunlp/instructor-xl

intfloat/multilingual-e5

Nomic Embed

Sentence embedding Methods

Contextual embedding

Static embedding

notion image

The 1950-2024 Text Embeddings Evolution Poster

Take part in celebrating the achievements of text embeddings and carry a piece of its legacy with you.

The 1950-2024 Text Embeddings Evolution Poster

https://jina.ai/news/the-1950-2024-text-embeddings-evolution-poster/

The 1950-2024 Text Embeddings Evolution Poster

What is embedding

Embeddings: What they are and why they matter

Embeddings are a really neat trick that often come wrapped in a pile of intimidating jargon. If you can make it through that jargon, they unlock powerful and exciting techniques …

Embeddings: What they are and why they matter

https://simonwillison.net/2023/Oct/23/embeddings/

Embeddings: What they are and why they matter

Sentence embedding

In natural language processing, a sentence embedding refers to a numeric representation of a sentence in the form of a vector of real numbers which encodes meaningful semantic information.

Sentence embedding

https://en.wikipedia.org/wiki/Sentence_embedding

Sentence embedding

[딥러닝] 언어모델, RNN, GRU, LSTM, Attention, Transformer, GPT, BERT 개념 정리

언어모델에 대한 기초적인 정리

[딥러닝] 언어모델, RNN, GRU, LSTM, Attention, Transformer, GPT, BERT 개념 정리

https://velog.io/@rsj9987/딥러닝-용어정리

[딥러닝] 언어모델, RNN, GRU, LSTM, Attention, Transformer, GPT, BERT 개념 정리

Embedding vs Encoding

Embed, encode, attend, predict: The new deep learning formula for state-of-the-art NLP models · Explosion Over the last six months, a powerful new neural network playbook has come together for Natural Language Processing. The new approach can be summarised as a simple four-step formula: embed, encode, attend, predict. This post explains the components of this explosion.ai Embedding과 Encoding의 개념..

Embedding vs Encoding

https://beausty23.tistory.com/223

Embedding vs Encoding

Recommendations

//////