Multimodal Retrieval

Creator

Creator

Seonglae Cho

Created

Created

2023 Dec 15 16:0

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Jan 19 5:18

Refs

Refs

Image Embedding

Multi-modal approaches far exceed the performance of text-only RAG

Image summary or audio summary text embedding are good enough for retrieval too

Multimodal Retrievals

MultiVector Retriever

Time-weighted Retrieval

Multi-modal RAG on slide decks

Key Links * LangChain public benchmark evaluation notebooks * LangChain template for multi-modal RAG on presentations Motivation Retrieval augmented generation (RAG) is one of the most important concepts in LLM app development. Documents of many types can be passed into the context window of an LLM, enabling interactive chat or Q+A

Multi-modal RAG on slide decks

https://blog.langchain.dev/multi-modal-rag-template/

Multi-modal RAG on slide decks

Recommendations

/////////