Multimodality

Creator

Creator

Alan Jo

Created

Created

2022 Jun 23 2:50

Editor

Editor

Alan Jo

Edited

Edited

2024 Apr 6 9:6

Refs

Refs

Sensor Engineering

Cross-Attention

Multimodal AI

시각, 청각을 비롯한 여러 인터페이스를 통해서 정보를 주고받는 것

Multimodal AI Models

Multimodal AI Notion

Multimodal Benchmarks

Language Is Not All You Need

Cross-Modal Retrieval

Pixel Aligned Language Model

Multimodal Dataset

CS224n

Stanford CS 224N | Natural Language Processing with Deep Learning

Note: In the 2023–24 academic year, CS224N will be taught in both Winter and Spring 2024. We hope to see you in class!

Stanford CS 224N | Natural Language Processing with Deep Learning

https://web.stanford.edu/class/cs224n/

Notion

Multimodality and Large Multimodal Models (LMMs)

For a long time, each ML model operated in one data mode – text (translation, language modeling), image (object detection, image classification), or audio (speech recognition).

Multimodality and Large Multimodal Models (LMMs)

https://huyenchip.com/2023/10/10/multimodal.html

Multimodality and Large Multimodal Models (LMMs)

What Is Multimodal AI?

Applications, Principles, and Core Research Challenges in Multimodal AI

What Is Multimodal AI?

https://app.twelvelabs.io/blog/what-is-multimodal-ai

What Is Multimodal AI?

[미라클레터] 인공지능이 하나가 된다 ?!?

미라클 모닝을 하는 일잘러들의 '참고서'

[미라클레터] 인공지능이 하나가 된다 ?!?

https://stibee.com/api/v1.0/emails/share/Ue8q7jhsWwUJw3s7XtO19eGpnGJDUJ4=

[미라클레터] 인공지능이 하나가 된다 ?!?

thegenerality.com

https://thegenerality.com/agi/?fbclid=IwAR3aPmwPe6CC7INiPtKlq3SxShfP_-l4LsfvCmS-I6ChQgIAl5qfuQLz_YE

UX

Multi-Modal AI is a UX Problem

Transformers and other AI breakthroughs have shown state-of-the-art performance across different modalities * Text-to-Text (OpenAI ChatGPT) * Text-to-Image (Stable Diffusion) * Image-to-Text (Open AI CLIP) * Speech-to-Text (OpenAI Whisper) * Text-to-Speech (Meta’s Massively Multilingual Speech) * Image-to-Image (img2img or pix2pix) * Text-to-Audio (Meta MusicGen) * Text-to-Code (OpenAI Codex / GitHub Copilot) * Code-to-Text (ChatGPT, etc.) The next frontier in AI is combining these mo

Multi-Modal AI is a UX Problem

https://matt-rickard.com/multi-modal-ai-is-a-ux-problem

Multi-Modal AI is a UX Problem

Recommendations

/////