Texonom
Texonom
/
Engineering
Engineering
/Data Engineering/Artificial Intelligence/Machine Learning/Neural Network/Neural Network Structure/Seq2Seq/Transformer Model/
Complex-valued Transformer
Search

Complex-valued Transformer

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2025 Nov 10 19:19
Editor
Editor
Seonglae ChoSeonglae Cho
Edited
Edited
2025 Nov 10 19:22
Refs
Refs

Domain specific now, but could be generalized?

The core of Transformers is dot-product attention, but in complex space it becomes
Hermitian inner product
, which introduces a phase term that makes softmax meaningless and probabilistic interpretation of
Attention Weight
impossible
 
 
complex-valued-transformer
lucidrains • Updated 2025 Nov 9 13:29
arxiv.org
https://arxiv.org/pdf/2306.09827
Unveiling the Power of Complex-Valued Transformers in Wireless...
Utilizing complex-valued neural networks (CVNNs) in wireless communication tasks has received growing attention for their ability to provide natural and effective representation of complex-valued...
Unveiling the Power of Complex-Valued Transformers in Wireless...
https://arxiv.org/abs/2502.11151
Unveiling the Power of Complex-Valued Transformers in Wireless...
 
 

Recommendations

Texonom
Texonom
/
Engineering
Engineering
/Data Engineering/Artificial Intelligence/Machine Learning/Neural Network/Neural Network Structure/Seq2Seq/Transformer Model/
Complex-valued Transformer
Copyright Seonglae Cho