FNet

Replacing

Self-Attention in transformer blocks with a

FFT-based token mixing layer that operates without attention. The Fourier Mixing first applies a 1D FFT in the sequence dimension and then applies it to the hidden dimensional direction. This reduces the computational complexity to $O(N\log N)$.