The attention mechanism is responsible for much of the recent success in NLP tasks such as text classification, named-entity recognition, Q&A, translation to name a few. However, computing attention is expensive. In this post, we summarize a new approach that replaces self-attention with a Fast Fourier Transform, which achieves 92% of the accuracy of BERT while being 7 times faster.