Fractional fourier transform meets transformer encoder
IEEE Signal Processing Letters
Institute of Electrical and Electronics Engineers
2258 - 2262
Item Usage Stats
Utilizing signal processing tools in deep learning models has been drawing increasing attention. Fourier transform (FT), one of the most popular signal processing tools, is employed in many deep learning models. Transformer-based sequential input processing models have also started to make use of FT. In the existing FNet model, it is shown that replacing the attention layer, which is computationally expensive, with FT accelerates model training without sacrificing task performances significantly. We further improve this idea by introducing the fractional Fourier transform (FrFT) into the transformer architecture. As a parameterized transform with a fraction order, FrFT provides an opportunity to access any intermediate domain between time and frequency and find better-performing transformation domains. According to the needs of downstream tasks, a suitable fractional order can be used in our proposed model FrFNet. Our experiments on downstream tasks show that FrFNet leads to performance improvements over the ordinary FNet.
Published Version (Please cite this version)https://doi.org/10.1109/LSP.2022.3217975
Showing items related by title, author, creator and subject.
Sparse representation of two-and three-dimensional images with fractional fourier, hartley, linear canonical, and haar wavelet transforms Koç A.; Bartan, B.; Gundogdu, E.; Çukur, T.; Özaktaş, Haldun M. (Elsevier Ltd, 2017)Sparse recovery aims to reconstruct signals that are sparse in a linear transform domain from a heavily underdetermined set of measurements. The success of sparse recovery relies critically on the knowledge of transform ...
miR-200c: a versatile watchdog in cancer progression, EMT, and drug resistance Mutlu, M.; Raza, U.; Saatci, Ö.; Eyüpoğlu, E.; Yurdusev, E.; Şahin, Ö. (Springer Verlag, 2016-06)MicroRNAs (miRNAs) are 20–22-nucleotide small endogenous non-coding RNAs which regulate gene expression at post-transcriptional level. In the last two decades, identification of almost 2600 miRNAs in human and their potential ...
Fast and accurate linear canonical transform algorithms Özaktaş, Haldun M.; Koç, A. (IEEE, 2015)Linear canonical transforms are encountered in many areas of science and engineering. Important transformations such as the fractional Fourier transform and the ordinary Fourier transform are special cases of this transform ...