Awesome

Originally a fork of torch-dct, this repository was used to run a small investigation into whether ACDC layers can be used in convolutional layers.

Initially, this was just because we saw that someone had implemented DCTs in PyTorch and thought that would make it relatively easy to try it out.

Is there anything useful here?

There's some benchmarking of the speed of different DCT implementations in PyTorch. The fastest implementation on GPU was simply using the DCT matrix in a Linear layer. This was because the tensor manipulations in order to use Makhoul's method involving an FFT add much more time than the FFT itself.

The ACDC implementations appear to be correct, stable and in line with what the paper describes, but we've only done a basic replication of section 6.1 in the experiment script.

Full details of the investigation can be found in the research log.