Working towards pre-planned transforms

Published: 08/20/2019

What did you do this week?

The main feature I've been working on this week has been adding pre-planned transforms to pypocketfft in (pypocketfft!28). This will allow users to speed-up cases where many transforms of the same shape are done, without needing to rely on any internal caches. This gives the users a small performance improvement in pypocketfft but adding this to the scipy.fft interface will also allow other backends to benefit. e.g. pyFFTW can have a significantly more expensive planning phase.

Plans have now been added into the c++ interface of pypocketfft but I have still got to expose these in the python interface. Additionally, I've made a number of other contributions in several other places:

  • Refactored the real transform tests to use pytest and to test all the transforms in long double precision as well (scipy#10669)
  • Fixed a race condition in my custom threadpool code that was causing crashes on macOS (pypocketfft!27)
  • Added benchmarks to my multithreading PR (scipy#10614)
  • Opened an issue in mkl_fft about adding a scipy.fft interface (mkl_fft#42). Unfortunately, mkl_ffl doesn't seem to be very active and the issue hasn't received any response yet.
  • Fixed a compile warning in uarray (uarray#195)

What is coming up next?

The coding period has officially ended and it's now time to "submit the code". Most of my code has already been merged into the SciPy master branch so this will just involve writing a summary of my work over the whole GSoC project.

I would also like to finish up the pre-planned transform code but it's likely too late to get this merged in time for the evaluation.

Did you get stuck anywhere?

Fixing the race condition in pypocketfft was quite challenging. The only symptom was a SEGFAULT that occurred only rarely and only on macOS, a platform which I don't use and so couldn't reproduce the issue locally. This made diagnosing and solving the issue very tricky. Thankfully, Martin (pypocketfft's author) had access to apple hardware and was eventually able to give me enough details that I found an issue from another project that had run into almost exactly the same issue.