For the most part, I spent time in reading about parallel Non-negative Matrix Factorization as this would most likely be the next course of action to take. Unlike PCA which had many scalable, parallel implementations available, NMF seems to be less well-studied and I could not find an implementation that would be suitable for reference. At the same time, I fixed some things and the performance time increased back to ~15 minutes. I also pushed a prototype jupyter notebook that replicates what the current PCA implementation in LiberTEM does, just with standard python libraries (e.g., sklearn, fbpca), and obtained satisfactory performance in reconstruction error. As per my mentor's suggestion, I will be looking into testing edge cases and if the current PCA works under these edge cases.
What did I do this week?
Improved overall PCA performance
What will I work on next week?
Devising edge test cases for PCA. Study Parallel NMF in depth