What did you do this week?

This was the final week for GSoC and we (somehow) managed to get my PR merged! It broke the Gitpod integration but it was immediately fixed afterwards in scipy#14603. I also wrote and submitted my final report as my final evaluation.

As per my proposal, I was able to achieve all the listed milestones. Although I was able to add two new generators from UNU.RAN, there are a lot more which would be good to have in SciPy and would improve the performance significantly (e.g. NumericalInversePolynomial and AutomaticRatioOfUniforms). I have filed an issue scipy#14600 with a list of generators that can be added to SciPy over time. I am willing to continue the work on this project after GSoC and hopefully resolve the issue. I have started to work on adding NumericalInversePolynomial in scipy#14619 and already received some reviews from one of my mentors! I hope to add at least the most important generators before the 1.8.0 release.

I am happy I got a chance to work on such an interesting project and was able to achieve my goals. As always, my mentors and many other developers supported the project and contributed their expertise to increase the quality of my work. The output was orders of magnitude better than what I initially proposed. Thanks to everyone involved in this project!

What did you do this week?

Last week I got stuck at some failures on Linux Python3.7-dbg (debug builds). I was able to replicate it locally and thankfully also found a fix for it. Turns out, I was accessing a Python attribute while a live exception was set in Python. This is not allowed because the Python objects of the class might have been destroyed or garbage collected and my attribute lookup might fail with a segmentation fault. This was caught by the debug build of Python 3.7 and was fairly simple to fix: just removed the offending line and the tests passed again! To ensure this doesn't happen again, I refactored some code to make more careful use of PyErr_Occurred().

There was also some discussion around what to name the seed parameter: the new Generator API favors something other than random_state which has been well-established in SciPy. As there was no consensus among what to name it - some favored consistency while others wanted to differ between the new and the old API - we decided to go with the plain old random_state and change its name in some other PR at all places in SciPy.

What is coming up next?

Almost all the reviews have now been addressed, the CI is green, and I am happy with the current state of the PR. It seems like a good starting point and if things stay this way, we might merge by the end of this week. Once that's done, adding new methods is fairly straightforward. I will try to add as many methods as possible before the end of GSoC.

Did you get stuck anywhere?

No blockers this week.

What did you do this week?

I addressed more reviews this week and added pickling support to all the generators. Most of the changes focused on the documentation, tutorials, and the UI. We removed the params keyword (used to pass positional arguments to the passed callable) from the method constructor and had a discussion on renaming the seed parameter to something like numpy_rng. I also added some histogram plots to the docs and tutorials. Lastly, as all the generators are Cython extensions classes with a non-trivial __cinit__ method, Cython was not able to generate code to pickle object automatically. Hence, I added a rudimentary __reduce__ method which enables pickling support. Relevant tests have also been added.

What is coming up next?

We need to finalize renaming seed to something more apposite like rng or numpy_rng. All other blocking comments have been addressed, except a recent failure on CI. I will investigate the cause of the failure and try to resolve it by the end of this week. As the GSoC deadline is imminent, I hope to complete that by next week so we could merge the PR.

Did you get stuck anywhere?

I noticed a few failures on the linux workflows on Python3.7-dbg build. For some reason, Python crashes when TransformedDensityRejection is called with a invalid PDF. It seems like there is some internal logic error in the C/Cython code. As this is not a Python error with a proper traceback, it makes it very difficult to even locate the error, let alone solving it! I am thankfully able to replicate it locally and hope to figure out the failures as soon as possible.

What did you do this week?

Sorry, I am late publishing this blog! I will resume my regular schedule next week. I spent this week addressing the reviews on my PR. There were comments on the documentation suite and tutorials on formatting issues, obscure phrasing, and naming conventions. As we are getting near the deadline, we decided to leave larger enhancements to docs and tutorials to a follow-up PR. Only a few comments are left to be addressed which I hope to complete by the next week. Also, Nicholas reviewed the C/Cython code again and approved those changes.

What is coming up next?

As GSoC deadline is approaching, we have also decided to try and merge the PR early in the week of August 8 so that more UNU.RAN methods can be added in these last few weeks of the program. I also want to clean up the C/Cython code and go through the docs and tests again to resolve typos and grammatical errors, if any. There are two approvals on the PR already so, hopefully, we could achieve this deadline.

Did you get stuck anywhere?

No.

What did you do this week?

I got a few more reviews on my PR last week. It engendered a interesting discussion on the level of detail we should add in the tutorials. There a few subtle concepts that need to be elucidated for general users to adopt the new API easily. Christoph committed a draft with new description to help explain those concepts better. There also also some other discussions on what to name of the method to sample random numbers from and how to point the users to the new Generator API. As I have an exam coming up next week (details in next section), I was a little quite this week.

What is comping up next?

I will be addressing the reviews on my PR. I also want to benchmark some methods to sample from discrete distribution sometime later. I have my GRE examinations next week on 2'nd August (Monday) so this week might be more silence from my side :) But I will resume my regular schedule right after the exam.

Did you get stuck anywhere?

No.