Cythonizing Bottlenecks in the NODDIx Model

+

Over the past few days, I have been trying to remove the bottlenecks in the Python code by vector implementation using Numpy extensions and a variety of other code optimization techniques.

However, I have come to realize that even Numpy at times does not solve problems like Global Interpreter Lock and Bounds Checking. Cython, on the other hand gives a very nice interface with the lower level C implementations by inserting an intermediate layer between Python and its interpreted code in C. The following diagram will help demonstrate what I mean:

Courtesy: https://www.software.ac.uk/blog/2017-07-25-speeding-python-cython-some-thoughts-python-programmers

Typed MemoryViews in Cython… Blissful interaction with NumPy Arrays!

So, it is often the case that we need to cythonize only a part of our code and not all of it. This requires smooth interaction of other functions with the cythonized functions. MemoryView provides a really good way of  doing this by allowing efficient access to memory buffers, such as those underlying NumPy arrays, without incurring any Python overhead. Memoryviews are similar to the current NumPy array buffer support (np.ndarray[np.float64_t, ndim=2]), but they have more features and cleaner syntax.

Memoryviews are more general than the old NumPy array buffer support, because they can handle a wider variety of sources of array data. For example, they can handle C arrays and the Cython array type (Cython arrays).

A memoryview can be used in any context (function parameters, module-level, cdef class attribute, etc) and can be obtained from nearly any object that exposes writable buffer through the `PEP 3118`_ buffer interface.

Following is an example of the Legendre Gauss Integral in Python and its Cythonized equivalent following it:

The Cythonized Version:

Need to Zoom in for this one..

The entire code for along with other cythonized functions can be found on this branch: noddix_speed !

References:

https://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html

Leave a Reply

Your email address will not be published. Required fields are marked *