Summary and Final Submission

After an amazing summer, GSOC has finally come to an end. My project with DIPY under the umbrella of Python Software Foundation was one of the most interesting and challenging projects I have ever worked on.

All my merged pull requests are listed here and the pending ones are available here

The Project

dipy.viz provides many visualization capabilities. The goal of my project was to improve DIPY’s current User Interface widgets and create new ones so as to have a complete library from which users can build interactive applications.

Overview of the work done

Before my project, the library already had working implementations of button, rectangle, disk, sliders, panel, text-box and file menu.

VTK is a great framework as its provides a high level abstraction over the complicated code for graphics rendering. However, there isn’t much documentation and working examples available on the internet to refer to. Therefore, whenever I experimented with new stuff like 3D widgets, animations and filters, progress was slow. But the weekly meetings with my mentors and the targets given kept me going on and I was able to create a lot of new elements.

Since we were building a highly customizable UI, we kept a lot of parameters as arguments so that users can create elements according to their needs. Most of the parameters were also given default values to allow users to simply instantiate an element and see how it works.

Here is an overview of the various widgets that will soon be provided by the library:

ImageContainer2D

PR: https://github.com/nipy/dipy/pull/1522

My first widget was a simple image element which takes the image path and the required size as inputs. I started off with this because the code for this was pretty simple and most of it was already written in other classes.

Option, Checkbox and RadioButton

PR:  https://github.com/nipy/dipy/pull/1559

The Option class, which is a set of a `Button2D` and a TextBlock2D, acts as a single option for check-boxes and radio buttons. It keeps a checked attribute to facilitate checking and unchecking in `Checkbox` and RadioButton classes. The toggle logic is used to change its state.

The Checkbox class implements check-boxes, which is a set of Option objects, where multiple options can be checked at once.

The RadioButton class, which inherits from Checkbox implements radio buttons, which is a set of Option objects, where only one option can be checked at a time.

LineDoubleSlider2D and RangeSlider

PR:  https://github.com/nipy/dipy/pull/1557 

The LineDoubleSlider2D allows the user to have two handles on the same slider. These handles can slide on the track, while not crossing each other at any point. This element is useful for setting a range for a parameter.

This element uses a LineDoubleSlider2D to select a range which restricts a LineSlider2D to move within that range. This is done by updating the min_value or max_value of the LineSlider2D according to the positions of the handles of LineDoubleSlider2D.

Scroll Bar

PR:  https://github.com/nipy/dipy/pull/1564

The scroll bar is an addition to the ListBox2D element which used up and down arrow keys for navigation earlier.

FileMenu

PR:  https://github.com/nipy/dipy/pull/1592

This element allows the user to browse their file system and look at files and folders. They can navigate into and out of directories by clicking on them in the menu.

Orbital Menu

Code:  https://github.com/karandeepSJ/dipy/commit/6a7aed9d9ecc1c94664317601b63581ea037ede7

This is a disk menu around the menu objects which always faces the camera. This is my first try at 3D elements. I haven’t made a PR for this because it requires a major change is the entire library since all the current elements only work in a 2D setting. This has been put on hold and will be continued later. Meanwhile, I assembled a working sample of how it would look like by manually making some changes to the other elements.

Preloader

PR:  https://github.com/nipy/dipy/pull/1611

A Preloader is a visual indication of the process of ‘loading’, displayed to tell the user that the program is running in the background and has not stopped. These include those rotating circles and moving bars we all hate to see on our screens.

Color Picker

PR:  https://github.com/nipy/dipy/pull/1615

The Color Picker I made follows the HSV color model and converts the selected color to RGB. It consists of a vertical bar that selects the hue and a square that selects the saturation and value for the selected hue. There is a small ring inside the square to track the colors selected. The colors in the square get updated whenever the hue is changed.

Future Work

Here’s a list of what we want to do in the future:

  • Generalize the entire viz module, so that all elements work in both 2D and 3D settings. 2D elements should not transform when the camera is moved in a 3D environment.
  • Change the way callbacks are bound to elements. Currently, there are multiple ways this can be done. We want to make a single function to bind callbacks to actors before the next release this year.
  • Style the elements to give a futuristic/ sci-fi appearance.

Acknowledgements

I would like to thank my mentors, Eleftherios Garyfallidis , David Reagan and Ranveer Aggarwal for helping me throughout the summer and holding regular meetings to discuss the project. They, along with other contributors like Marc-Alexandre Côté and Serge Koudoro regularly provided feedback and suggestions to improve my code.

Apart from improving my coding style, this project has helped me to overcome my fear of starting working on large projects.

Thank You !!

 

The Final Weeks

These final weeks were mostly filled with bug fixes and improvements in my previous PRs.

Color Picker

I completed the Color Picker I talked about in the previous post. It is a rectangular color picker based on the HSV model.

It consists of a vertical bar that selects the hue and a square that selects the saturation and value for the selected hue. The horizontal direction of the square corresponds to saturation and the vertical axis to value.
There is a small ring inside the square to track the colors selected. The colors in the square get updated whenever the hue is changed.

The code for this can be found at https://github.com/nipy/dipy/pull/1615.

The Final Challenge

I was given a final task by my mentors which directly deals with brain images. For the tractogram images being generated by DIPY, I was asked to make an ROI (Region of Interest), that could filter the streamlines in the tractogram to show only those that pass through the ROI (something similar to https://vimeo.com/17680190). Tractograms can be generated using the code provided at https://github.com/nipy/dipy/blob/master/doc/examples/viz_advanced.py.

This task involves two features:

  • Filtering the streamlines to show only those that pass through the ROI.
  • Allowing the user to move and distort the ROI in real time.

For filtering polydata, vtk provides a number of filters. I looked at two of them.

  1.  vtkIntersectionPolyDataFilter: This filter takes two vtkPolyData objects and produces the intersection of the polydatas.
  2. vtkSelectEnclosedPoints: This filter takes an array of points and the polydata of a surface, and evaluates all the input points to determine whether they are enclosed by the surface. The filter produces a (0,1) mask that indicates whether points are outside (mask value=0) or inside (mask value=1) the provided surface.

This is a work in progress and there seems to be some issues in these filters. I plan to continue working on this and many other things after the GSOC period ends.