GSOC 2018 Summary: Interactive 3D Visualizations of Human Brain Activity in Jupyter Notebook Using Ipyvolume

Project goals

The aim of the ipysurfer project is to improve interactive, 3-D visualization of brain surfaces using MNE and the Jupyter Notebook. It can be regarded as an alternative to mayavi and PySurfer-based human brain visualization, and is built on top of other open source tools such as ipyvolume, bqplot, pythreejs, and ipywidgets.

Project overview

During the GSOC period, the initial concept of how cortical surfaces can be visualized in Jupyter Notebook was transformed into the ipysurfer package. Two related packages, MNE-Python and PySurfer, were used as the guiding lines during the development process. As a result, I have used similar or the same naming conventions for code styling, classes, methods and parameter names. Most of the original features supported by MNE-Python and PySurfer are now implemented in ipysurfer.

All code is available in a public GitHub repository.

Changelog

Below are the major features that this project has created in ipysurfer or added to ipyvolume:

  • plotting of cortex meshes in Jupyter Notebook using ipyvolume, ipysurfer.Brain class should be used for this purpose. A user can plot the left or right hemisphere alone, both hemispheres on the same plot or on separate plots. Check in BinderBoth hemis

Split hemis

  • displaying of activity patterns as surface color variations of a mesh, a user should apply ipysurfer.Brain.add_data method for this purpose. Visual controls over color map used for data visualization are provided by the same method. Check in BinderColor bar
  • plotting activation data as dependency over time with visual controls, for further information look at ipysurfer.TimeViewer class. Check in BinderTime viewer
  • creation of a function for plotting MNE objects data, instead of raw data, see ipysurfer.plot_source_estimates. This function can be integrated into MNE. It is built with classes mentioned above, so a user can display color map controls or time viewer effortlessly. Check in BinderDivergent color map
  • creation of a stand-alone package that combines code for the above mentioned features, i. e. ipysurfer.
  • Examples of the package usage and code documentation were added as well.

Enhancements to ipyvolume
ipyvolume is a package for visualizing 3-D data in the Jupyter Notebook. ipysurfer utilized this package heavily, and several improvements were made to ipyvolume as a part of this GSOC project.

  • addition of alpha blending support to ipyvolume, see #139, #145.
  • code styling improvements and unit testing configuration changes, see #146, #142.
  • a partial fix of problems with camera orientation changes in ipyvolume, see the problem #156 and the corresponding pull request #158.

What’s left to do

Several things that should be tackled after the end of the program:

  • integration of ipysurfer into MNE, #23.
  • improvement of animation performance, corresponding issue.
  • full solution for the problems with displaying different views in ipysurfer should be found, see issues #31 and #156.
  • improve support for other visualization parameters used in MNE/PySurfer packages, see #14, #28 and #30.

Examples

Examples of possible ipysurfer usage can be found via this link. You can also see a presentation below that shows major features. This presentation was generated from ipysurfer sample notebooks, and can be found here.

Presentation

Thanks and credits

I would like to thank my mentors Jean-Rémi King, Eric Larson and Chris Holdgraf; Alexandre Gramfort, Maarten Breddels and other MNE-Python, ipyvolume and bqplot developers for helping me during my work on this project.

Leave a Reply

Your email address will not be published. Required fields are marked *