Moving on to 3D elements

After the first evaluations, my mentors and I have decided to put a hold on the 2D elements and move to creating 3D UIs and widgets.

FileMenu

Apart from continuously fixing bugs in my previous contributions, I have created a new element, which is a file select menu that allows the user to browse their file system and look at files and folders. They can navigate into and out of directories by clicking on them in the menu.

This element uses the ListBox2D I talked about in my previous post. Files are shown with Blue color and directories with Green.

The PR for this can be found here.

Orbital Menu

The first 3D element that I am creating is an orbital menu. A basic version of this was designed by my mentor 2 years ago.

The main challenge here is to ensure that the menu(disk here) around the object(cubes here) follows the camera always so that they can always be visible. For this, the vtkFollower class is used.

My task here is to first adapt the old version of the code to work with the current codebase. Then, I have to discuss various ideas to make this menu user-friendly and sci-fi and implement them.

Experimenting with Assemblies

Assemblies are a very important part of 3D graphics as they allow us to create a hierarchical model of the elements in a scene. Imagine a 3D model of a human arm. There is a three-level hierarchy – the arm, then the hand and finally the fingers. Any transformation on a parent element in the hierarchy must be applied to the children, while allowing the children to have their own independent motion about their parent.

Thus, if a hand moves, the same motion applies to the fingers. Additionally, the fingers have their local transformations with respect to the knuckles. Assemblies help us in achieving this hierarchy.

in vtk, the vtkAssemblyclass create hierarchies of elements. Parts can be added using the AddPart member function.

I have been experimenting with this a lot to fully understand how assemblies work and wrote a small code to create a 3-level hierarchy of 3 cubes.

import dipy.viz.ui as ui
import dipy.viz.window as window

"""
Cube actor
==========
"""

def cube_maker(color=None, size=(0.2, 0.2, 0.2), center=None):
     cube = window.vtk.vtkCubeSource()
     cube.SetXLength(size[0])
     cube.SetYLength(size[1])
     cube.SetZLength(size[2])
     if center is not None:
         cube.SetCenter(*center)
     cube_mapper = window.vtk.vtkPolyDataMapper()
     cube_mapper.SetInputConnection(cube.GetOutputPort())
     cube_actor = window.vtk.vtkActor()
     cube_actor.SetMapper(cube_mapper)
     if color is not None:
         cube_actor.GetProperty().SetColor(color)
     return cube_actor

cube_actor_1 = cube_maker((1, 0, 0), (25, 25, 25), center=(0, 0, 0))
cube_actor_2 = cube_maker((0, 1, 0), (10, 10, 10), center=(50, 0, 0))
cube_actor_3 = cube_maker((0, 0, 1), (5, 5, 5), center=(100, 0, 0))
cube_actor_3.SetOrigin(cube_actor_3.GetCenter())

"""
Creating assembly
=================
"""

assembly1 = window.vtk.vtkAssembly()
assembly2 = window.vtk.vtkAssembly()

assembly2.AddPart(cube_actor_2)
assembly2.AddPart(cube_actor_3)
assembly2.SetOrigin(cube_actor_2.GetCenter())

assembly1.AddPart(cube_actor_1)
assembly1.AddPart(assembly2)


"""
Adding sliders to transform the cubes
=====================================
"""

def translate_blue_cube(slider):
     value = slider.value
     cube_actor_3.SetPosition(value, 0, 0)

def translate_green_cube(slider):
     value = slider.value
     assembly2.SetPosition(value, 0, 0)

def translate_red_cube(slider):
     value = slider.value
     assembly1.SetPosition(value, 0, 0)

def rotate_blue_cube(slider):
     angle = slider.value
     previous_angle = slider.previous_value
     rotation_angle = angle - previous_angle
     cube_actor_3.RotateY(rotation_angle)

def rotate_green_cube(slider):
     angle = slider.value
     previous_angle = slider.previous_value
     rotation_angle = angle - previous_angle
     assembly2.RotateY(rotation_angle)

def rotate_red_cube(slider):
     angle = slider.value
     previous_angle = slider.previous_value
     rotation_angle = angle - previous_angle
     assembly1.RotateY(rotation_angle)

ring_slider1 = ui.RingSlider2D(text_template="{angle:5.1f}°", center = (100, 500))
ring_slider1.on_change = rotate_red_cube

ring_slider2 = ui.RingSlider2D(text_template="{angle:5.1f}°", center = (300, 500))
ring_slider2.on_change = rotate_green_cube

ring_slider3 = ui.RingSlider2D(text_template="{angle:5.1f}°", center = (500, 500))
ring_slider3.on_change = rotate_blue_cube

line_slider1 = ui.LineSlider2D(center=(100, 100), initial_value=0, min_value=-10, max_value=10, length=150)
line_slider1.on_change = translate_red_cube

line_slider2 = ui.LineSlider2D(center=(300, 100), initial_value=0, min_value=-10, max_value=10, length=150)
line_slider2.on_change = translate_green_cube

line_slider3 = ui.LineSlider2D(center=(500, 100), initial_value=0, min_value=-10, max_value=10, length=150)
line_slider3.on_change = translate_blue_cube

"""
Adding Elements to the ShowManager
==================================

Once all elements have been initialised, they have
to be added to the show manager in the following manner.
"""

current_size = (600, 600)
show_manager = window.ShowManager(size=current_size)
show_manager.ren.add(assembly1)
show_manager.ren.add(ring_slider1)
show_manager.ren.add(ring_slider2)
show_manager.ren.add(ring_slider3)
show_manager.ren.add(line_slider1)
show_manager.ren.add(line_slider2)
show_manager.ren.add(line_slider3)
show_manager.start()

I use the “Peek” screen recorder on Ubuntu to record these demos.

Refactoring and New Elements

Apart from continuously improving previous UI elements, I have worked on three major ones in the past two weeks.

Refactoring

A huge portion of the module I am working on was refactored by another contributor to the library. After this was merged, I had to read through all the changes and modify all my code according to them. It took some time to get used to the changes but now coding new elements is a lot easier than before.

The base class for all the elements was refactored to implement some functions that were common to all or most elements, such as retrieving attributes like position, center and size of the elements.

Checkbox and RadioButtons

These elements have undergone major changes since the previous blog post.  The code for the same is available at https://github.com/nipy/dipy/pull/1559.

Option

The Option class, which is a set of a Button2D and a TextBlock2D, acts as a single option for check-boxes and radio buttons. It keeps a checked attribute to facilitate checking and unchecking in Checkbox and RadioButton classes.

Checkbox

The Checkbox class implements check-boxes, which is a set of Option objects, where multiple options can be checked at once. The toggle_check callback to the on_left_mouse_button_pressed event of the buttons handles the checking and unchecking of options.

def toggle_check(self, i_ren, obj, button):
    """ Toggles the checked status of an option.

    Parameters
    ----------
    i_ren : :class:`CustomInteractorStyle`
    obj : :class:`vtkActor`
    The picked actor
    button : :class:`Button2D`
    """
    event = []
    button.next_icon()
    for option in self.options:
        if option.button == button:
            option.checked = not option.checked
        if option.checked is True:
            event.append(option.label)
    i_ren.force_render()
    print(event)

RadioButton

The RadioButton class implements radio buttons, which is a set of Option objects, where only one option can be checked at once. It inherits from Checkbox class, with a different toggle_check function to uncheck all other options when one is clicked.


Range Slider

This is an addition to the already present LineSlider2D element, which is a line on which a handle slides. Each position of the handle represents some value.

I  have added two new classes as additions to this element.

The code for this is available at https://github.com/nipy/dipy/pull/1557

LineDoubleSlider2D

This element allows the user to have two handles on the same slider. These handles can slide on the track, while not crossing each other at any point.

This element is useful for setting a range for a parameter.  For example, in CT images, this can be used to select a mapping for some window of pixel values to values between 0-255 (Windowing).

RangeSlider

This element uses a LineDoubleSlider2D to select a range which restricts a LineSlider2D to move within that range.

This is done by updating the min_value or max_value of the LineSlider2D according to the positions of the handles of LineDoubleSlider2D.

def range_slider_handle_move_callback(self, i_ren, obj, slider):
    """ Actual movement of range_slider's handles.

   Parameters
    ----------
    i_ren : :class:`CustomInteractorStyle`
    obj : :class:`vtkActor`
        The picked actor
    slider : :class:`RangeSlider`
    """
    position = i_ren.event.position
    if obj == self.range_slider.handles[0].actors[0]:
        self.range_slider.set_position(position, 0)
        self.value_slider.min_value = self.range_slider.left_disk_value
        self.value_slider.update()
    elif obj == self.range_slider.handles[1].actors[0]:
        self.range_slider.set_position(position, 1)
        self.value_slider.max_value = self.range_slider.right_disk_value
        self.value_slider.update()
    i_ren.force_render()
    i_ren.event.abort() # Stop propagating the event.

This can be useful when a user wants to fine tune the value of a parameter by narrowing the permissible range for that parameter.


Scroll Bar

There is a pending PR by another user here that implements a ListBox, which shows a list of items that can be selected using mouse clicks. I was asked to add a scroll bar to it to extend the current existing scrolling capability through mouse wheel and buttons.

The most challenging part of this was finding a formula for the height of the bar and the amount of movement of the bar(in pixels) that should correspond to a unit scroll.

The code for this is available at https://github.com/karandeepSJ/dipy/commit/5c7295da5f20fd966a9f1995184d17f8d180473d

The next goal is to implement a FileSelectMenu that uses this ListBox to make a file dialog to browse the file structure and select a file.