Integrating animation in DIPY to help in quality assessment for big datasets.


I will start with the good news, the image registration workflow is in the process of being merged.

The code went under several iterations of improvement and optimization (after the PR was created), thanks to the community developer for sharing their useful comments.

Wonder, what the following animation is saying ? read on to learn more about it.

For the past few days, I have been brainstorming and experimenting with the following objectives,

Objectives for this work

1) Adding an intuitive component to help in the assessment of the quality of registered images.

Please see my earlier posts about visualizing the results of Image Registration,

Visualizing the Registration Progress

2) Trying different options about what possibly is the optimal and platform independent way of communicating the results for quality assessment

Adding an intuitive Visualization Component

I have already discussed the usage and application of ‘Mosaics’ in assessing the visual quality of the registered data. A mosaic presents a holistic view of the registered image by displaying all the slices (from a specific plane) parallelly so that the change in the voxels can be observed in a sequential manner.

What are mosaics? and their limitations

Mosaics can be handy when a user wants to view the slices and not the entire volume. This provides for more fine-grained control over the quantity of information presented.

However, there is one thing lacking in the mosaic based approach (not that it is not useful), in case of large volumes the number of slices can be overwhelming.  A large number of slices present difficulty not only in creating the mosaic but also makes it hard for the user to understand the data/results easily. Just imagine looking at hundreds of slices while trying to track the changes from one to the other.

Previous Commit LinkMosaics

Making the registered slices into an ‘Animations’

So, I needed another method to go with the mosaics and complement the quality assessment arsenal in DIPY. After discussing with the project mentors, it was decided to use an animation (preferably a GIF) to show the changes in the moving image as the registration goes on.

I started experimenting in a separate branch for creating the animations to visually convey the quality of the registered data.

Commit Link: Adding the animation 

After multiple rounds of debugging and data specific and exploration (adjusting the code for the specific number of slices and volumes), I finally was able to create an animation for showing the progress of image registration.

Animating the slices from the ‘Center of Mass’ based registration

The static image (in RED) and the moving image (in GREEN) shown to be overlapping. The dots of yellow indicates the areas where the overlap is good.

Animating the slices from the ‘Translation’ registration

The animation below shows the slices (from both the static and the moving images) in the axial plane overlapping with each other. This particular image is a bit blurry due to the enlarged size, but I am working on to improve the quality of the results further.

Translation based registration using Mutual Information Metric: The static image (in RED) and the moving image (in GREEN) shown to be overlapping.

Animating the ‘Rigid body based’ registration

The following animations show the effect of rigid body based registration on the moving image when the rigid registration is done using the Mutual Information metric.

Rigid registration using Mutual Information Metric: The static image (in RED) and the moving image (in GREEN) shown to be overlapping. The portion in yellow are the ones that aligned well.

Animating the slices from the ‘Affine’ registration

The best results were obtained in terms of the affine registration as is being shown in the following animation. The portions of ‘RED’ are most superimposed by the GREEN in the AFFINE registration (relatively better when compared to the other modes).

Full Affine registration using Mutual Information Metric: The static image (in RED) and the moving image (in GREEN) shown to be overlapping.

Correcting the orientation

By interpolating the moving image and normalizing it (for the color range of a GIF), the animations were reproduced in the correct orientation.

The MP4 video below shows the animation in the correct orientation (this is the center of mass based for demonstration purposes but the similar results can be produced for other modes too.)

Video: The animation in the correct orientation (scroll to the bottom of the page)

Commit Link: Visualising with the correct orientation

Merging the experimental code with the local repo

Since the initial experiments with the animations yielded satisfactory output so I started adding the changes in the branch from my local repository and also made the code more generalized.

Commit Link: Complete Code Affine Visualizations

These changes are still under testing and will be merged with the experimental branch of DIPY for large data (nipy/dipy_qa).

That’s about it for now. Future posts will focus on developing new workflows with DIPY (now that Image Registration is complete with robust quality assessment metrics)



Leave a Reply

Your email address will not be published. Required fields are marked *