Week 14 - August 30th, 2023
View Blog Post
What I did this Week:
This week I fixed the test for the isotropic source of kurtosis so now it's working for all DTDs. I also created tests for the ``K_micro`` function. Initially while running the test, I got some errors which made me look deeper into the actual function. The error was that I was doing sqrt of some elements when actually I was supposed to calculate square of them. Also I was using a '1/5' factor which was not actually required. On fixing these issues, the overall map image of ``K_micro`` improved significantly.
Previously the multi-voxel test case was failing due to different eigenvalues in the isotropic total diffusion tensors simulations. Removing the eigenvalue assertion made the test pass, as verifying the kt, cvt, and evals values sufficed.
I also provided documentation to some functions in the test file such as ``_perpendicular_directions_temp_`` and ``from_qte_to_cti`` etc.
Also had to change the name of some functions to make it sound more relatable to what they were actually doing.
This week I almost finished with the CTI tutorial. The only thing remaining is to create a fetcher for the data so that all the users can download the data and use it. Currently the path given for the data retrieval is of my local system.
I also added some references and overall improved the wording and information in the tutorial.
I also finished writing my final work report and got it reviewed by my mentors and then updated it.
Finally, before pushing the file onto the main PR, I cleaned up the code by removing all the extra comments and some unnecessary code and overall made sure that the entire code was following the pep8 standard.
What is Coming Up Next:
Now that almost everything is completed and all the unit tests are passing, I'll just focus on getting the PR merged.
I will also aim to achieve a 100% coverage report, thereby ensuring that every line of code is thoroughly tested.
At present, the merging of my PR is on hold as we're actively seeking a pair of reviewers who can conduct a thorough examination of the entire codebase and provide their expert feedback.
Additionally, I'll try to get the QTI simulation merged as well .
Week 13- August 22nd, 2023
View Blog Post
What I did this week
This week, I finished writing tests for the sources of kurtosis. While the isotropic source passed the test for the anisotropic DTD, the anisotropic source passed tests for all DTDs. As a result, I integrated the test for the anisotropic source within the test_cti_fits function, eliminating the need for a separate function. I also created tests for multi-voxel cases but the tests passed only for single voxel cases. One reason I think this might be happening is because of the way we're accessing the covariance and diffusion tensor elements. I intend to look further into this. I worked on the real life data, trying to plot maps, but it didn't work out because the problem was really related to the fact that the current kurtosis source implementations do not handle multiple voxel cases. I worked on real life data, attempting to plot maps. Even though I was not able to get the desired result, I'm sure I'll figure it out with further research and possible collaboration.
What is coming up next Week
As soon as mapping of the images is done, I'll finish with the documentation of my model. I also intend on adding some references to some of the important methods. If everything is done early, and time remains then I'll look into finishing the QTI simulation tutorial. I also intend on creating a final work report of my project and submit it to the GSoC dashboard.
Week 12 - Aug 15th, 2023
View Blog Post
What I did this week
According to last week's situation, I was trying to make the ``test_cti_fits`` function work as well as the ``test_split_cti_params`` function. This week I figured out the problem with these functions and was able to fix it. The major problem was occurring while I was trying to compare the parameters. Firstly, I started by removing all the extra fit methods in common_fit_method list such as NLS, CLS and CWLS as I realized that we won't immediately be needing an extra multi_tensor_fit function in CTI.
I also realized that our fit method needs to accommodate multiple voxels, so I had to add "@multi_voxel_fit" at the top of that function.
Major fact that I realized was that the already implemented ``ls_fit_cti`` function was not implemented incorrectly. The cti_return (obtained directly from ``ls_fit_cti``) was not matching cti_params because of the eigen value decomposition. Since the total diffusion tensor in my test was isotropic, when the tensor was decomposed inside the function ``ls_fit_cti``, it created a different eigenvector due to floating point precision. So, I instead tried to compare the more broken down variables, hence I compared the evals from cti_params to evals of cti_return, and the same for covariance tensor elements and kurtosis tensor elements. And on doing this the variables matched perfectly.
Also, last time I had the tests and functions for axial, radial, mean and apparent kurtosis which are basically different measures derived from the diffusion kurtosis tensor, and they help in providing different perspectives on the microstructural complexity of the tissue. So, since last time the fit function itself could not be tested, consequently I couldn't run tests for these functions as well. So, after the tests for the fit function were working, I ran tests for these functions and realized some errors. I fixed the errors by having to take care of the implementation part of these functions as well as changing direction assertion to np.allclose as we also need to take care of the floating point precision detail.
However, for implementing tests for the sources of kurtosis, we'd previously decided that we'd take QTI tests into consideration as k_bulk appeared similar to k_iso and k_shear seemed similar to k_aniso, but upon careful examination we realized that this was not actually the case. This is because in case of QTI, when they're calculating D from self.params, they already have the top 6 elements in voigt notation. We can also observe that when they convert ``from_3x3_to_6x1``, they're multiplying some factors. So, the end point being that "D" in our case and the diffusion tensor "D" in their case is different. And this matters because they're using D to calculate k_bulk (line 1059 in QTI.py) and we're using it to calculate K_iso. Therefore I couldn't use their tests anymore. So, the only solution now in order to be able to implement tests for the sources of kurtosis is to look at the actual paper and implement their ground truth values.
This week I also spent time creating a basic draft of the CTI simulation.
What is coming up Next Week
This week I intend on making sure that the tests for the sources of kurtosis are implemented properly and they're working as expected.
Also, the real life data when run on our model at the moment is giving some overflow errors. I intend on working towards this as well.
I'm also going to make sure that at the end of this week, I'm able to create a good near finish draft of the CTI simulation.
Week 11- Aug 8th, 2023
View Blog Post
What I did this week
Previously I had the function for different sources of kurtosis outside the Fit class. Then upon suggestion from my mentor, this week I put them inside the Fit class. This required me to make changes to how certain variables were being called inside those functions as well as what arguments needed to be passed to those functions. I also added documentation to those functions which included their basic information, as well as the math involved. I also wrote tests for those functions inside the test_cti_fits function. In order to implement this, I took help from QTI.py as our K_aniso resembles the QTI k_shear function, and K_iso resembles the QTI k_bulk function. Taking these points into consideration, my job of writing tests for them was made easier. Although while implementing the tests, I had to keep in mind the conversion factors. And every variable which involved conversion from a 6x6 to 21 x 1 matrix, required its own conversion function such as k_shear, k_bulk etc.
I also implemented tests for the mean, axial, radial and apparent kurtosis functions which were already present in the CTI Fit class. These are the different measures derived from the diffusion kurtosis tensor. These measures provide different perspectives on the microstructural complexity of the tissue.
This week I was also given some real time data in order to construct bvals, bvecs with files that had extension .bvals, .bvecs. I was able to run these files successfully on my computer and was able to make sure that the gtab created from these files were giving the expected signals.
Majorly this week I worked on modifying the tests of the functions as required and tried to make them work. But while doing this, I realized that the 'fit' function inside the model class was not implemented properly and hence the test functions were not able to work as expected.
I also started with the tutorial for CTI.
What is coming up next week
This week I'll try to make sure that the test functions are able to work properly. I'll try to make sure that the function 'ls_fit_cti' in particular is giving the required output. And after this function works, I'll move on to implementing the 'fit' function which would later help in the testing of the Fit class.
Did I get stuck anywhere
I didn't exactly get stuck as I kept learning new things while encountering problems in testing the function. But when I felt that I could not move forward with one tast, I immediately started with other tasks such as providing documentation to function as well as getting started with the tutorials.
Week 10 - Aug 1st, 2023
View Blog Post
What I did this week
In the last week, we had decided to generate a DTD which would make the model more robust by accounting for situations where we had almost all the parameters as non zero, however in that situation the signals weren't matching exactly. So this week I fixed that issue, and so now we can safely say that all DTDs would match the ground truth signals, no matter which parameters are non zero and what changes we make. This was made possible because we were able to figure out the correct order of ccti parameters which is basically covariance parameters which takes the root2 and '2' factor into consideration.
Also, I removed the mean, axial, radial and kurtosis functions from our CTI Fit method because I realized that while importing the DKI Fit class, we're also importing all the necessary features. But I still need to write tests for these methods to make sure that it's all working properly.
The most important task this week has been the implementation of ls_fit_cti, cls_fit_cti and split_cti_params function as this is the most important function in CTI Fit class. I also wrote tests for this class. This involved me understanding the previously implemented similar functions and the parameters that these functions required. One such example is "data" which basically can be interpreted as the "cti_pred_signals".
The importance of writing tests is to see if given the signals, gtab1 and gtab2 can we obtain the cti_parameters correctly or not. So, I'm still analyzing all the functions which have been implemented in DKI, and the tests that have been implemented for them.
So, the important testing function in order to be able to is the test_fit function in CTI.
Among all that, I also implemented the test_split_cti_params and test_cti_fits in the test_cti.py file.
Also, in CTI, even though we're not implementing the radial, axial, anisotropic and mean kurtosis ourselves, we still need to implement tests for these. So in order to write tests for the radial, axial, anisotropic and mean kurtosis ground truth values would be when we call the functions individually, the ones which are outside the Fit class. I also implemented tests for these.
One other point to note is that in DKI, we just needed to make sure that we've multiple b-values. But in case of CTI, we also need to have symmetric b1 and b2s and also parallel and perpendicular experiments.
Now, in the case of CTI, we not only consider the different components of kurtosis, but we also pay attention to the sources of kurtosis. The three sources of kurtosis that we identify in CTI is isotropic kurtosis, anisotropic kurtosis and microscopic kurtosis. I managed to look through the formulas present in the original CTI paper, and implement functions for these.
Things to do next week
Even though I've implemented the function which could calculate the sources of kurtosis, it's still outside the CTI Fit class, and needs to be incorporated in the Fit class. I also need to write tests for these functions. In order to do this, I intend on referring to the QTI paper to look for similarities.
Also, even though I've written tests for the ls_fit_cti and cls_fit_cti function, I still need to run these tests and make sure that it's working as expected.
I also need to make sure that my model runs in case of given multi-voxels. A voxel in a 3D image which can be thought of as similar to a pixel in a 2D image in that both represent a discrete element of the image data. In DIPY, a voxel model refers to a representation of a 3D image volume as a collection of individual voxels, each of which represents a small volume element within the image.
After I write tests for multi-voxel cases, and if my model is working on everything, then I'll move on to running my model on real-time data.
If I get stuck anywhere, then I intend on writing a tutorial for the CTI implementation .