What I did this week

So, among several other things, this week I first started on figuring out how to run spyder on ubuntu as initially I couldn't make it run due to technical problems, but then there was this need to make sure that I'm able to edit my code while keeping the pep8 standards and the automatic formatting of my code wasn't working. So, after having done this, I made some changes in the utils.py file to make the design_matrix more readable. I also made changes in the B[:,3] and B[:, 4] which are the diffusion tensor elements as there was a typo in the sign because we realized that the diffusion tensor part should have a negative contribution as it represents a signal decay

One of the other tasks that I implemented was mapping all the covariance parameters from paper to its actual code implementation which created some confusion, as the conversion shown in the paper didn't quite match its implementation. This was what initially created the need to talk to the authors of the original paper. 

The other more important work I did was trying to figure out the matching of the ground truth signal values in case of anisotropic and combined DTDs. This is because the isotropic DTD signals that were being generated matched exactly the QTI signals, as in case of isotropic we've 6 non zero elements, and the rest are 0s. However in anisotropic case we had more non-zero covariance parameters (9 non-zero), similarly as in the case of combined DTD. So we figured out that the non-zero elements are being multiplied to some value which isn't correct and that this needs modifying the ccti conversion.
So, I worked on reading more about voigt notation, as the QTI parameters were implemented using that notation.
Then we looked again into the QTI paper, and felt the need to contact their author and code implementer and realized that the coding was done while keeping in mind the voigt notation conversion as well as some other factors. At the end of this we figured out the correct conversion of the ccti parameters. We noticed that some factors needed the (root2) division, while some others needed (2). Therefore, we were able to successfully figure out the correct factors that needed to be multiplied/ divided to each of the covariance parameters. 
And hence, now the signal values of all the DTDs match as expected. 
Then the other major ongoing task this week has been the implementation of the Fit class in CTI. This required me to implement some functions which might've been implemented in DKI/ QTI. This is an ongoing task and would require more work. 

What is coming up next week 

After the matching of the correct signals which matched the ground truth values, we realized that a DTD with more non-zero covariance parameters might make the ccti conversion more robust while taking all cases into consideration. So, we created a DTD with mevals, its angles and the fractions. However the signals didn't match exactly. But rather than being stuck on this case, we decided to move forward for the time being. So, I'll work on making sure that all the ground truth signals match. 
But the more important work I will be doing this coming week would be to implement all the required functions in the ccti module such as the different sources of kurtosis which hasn't been implemented before as this is one of the differentiating factors of CTI. And then hopefully move on to generating the tests for these functions. 

Did i get stuck this week

Not at all. Some things were kind of vexing, but I didn't get stuck as there was always something else that could be done. 
 

I was out this week as I had exams, I'll be working this coming week, and hence will submit a blog around 26th of July . 

What I did this Week

One of the tasks I did this week was modify the cti_design_matrix again, as asked by my mentor to make the code more readable. The initial code was following pep8 standard but it wasn't very easy to read, but now it is.
Also, I realized that the main reason my signals weren't matching the ground truth values before at all was because the eigen values and eigenvectors of the diffusion tensor distribution were wrong. This was because, before I tried getting D_flat by doing: np.squeeze(from_3x3_to_6x1(D)) which returned a tensor of shape ( 6, ). But in this case, it returned the diffusion tensor elements in the order : Dxx, Dyy, Dzz  and so on which isn't the correct format of input expected for "from_lower_triangular" function. So, initially, we were doing : evals, evecs = decompose_tensor(from_lower_triangular(D_flat)) where the from_lower_triangular function is returning a tensor of shape: (3,3). But then I realized that rather than calculating D_flat, we can simply do: evals, evecs = decompose_tensor(D_flat). Following this approach gave the correct value of evals and evecs. So, now we have the correct values of evals and evecs which made the signals come closer to the ground truth signals, but we still don't have the signals completely matching the ground truth signals.
Another problem we realized was that while passing "C", covariance tensor parameters, we needed to make sure that we were passing the modified C parameters, that is "ccti". This again helped in bringing the signals to the expected values.
So, after talking things through with my mentor, and analyzing the QTI paper, we came to a few conclusions which could be done to improve the signal values. 

What is coming up next Week

We realized that there might be a slight probability that there is a typo somewhere in the actual implementation of the QTI signals. So we decided to contact the original author and code implementer of QTI. 
Also, one important thing I intend on doing is check the papers, and see the conversions are being done correctly, that is we are taking into consideration the (root2) factor which is present for some covariance tensor elements in the original paper. This is because, for the isotropic case we observe that the signals are matching perfectly because in the isotropic case all the (root2) parameters of the original covariance elements are zero. 
Another thing that I intend on doing is to create a new test method which will have some similarities to the test method in dki_tests.

Did I get stuck anywhere

I didn't get stuck anywhere, but trying to figure out the problem with the current signal generation did take some time and required looking into the research papers. 

What I did this week

This week, I worked together with my mentor to come up with a new way of arranging the elements of the design_matrix. So, first I rearranged all the parameters of the covariance parameters so that they'd match with the ones in QTI. So now, the order is: the diffusion tensor, the covariance tensor, and then the kurtosis tensors. But then we decided that it would be better to put the kurtosis tensors first because then we wouldn't have to re-implement all the kurtosis methods again. So, I changed the order of kurtosis and the covariance tensors. 
Also, in order to maintain the coding standards of the previously implemented models, we decided that the diffusion tensor should be divided into evals and evecs. 
Therefore, because of all these changes I had to re-implement a lot of already implemented functions in CTI which also required changing the description of those functions and not only the code.
But my major time went towards writing tests for these modified codes. While writing codes, I realized that a lot of the functions needed to be modified a bit. Also, I had to import several new libraries in order for the functions to work. 

What Is coming up next week

The testing part for the implemented method is not yet done correctly, as the signals don't yet match the expected output. So, I intend on re-implementing them by taking into consideration the suggestions provided by my mentor. This would require modifying code of some already implemented functions as well as re-writing the tests, particularly the generation of the eigenvalues and the eigen vectors. 

Did I get stuck anywhere

I didn't exactly get stuck, but implementing the tests requires you to make sure that the shape of the tensors you're passing into a function is correct and is as expected. This took me a while to figure out. 

Design Matrix Implementation and Coding with PEP8: Week4
=================================================

What I did this Week: 

This week I worked on improving the design_matrix that I'd previously created. It wasn't according to the PEP8 standards, I managed to modify it according to the appropriate format. 
I mostly spent a bunch of time implementing functions under the Fit class in CTI. In order to achieve this task I had to look into functions which had already been implemented in DKI and QTI, as CTI is an extension of DKI and has similarities with the QTI model. 
My procedure for this part was going through the 2 different modules and comparing the functions, and making note on functions which would have to be implemented in CTI as CTI has a very different design_matrix. 
Although this implementation is not completely done, I learnt a lot. 

What is coming up next Week

This week I intend to further implement the Fit class and also generate tests for the already implemented functions under the Fit class. 
And also write tests to make sure that the signals generated in the QTI model are the same as the ones done in CTI_pred. 
I also intend on changing the order of parameters of covariance tensor as CTI has a lot of similarities with the QTI module, and in order to use QTI functions, we need to make sure that the order of parameters under covariance tensor in QTI is same as order of parameters in CTI.