anandbaburajan's Blog

GSoC: Week #6

anandbaburajan
Published: 07/06/2020

Hi!

What did you do this week?

I implemented the offset feature in ‘get_tiles_straight’ both with and without a region of interest for raw and ‘memory’ datasets, fixed the bug in ‘get_tiles_w_copy’ which marks the completion of the offset feature for rest of the formats except HDF5 and wrote tests for benchmarking. I spent rest of the time looking into possible ways of implementing the reshaping feature.

What is coming up next?

I’ll benchmark the offset feature for the different formats, try to improve it and finish it for HDF5. I’ll continue working towards the general reshaping feature and work on its integration with the UDF interface.

Did you get stuck anywhere?

I haven’t figured out how to deal with the K2IS format properly. I’m not sure if I should implement my feature into the existing sync flag, but I’m working on it. In raw files, setting a negative offset results in empty but colored frames in the GUI but not when used with the Python API in a Jupyter notebook.

View Blog Post

GSoC: Week #5

anandbaburajan
Published: 06/29/2020

Hi!

What did you do this week?

I worked on the API to properly separate additional info for the offset and reshaping features from the dataset parameters. I’m almost finished with the offset feature in the general tile generation function used for most of the formats. Both positive and negative offsets work, both with and without an ROI applied, but I’ve to fix a bug which inserts real frames instead of empty ones when a negative offset is set.

What is coming up next?

I’ll fix the bug to complete the feature, make sure the K2IS format works as expected, implement the offset feature for the ‘memory’ dataset for testing and work on the general reshaping feature.

Did you get stuck anywhere?

No.

View Blog Post

GSoC: Week #4

anandbaburajan
Published: 06/22/2020

Hi!

What did you do this week?

I found out that for the manual sync feature to work correctly, the partitions need to span the missing frames introduced by setting a negative offset too, along with spanning any missing data at the end. That means only the tiling part should be changed, not the partitions. So I wrote a simple prototype for setting a positive sync offset for the HDF5 format’s get_tiles function. If the approach is good, a similar one can be used for setting an offset for rest of the formats, the difference being the way data is read. I also found out that this way, the K2IS format’s sync flag can be handled too. I also implemented reshaping in the GUI form this week.

What is coming up next?

I’ll work on handling the offset with a ROI, make changes to the API to handle additional info for the features and improve the prototypes. I’ll start working on generalizing them for n-D datasets after I get my mentors’ approval.

Did you get stuck anywhere?

No.

View Blog Post

GSoC: Week #3

anandbaburajan
Published: 06/14/2020

Hi!

What did you do this week?

Last week’s manual sync feature wasn’t correct as the partitions are supposed to span the missing data and ignore the tiles, not just skip the missing frames. So I fixed that issue this week, wrote some tests accordingly, added some UX improvements and finished implementing the manual sync feature for the MIB dataset. I wrote a prototype for reshaping a 3D dataset with flattened scan dimensions into 4D. It doesn’t actually reshape the data; instead, it reshapes the 4D slice objects into 3D and then reads the data using the slices.

What is coming up next?

I’ll implement the manual sync feature with tests for K2IS, HDF5 and the memory format this week. The K2IS format would need some more looking into as it is quite different from the other formats and allows specifying a sync flag. The prototype for reshaping might also need to be improved or completely changed, so I’ll work on that too.

Did you get stuck anywhere?

No.

View Blog Post

GSoC: Week #2

anandbaburajan
Published: 06/07/2020

Hello again!

What did you do this week?

I added tests and made some improvements according to the feedback I got on the manual sync feature for SEQ files. I also added some tests for the JS based GUI, and that’s how I was introduced to Jest. Rest of my time was spent on looking into ways to implement the dataset reshaping feature.

What is coming up next?

LiberTEM supports a variety of data formats and each of them have their own quirks. So my next challenge is to find the best approach to reshaping which is flexible enough to support all the formats.

Did you get stuck anywhere?

Yes, while working on the manual sync feature’s offset validation. I couldn’t figure out how to dynamically change the json schema on the server side. Also, the form implementation includes a nested Formik form field component and the error handling felt a bit off as for some reason, validation error messages weren’t showing at all. So either I’m doing it wrong or I’m not supposed to be doing it. But I learned quite a lot about Formik and I’ll be taking a look into it again after my prototype for reshaping is ready.

View Blog Post