Hello!

What did you do this week?

I wrote docs, looked into K2IS’ sync_flag and completed implementing positive sync_offset for HDF5. I also extracted the recursive slice splitting code in HDF5 into its own function and wrote tests for it. While working on HDF5, I faced a strange bug in which the unit tests passed when ran on real datasets but failed when ran on randomly generated temporary data.

What is coming up next?

Handle combinations of sync_offset and reshaping, fix sig_shape reshaping and the bug mentioned above for HDF5. I’ve almost figured out how the sectors are synced in K2IS, so hope to finish it soon. I’ve a few more things on my todo list before my PR can be merged and I think this would be my last GSoC blog, so thanks a lot for reading!

Did you get stuck anywhere?

No, but I must say that the best and most challenging part of this journey was deciding between different algorithmic/design solutions to solve problems in my project.

Thanks again to my mentors, PSF and Google for this opportunity!

:-)

Hello!

What did you do this week?

I worked on a recursive function which splits tile_slices in order to properly read from reshaped HDF5 files. Now the scan dimensions can be reshaped for n-D HDF5 data!! But there’s a bug to be fixed before the detector dimensions can be reshaped. Splitting tiles was much faster than reading each frames using their indices. I also improved the coordinates generation by making it a cached property and I fixed some bugs in the RAW and EMPAD dataset to fix the errors thrown while specifying a shape larger than their image_count.

What is coming up next?

I’m working on HDF5’s sig_shape reshaping and sync_offset. K2IS’ sync_flag is still WIP. A few changes need to be done on the client side to handle the deprecation of ‘scan_size’ and ‘detector_size’. I’ll update the docs as well.

Did you get stuck anywhere?

No.

Hello!

What did you do this week?

I improved the coordinates generation code according to the feedback I got from my mentors. Now it performs better and is more efficient as only the UDFs which need the coordinates call the generation code. There are quite a lot of ways to read from a HDF5 file, so to find the optimal solution to HDF5’s sync_offset problem, I created another prototype to read binary data from a HDF5 file using indices but it turned out to be 10x slower than reading slices directly using read_direct. I also got a better understanding of the K2-IS dataset’s meta and implementation in LiberTEM.

What is coming up next?

I’ll continue working on HDF5 and K2IS's sync flag. There are a few more things I can do to improve the coordinates generation code, so I’ll work on that. There are also a few bugs to be fixed and usability improvements to be made on the client side before the PR can be merged.

Did you get stuck anywhere?

No.

Hello!

What did you do this week?

I added the feature for getting the coordinates of frames in a slice for UDFs, wrote tests, fixed a few bugs and continued working on HDF5.

What is coming up next?

I’m finally going to work on the integration of sync_offset into K2IS's sync flag. I’ll continue working on HDF5’s sync_offset and make sure it works with reshaping and ROI. Also make sure that FRMS6's dark frames correction works properly with a sync_offset.

Did you get stuck anywhere?

No.

Hi there!

What did you do this week?

I completed ‘sync_offset’ for SER, fixed a few bugs, wrote tests for ‘reshaping’ all the formats and continued working on HDF5, FRMS6 and K2IS.

What is coming up next?

My next aim is to figure out how to integrate the reshape feature into the UDF interface and write tests for the same. This would be helpful to get the coordinates in the actual navigation space for UDFs. I’ll also continue working on offset for K2IS and HDF5. The FRMS6 format is almost done, I just need to figure out how to handle the dark frame correction when the signal dimensions are changed.

Did you get stuck anywhere?

No.