Week 4-5 check-in! 🐳

Hey friends!
Welcome to a new weekly checkin, we have made big progress in our journey of GSOC and this week was pretty significant!

What Did I do this week?

Before we get started, it's story time! Few months ago, my mentor Dyllan had developed a version of hub-auto which worked on an early release of Hub 2.0. This version of hub-auto did the job of structuring datasets and worked locally.
In the past few weeks, Davit, Abhinav, Kristina and Dyllan (all part of Activeloop.ai) have worked on Wasabi Integration, Compression, new index meta and a complete refactor of chunk-engine. This broke the hub-auto code which has not been changed in months. So naturally, this week I upgraded hub-auto to work with the latest release of Hub 2.0! I encountered several bugs but it took me few days to get the hang of what hub-auto is supposed to be doing in the latest release.

What will I do in the coming week?

Now that I have upgraded hub-auto to work with the latest release of Hub 2.0. I am going to use that to upload datasets directly to Hub cloud ☁ What earlier took hours, will now be done in seconds. I had prepared a Notion doc with over 200 datasets and their response to hub-auto. Dyllan asked me to create a spreadsheet with that data. I'm currently working on the spreadsheet part. Once that is ready, we can essentially use all that data to improve hub-auto in the coming weeks.

Where did I get stuck?

While upgrading hub-auto, I fixed several issues with the code. There were many libraries to be dropped and chunks of code to be removed. Particularly to make the code work with the refactored chunk-engine. Fortunately, I wasn't very stuck with bugs like I was used to during the earlier days of contributing to Hub. Experience is the best teacher after all.

Special thanks to my mentor Dyllan, who has been super proactive in helping me improve hub-auto and fixing bugs that were out of my scope. Much respect and love to you man! 💜