In the past week, I was working on setting up Hadoop and trying to import data from it. I got my PRs reviewed by my mentor and working on the changes he suggested.

What did I do this week?

I had initially set up Hadoop in my Ubuntu system. But setting this up would be difficult in Travis CI. So I was exploring other options. The easy way to do this is through docker but there is no official Hadoop distribution in docker. I was checking out cloudera's quick start VM but when I was trying to set this up my laptop started to hang. I will continue to look into other options. Also my mentor had reviewed my HDFS source PR and guided me on how to proceed further. 

What is coming up next?

I'll have to work on the docker set up for Hdfs source. I'll probably have to write a script or a docker-compose script. MySQL PR had an issue while my mentor was adding a merge test. Will work on that as well. We'll be preparing for a release soon.

Did you get stuck anywhere?

I struggled a bit with the Hadoop set up. My mentor gave me some input on this and hopefully I'll be able to create a docker set up by next week.

In this week, I made some progress on the second phase of the project. With the database established, I made a PR.

What did I do this week?

To know that I'm approaching in the right direction, I made a work in progress PR and got it reviewed by my mentor. He had suggested some changes and I've been working on it.

What is coming up next?

I still have to refine those changes and get it reviewed again. Once that is done, I can move on to a NoSQL Source.

Did you get stuck anywhere?

It was initially difficult to understand how the application interacts with the database and also how i should define my database. With help from my mentor, I was able to clarify it.

In this week, I started working on the second phase of the project.

What did I do this week?

In my project, there was an example demo app that had the necessary connections to be made and interaction with the database, I went though the code base and add the required config classes for setting up mysql source. I'm yet to refine it and make it more efficient and work towards testing this feature.

What is coming up next?

I'll work on adding test cases to this feature. Once my mentor reviews it, I'll work on the changes requested, if any.

Did you get stuck anywhere?

This week has been pretty smooth.

In the past week, I have started working on phase 2 of my project and wrapping up phase 1.

For phase 2 of my project, after having a discussion with my mentor, I learnt docker and mysql and set up a mysql docker container. Once that was done i was going through the example in the code base which had a demoapp that interacts with MariaDb similar to mysql. I tried understanding the code and planned on how to approach the next phase.

For my previous phase, implementing Label for CSVSource caused a problem in merging. So my mentor and I discussed it and i tuned the logic a bit. Once the test_merge is fixed, I'll have to implement the same for CSVSource again.

This week also had the first evaluation. I have passed it and received good feedback from my mentor. I'll work towards it and hoping to add more test cases in future!