adityaa30's Blog

Weekly Check In - 5

adityaa30
Published: 07/06/2020

What did I do till now?

I was going through Twisted's implementation of HTTP/1.x and how they are handling multiple requests. I was focusing on their implementation of HTTPConnectionPool which is responsible for establing a new connection whenever required & using an existing connection (in cache). 

Besides this, I did the requested changes on the HTTP/2 Client implementation. 

What's coming up next?

Next week I plan to finish coding H2ConnectionPool and its integration with HTTP2ClientProtocol. Along with the integration I plan to write unit tests as well.

Did I get stuck anywhere?

No. I mostly read lots of documentation & Twisted codebase throughout this week and fixed the bugs found in HTTP/2 Client implementation. 

View Blog Post

Weekly Check In - 4

adityaa30
Published: 06/30/2020

What did I do till now?

Last week I was working on

  • Writing tests for HTTP2ClientProtocol
  • Add support for large number of requests over a single connection

I finished both of the tasks above. I added inline docstrings for most of the methods. Still working on public documentation!

What's coming up next?

Next week I plan to

  • Start working on H2ConnectionPool and H2ClientFactory which are responsible for handlng multiple connections to different authorities. Present implementation is capable of handling large number of request over single connection to only one authority.
  • Finish the public documentation of HTTP2ClientProtocol

Did I get stuck anywhere?

I am very new to writing tests using TwistedTrial so was having minor bugs while setting up the testing environment and writing tests. Apart from this there was no major blockers during the last week 😁

View Blog Post

Weekly Check In - 3

adityaa30
Published: 06/22/2020

What did I do till now?

Finish the HTTP2 Client Protocol implementation. 

What's coming up next?

Next week I plan to 

  • Write unit tests for HTTP2 Client Protocol
  • Add required documentation

I have kept the goals for the next week simple as I think there will be some errors uncovered while unit testing which can take time. As the HTTP2 Client Protocol is the core component of this project I have planned this whole week for it.

Did I get stuck anywhere?

 Yes I was stuck with the bug where the HTTP2 Client was working for all the request sending data which can fit in one DATA Frame. When the request body became large, the body had to be broken into a lots of data chunks and send frame by frame and along with this I had to manage flow control for the stream (on which request was initiated) -- This was not working at all. Generally there should be a WINDOW_UPDATE frame send from the remote peer to notify that the sent data chunks were received by the peer and can receive more now. I was getting a WINDOW_UPDATE for the whole HTTP/2 connection but not for the stream on which request was initiated. Initially I didn't know what to do because this was something very new to me and unexpected at the same time 😟. After some discussions with mentors and reading up HTTP/2 RFC I realized that it was okay to not receive WINDOW_UPDATE frame for a specific stream and instead receive for the whole connection and in terms of flow control both are same. So, finally I was able to fix this bug finishing a working implementation of HTTP/2 Client. Yaaay 🥳 . 

View Blog Post

Weekly Check In - 2

adityaa30
Published: 06/15/2020

What did I do till now?

Add support for both GET and POST requests in the HTTP/2 Client. I read up setting up tests with Twisted. 

Whats coming up next?

Next week I plan to 

  • Finish up with HTTP/2 Client Protocol implementation
  • Add tests & documentation

Did I get stuck anywhere?

Initially in my first approach while testing I realized that the client works for requests having response size which is less than the total flow control window. However, for the case when really large response is expected the client was indefinitely waiting and eventually timeout. The fix for that was relatively very simple --  acknowledge each data frame received 😁. This week I also tried to setup testing environment using the inbuilt MockServer in Scrapy which I have not been able to successfully setup due to the issue with setting up HTTP/2 connection with my client and the custom server. Still working on that! 

View Blog Post

Weekly Check In - 1

adityaa30
Published: 06/11/2020

What did I do till now?

As the Community Bonding phase finished I started coding the HTTP/2 Client Protocol. I started simple with adding support for GET requests.

Whats coming up next? 

Next week I plan to

  • Add support for GET and POST requests for HTTP/2
  • Setup base classes used for testing the Client Protocol

Did I get stuck anywhere?

Initially I was intimidated with some of the libraries that I was using for my project. Now, I am comfortable working with them. I was stuck with the issue of combining different chunks of data received from the server for multiple streams in proper order but now its fixed 😊

View Blog Post