GSoC Final Report

Published: 08/18/2021


GSoC comes to an end this week. Here I have presented my contributions to the Scrapy project that I made this summer. All the features that I have proposed were implemented. The last feature had to be divided into 2 parts: soft limits and hard limits. The soft limits part has been implemented but hard limit is yet to be completed.


1. Issues

2. Pull Requests 3. Blog Posts: All my blog posts can be found here.

Future Work

Though the features have been implemented, there are scope of improvements:

  • Make hard limits for batch delivery triggers
  • Switching batches based on item content
  • batch scheduling based on cron expression