Articles on gavishpoddar's Blog

Part of the journey is the end | Final Report

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 24 Aug 2021 05:11:25 +0000

Hi, community

Part of the journey is the end. It is time for me to work on my final work report for final evaluation of Google Summer of Code 2021. This week, I will devote my time mainly to write my final report.

Final Work Submission Report

Name: Gavish Poddar
Organisation: Python Software Foundation
Sub-Organisation: Zyte
Project: dateparser Better language detection & reimplementing search_dates
Proposal: dateparser - better language detection

Hello Everyone! My name is Gavish Poddar and I'm excited to tell you about my GSoC journey. For the past couple of months, I have been working on an awesome project dateparser. The dateparser aims to parse datetime from a string.

My GSoC journey would not have been successful without the guidance of my mentors Marc Hernández, Konstantin Lopuhin and Kishan Mehta.

What I Have Learned?

The whole GSoC journey was full of learning thanks to my mentors. I learned how to find good open source dependencies to include in our project. I tried my hands on improving code coverage and writing tests for the code. I learned how to optimize code and the need for extensive research before feature addition.

What I Have Contributed?

As mentioned in my proposal I worked on the implementation of the Optional Language Detection for dateparser and fixing as many issues as possible in the search_dates function of the dateparser.

Optional Language Detection

PR - Optional Language Detection

Implemented optional language detection to improve language detection. This allows to plug in any language detection library with the dateparser. Out of the box, dateparser supports two libraries fasttext and langdetect. The optional language detection works with both parse and search_dates. This PR also introduces a new setting DEFAULT_LANGUAGES which is used if no language is detected by default language detection and the optional language detection.

Reimplimenting `search_dates` (extended goal)

PR - Reimplimenting search_dates

A reimplemented and simplified search_dates improves the results and fixes many issues. The entire search_dates is newly implemented and would be easier to maintain. This PR introduces a new feature search_first_date which returns the first date in the given string. This PR also fixes around 13 issues.

Other `search_dates` improvements

Adding support for date-related objects last decade, next decade, etc in search_date. This PR fixes 1 issue.

PR - Improvements in locale:translate_search fixes

Adding support search_date period separator support. Date string like 23.12.2000 can be parsed. This PR fixes 5 issues.

PR - search_date period separator support

Other Important Details

As part of our GSoC project, Python Software Foundation requires us to post a weekly blog where we usually post about what we have done in the week and what is coming up next. We can also write about any blockages or issues we are facing. I have also written my weekly blogs so if you want to know weekly details of my project you can refer them here.

Weekly Blogs

Future Work and Final Note

The project is very actively maintained the new search_dates and my contributions would improve the library. The main goal of the proposal is achived with the implimentation of the optional language detection and the PR is mergeable. I plan to keep working on the project and contribute as much as I can. Contribute to the search_dates function of the library (dateparser) would be my primary goal.

It was overall a wonderful experience and I learned a lot.

I would like to thank Google, Python Software Foundation and Zyte for providing me with the opportunity and my mentors Marc Hernández, Konstantin Lopuhin and Kishan Mehta.

Thank you for reading!

GSoC Weekly Check-In #6

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 17 Aug 2021 20:56:52 +0000

Hi, community

1. What did you do this week?

This week was a tots of coding and finalising the PR's and reviews from mentors.

This week I also created 1 new PR which solves 5 issues and extends support of search_dates function.

2. What is coming up next?

This week I look forward to solve issues from code reviews and wrap up all the workings during the GSoC.

3. Did you get stuck anywhere?

Nothing major, it was a great week with lots of coding.

Thank you for reading!

GSoC Blog Post #5

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 10 Aug 2021 15:52:22 +0000

Hi, community

1. What did you do this week?

In this week I worked towards the completion and the PR is now mergeable we did a lots of micro improvements on the PR and a lots of reviews, suggestions and discussions. The PR with reimplementation of the search_dates is also complete.

2. What is coming up next?

In this week I will wrap up the Optional Language Detection PR and prepare for the final submission.

3. Did you get stuck anywhere?

No.

Thank you for reading!

GSoC Weekly Check-In #5

gavishpoddar@hotmail.com (gavishpoddar) — Mon, 02 Aug 2021 19:07:21 +0000

Hi, community

1. What did you do this week?

This week we did a lots for code review and testing the language detection PR and improvising the search_dates function along with adding few new functions to the new search_dates. In the language detect function we tested and practicality checked most of the lines of code to make sure that they may not cause other issues. This week I also created 3 micro PR's .

2. What is coming up next?

In this week I look forward to complete the implementation of the new search_dates and take reviews form my mentors for improvements so that both the major PR's could be approved within the GSoC timeline.

3. Did you get stuck anywhere?

I didn't got stuck with any thing but this week was a lots of coding and feedback from my mentors that took and lot of time. Few tests cases are still failing but I expect to fix them within this week.

Thank you for reading!

GSoC Blog Post #4

gavishpoddar@hotmail.com (gavishpoddar) — Mon, 26 Jul 2021 18:41:38 +0000

Hi, community

1. What did you do this week?

I work on search_dates this week included creation of many files new supporting functions, created previous tests work with new implementation.

Debugged various issues, checked all settings. And creating docs and tests for language detection.

2. What is coming up next?

In the this week, I will try to complete the implementation of the language detection PR and complete the implementation of the new search_dates.

3. Did you get stuck anywhere?

I got stuck with a tox test (python segmentation fault) now its fixed.

Thank you for reading!

Weekly Check-In #4

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 20 Jul 2021 08:46:58 +0000

Hi, community

1. What did you do this week?

I started this week with work on search_dates reimplementing and simplifying the current implementation.

With a goal to make a maintainable and solving many issues with the search_dates. I created PR for the implemented and tested across the results.

I am also trying to address many issues with search_dates function.

2. What is coming up next?

In the this week, I will try to complete the implementation of the search_dates and write tests for the newly implemented function find_first_date along with the few fixes in the implemented language detection library.

3. Did you get stuck anywhere?

I got stuck with a few questions but mentors helped and supported me to understand and solve the the issues and now its solved.

Thank you for reading!

GSoC Blog Post #3

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 13 Jul 2021 14:30:50 +0000

Hello Everyone!

1. What did you do this week?

I worked with mostly on dateparser-download and preventing breaking changes.

Few more like dateparser-download setting default caching folder, researching on search_dates and how to improve that along with some codes, and updating CLDR data, reviewing the changes and improvising the tests.

2. What is coming up next?

In this week, I will try to complete the language detection PR and make it mergeable with the master and re-implement search_dates with the target of increasing accuracy and performance.

3. Did you get stuck anywhere?

This week was great I didn't got stuck. But in this week I spend significant time to reviewing all changes I made in the PR and improvised the code as much as possible.

Thank you for reading!

Weekly Check-In #3

gavishpoddar@hotmail.com (gavishpoddar) — Mon, 05 Jul 2021 20:41:44 +0000

Hi, community back with the third check-in.

1. What did you do this week?

This was a long week I learned many things

The biggest part was implementing dateparser-download entrypoints to download models from CLI.

Many more like shrinking and organising language data, langdetect set default DetectorFactory without changing global state, Setting fasttext default language and downloading if not already cached and Improving docs.

2. What is coming up next?

In the this week, I intent to fix all issues and code review, finish docs and make changes to prevent braking changes with the previous major version.

3. Did you get stuck anywhere?

I had some problem regarding on how to cache the downloaded fast text language detection models in the package. Now it's implemented and fixed.

Thank you for reading!

GSoC Blog Post #2

gavishpoddar@hotmail.com (gavishpoddar) — Mon, 28 Jun 2021 18:56:15 +0000

Weekly Update

Hello,

It was a really fun and productive week.

Few major milestones achieved this week were :-

Writing tests and docs.

Improving code and making language detection function easier by removing the requirement of apply_setting.

Language Mapping (CLDR & ISO 639).

Check working with other settings along with current.

Removing unsupported locale from language detecting.

Functionality test of the implemented library and discovered the speed and accuracy.

Next Week

The plan for next week is to do a more robust testing, it's settings and then move to updating docs. Moreover setting default language detection, and updating the language translation data.

Thanks for reading

Weekly Check-In #2

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 22 Jun 2021 18:14:14 +0000

Hi, community back with the second check-in.

1. What did you do this week?

Fixes and Tests - This week I spend most of my time writing tests, settings, making the implemented functions simpler for the user and updating setup.py

2. What is coming up next?

In the next week, I am looking forward to writing documentation, language mapping, create tests for settings and check workings with other settings.

3. Did you get stuck anywhere?

Nothing major as such, I had some minor problems with the tests.

Thank you for reading!

Weekly Blog #1

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 15 Jun 2021 21:08:04 +0000

Weekly Update

Hello,

We have had 1 week of coding till now and overall am pleased with the progress of the project. It has been going smoothly without too many major issues we have had frequent feedbacks from mentors are a great help for the development.

This weak was mostly spent on implementing the functions for the language detection function and creating additional settings.

Implementation of an optional language detection function for the dateparser library which allows to plugin language detection library and along with a few selected internally supported library.

The Implemented function takes text and settings as parameter and returns list with language codes which is further used by the dateparser for the translation for the date-time. The function is passed into dateparser as parameter. Any one may plugin there own function to the dateparser for language detection.

The biggest issue we faced this week was on how to make the minimum file import requirements and allow easier use in CLI, Python Notebooks. We solved the issue by actually passing the function as a parameter to the dateparser's parse function.

Implemented settings include:

DEFAULT_LANGUAGE : Works as default language if no language is detected.
LANGUAGE_DETECTION_STRICT_USE : This setting makes the strict use of parsed languages only.
LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD : Which helps set minimum confidence score of detected languages.

Next Week

Functionality analysis of the implemented language detection.
Mock test of parse, search_dates and DateDataParser.
Unit tests for language detect functions.
Updating setup.py for optional language detection.

Thanks for reading

Weekly Post #1 ( 1st June - 7th June)

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 15 Jun 2021 20:12:52 +0000

Weekly Check-in #1

gavishpoddar@hotmail.com (gavishpoddar) — Tue, 08 Jun 2021 06:47:55 +0000

Greetings, Python community!

Hi, I am Gavish, and this summer I am working to implementing an optional language detection library in dateparse and fixing issues related to search_date. This is the first of many upcoming weekly blog posts where I will describe in brief the work I have done in the previous week and my plans for the upcoming week.

1. What did you do this week?

From the beginning of the community bonding period, I am engaging with the community and exploring many different open-source libraries that could be implemented with the dateparser library. The rest of the time was spent on configuring a local development environment, exploring the library in-depth and understand tox.

2. What is coming up next?

In the next week, I am looking to create the preview PR for the optional language detection. I would also work on tests for the optional language detection, and if time permits will implement the setting for the language detection.

3. Did you get stuck anywhere?

Nope. I learned a lot from constant feedback from my mentors. It was an awesome week 😃.

Thank you for reading!

Articles on gavishpoddar's Blog

Part of the journey is the end | Final Report

Hi, community

Final Work Submission Report

What I Have Learned?

What I Have Contributed?

Optional Language Detection

Reimplimenting search_dates (extended goal)

Other search_dates improvements

Other Important Details

Future Work and Final Note

It was overall a wonderful experience and I learned a lot.

GSoC Weekly Check-In #6

Hi, community

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

GSoC Blog Post #5

Hi, community

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

GSoC Weekly Check-In #5

Hi, community

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

GSoC Blog Post #4

Hi, community

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

Weekly Check-In #4

Hi, community

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

GSoC Blog Post #3

Hello Everyone!

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

Weekly Check-In #3

Hi, community back with the third check-in.

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

GSoC Blog Post #2

Weekly Update

Next Week

Weekly Check-In #2

Hi, community back with the second check-in.

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

Weekly Blog #1

Weekly Update

Next Week

Weekly Post #1 ( 1st June - 7th June)

Weekly Check-in #1

Greetings, Python community!

1. What did you do this week?

2. What is coming up next?

3. Did you get stuck anywhere?

Reimplimenting `search_dates` (extended goal)

Other `search_dates` improvements