akshansh's Blog

Week 4 - Progress Report

akshansh
Published: 07/09/2022

Tasks Accomplished This Week

Issues Raised

  1. [Parser] `Tokenizer Error` on list comprehension example · Issue #752 · lcompilers/lpython (github.com) [OPEN]

Issues Closed/Partially Completed

  1. [New] Parse raw, byte and formatted strings · Issue #512 · lcompilers/lpython (github.com) [OPEN]
  2. TODO: Extract type hints from the comments · Issue #485 · lcompilers/lpython (github.com) [OPEN]

PRs

  1. [Parser] Parse raw strings by akshanshbhatt · Pull Request #734 · lcompilers/lpython (github.com) [OPEN]
  2. [Parser] Parse indexing operations on attributes by akshanshbhatt · Pull Request #751 · lcompilers/lpython (github.com) [OPEN]
  3. [Parser] Parse `type-comments` in `for-loops` by akshanshbhatt · Pull Request #724 · lcompilers/lpython (github.com) [OPEN]

Problems Faced This Week

  • I didn't face any major problems this week.

Plans For Upcoming Week

  • Most of the major AST nodes are implemented now. I plan to rigorously test the new parser to its limit by parsing python files from some of the major Python projects' repos such as SymPy and NumPy.
  • Comparing the AST generated by the new parser from that of the existing parser would let us know the changes to be made in the current parser generator file.
View Blog Post

Week 3 - Progress Report

akshansh
Published: 07/02/2022

View Blog Post

Week 2 - Progress Report

akshansh
Published: 06/25/2022

View Blog Post

Week 1 - Progress Report

akshansh
Published: 06/18/2022

Tasks Accomplished This Week

Issues Raised

  1. [Tokenizer] Segfault during tokenization · Issue #612 · lcompilers/lpython (github.com) [CLOSED]
  2. [Tokenizer] Newline token not returned after comments · Issue #617 · lcompilers/lpython (github.com) [CLOSED]

Issues Closed/Partially Completed

  1. [New] Parse Ellipsis and None · Issue #510 · lcompilers/lpython (github.com) [CLOSED]
  2. [New] Parse IN and IS in the expression · Issue #511 · lcompilers/lpython (github.com) [OPEN]
  3. [New] Parse Comments after and inside Multiline statements · Issue #509 · lcompilers/lpython (github.com) [CLOSED]
  4. [Tokenizer] Segfault during tokenization · Issue #612 · lcompilers/lpython (github.com) [CLOSED]
  5. [Tokenizer] Newline token not returned after comments · Issue #617 · lcompilers/lpython (github.com) [CLOSED]
  6. [New] Parse raw, byte and formatted strings · Issue #512 · lcompilers/lpython (github.com) [OPEN]

PRs

  1. [Parser] Parsing rules for `...` and `None` by akshanshbhatt · Pull Request #538 · lcompilers/lpython (github.com) [CLOSED]
  2. [Parser] Rules for expressions containing `is` and `is not` by akshanshbhatt · Pull Request #557 · lcompilers/lpython (github.com) [CLOSED]
  3. [Parser] Rules for `not in` by akshanshbhatt · Pull Request #578 · lcompilers/lpython (github.com) [CLOSED]
  4. [Parser] Fix issues with comment and indentation by akshanshbhatt · Pull Request #619 · lcompilers/lpython (github.com) [CLOSED]
  5. [WIP][Parser] Add support for parsing prefixed strings by akshanshbhatt · Pull Request #624 · lcompilers/lpython (github.com) [OPEN]

Problems Faced This Week

  • I tried to close #511 altogether but failed to do so because of multiple shift/reduce conflicts raised during parser generation. The syntax for logical keyword in was conflicting with the syntax of for loop.
  • While working on parsing prefixed strings, I came across an error with byte strings. The current AST generation (the current slow parser) needs to be debugged.
  • Formatted strings, as of now, works well for most of the cases. However in some cases, there is gibberish character generation for the formatted value's id field. I have still not found the actual cause of this issue. The same example generates perfectly fine AST on my mentor's computer (Linux, x86) but causes issue on mine (MacOS, ARM).

Plans For Upcoming Week

  • I aim to complete PR #624 and close the issue related to it (#512). I aim to add basic parsing support for formatted, byte, raw and unicode string initially. We can iterate and make it better later.
  • I also aim to work on issues #508 and #620.
  • I will try to finally close #511 by refactoring the for loop parsing patterns.
View Blog Post