Articles on navaneethsuresh's Blog

Final week

navaneeths1998@gmail.com (navaneethsuresh) — Sun, 25 Aug 2019 11:39:06 +0000

This is going to be blog post about cleaning up the work I was doing and revisiting some old things I have done during the GSoC period.

What did I do this week?

I got a patch merged for adding `experimental` argument to the config registrar. But, reviewers asked me to send a compatibility fix as a follow-up as the extension `perf` was failing for older versions of Mercurial. So, I had sent a patch[1] about that and it got folded to the old merged patch by the reviewers. Also, I was asked some questions by my mentor on some of the patches in the stack of `--unresolved`. I investigated on them and answered. A couple of patches were stuck in the middle as I was not able to reproduce the test results in the bug description apparently. So, I investigated on that and updated the patches[2][3]. Then, I started documenting the things I have done during the whole summer as the final report of GSoC. You can read it here[4].

Did you get stuck anywhere?

I was stuck in reproducing the test results given in one of the bug descriptions. But, I found that it was due to a missing indentation. It might sound trivial but, it was not. This was later resolved and the patch was accepted by my mentor,

Coding period: week #12

navaneeths1998@gmail.com (navaneethsuresh) — Sun, 18 Aug 2019 13:22:42 +0000

This is going to be a blog post about fixing an urgent bug and splitting an old RFC patch of mine into stack to be landed. Also, finishing up something I was working for a long time.

What did I do this week?

My mentor introduced me to an issue[1] which is an urgent bug to get fixed. While pushing bookmarks, the bookmarks pointing to secret changesets are also pushed, but the changesets are not pushed. So, while pulling the bookmarks, the changeset it is pointing to will be unknown. We wanted to abort on pushing bookmarks which points to secret changes. I sent two patches. One[2] which demonstrates the issue which has tests and the other[3] one which fixes the issue. I also sent a patch[4] to abort on using both `--interactive` and `--keep` together with `unshelve` which got merged. I worked on two patches[5][6] to enhance the config registrar and its usage. Then, there was an old RFC patch[7] which introduces the first prototype of storing/restoring mergestate with `shelve`. I had to split that into a stack[8][9][10][11] to change its priority from RFC to patches which can be landed in code.

What is coming up next?

My mentor has requested some follow-ups to some of the merged patches. I shall be working on them. Also, I will be addressing the reviews to the WIP patches that are active now and cleanup the work. Then, I'll work on documentation of the work I did and the importance of that.

Did you get stuck anywhere?

Yes. While adding an abort on trying to exchange bookmarks which points to secret changesets, I was unable to solve a test case initially. My mentor asked me to sent a patch to the bitbucket and I did that. He gave me reviews there and it was later fixed by me and I sent the patch to the core.

Coding period: week #11

navaneeths1998@gmail.com (navaneethsuresh) — Sun, 11 Aug 2019 13:44:21 +0000

This is going to be a blog post about cleaning up things I did until now.

What did I do this week?

Last week, I worked on adding a template keyword called `defaultvalue` to store the default value of a registered config item. Also, I did work on adding `--registered` flag to `hg showconfig` command to show all the registered config items. In this week, I got more reviews on it and I had to fix it. So, I sent the following patches and some of them got merged.

[1]: D6709 config: add --registered flag to show all known configs

[2]: D6699 unshelve: abort on using --keep and --interactive together

[3]: D6712 config: remove pycompat.bytestr() for defaultvalue

[4]: D6720 config: fix fm.data() handling of defaultvalue

What is coming up next?

I was trying to add logic for the use of `--interactive` with `--keep` for `hg unshelve`. My mentor Pulkit suggested me to store the remaining shelved change with a new basename on using these flags together. I have coded up this feature. But, unfortunately, my regression tests are not running as expected while this is working fine when I locally built it and tested on another mercurial repo. I will try to fix these tests. Also, I'll be working on changing the default behavior of `hg grep` to the expected behavior which is hidden under a config option.

Did you get stuck anywhere?

Yes. While working on adding support for `hg unshelve --interactive --keep` for a partial unshelve, I couldn't get my regression tests running. I am still working on this to find a solution for this.

Coding period: week #10

navaneeths1998@gmail.com (navaneethsuresh) — Sat, 03 Aug 2019 13:44:54 +0000

This is going to be a blog post about coding up a much requested feature by the community.

What did I do this week?

This week my mentor asked me to work on an interesting feature which was requested by the community. The issue can be found here[1]. I have sent three patches to Phabricator in which two of them have been merged. You can find the patches here:

[2]: D6709 config: add --all flag to show all known configs

[3]: D6712 config: fix defaultvalue template keyword

[4]: D6704 config: add defaultvalue template keyword

Mercurial has a command called `showconfig` which by default without any flags shows all the config items as set by the hgrc. But, I had to add a flag to show all config items in the config registrar. So, I added an `--all` flag to show all known config items. Also, I added a new template keyword called `defaultvalue` to show the default value of the config item. Until now, nothing was shown on trying to see a default value of a config item which has not set a value by the user. Community was much interested in this and Yuja had suggested the UX. For the `--all` flag to work, I had to walk in such a way that I am digging up the registrar other than just user config options. I had to skip the items which had dynamic default values.

What is coming up next?

I haven't decided upon the exact feature on work on next week. My mentor asked me to select one of either the bug tracker or the WeShouldDoThat page depending upon the community interest.

Did you get stuck anywhere?

I had a starting trouble to find out a way to access the items from the config registrar. Then, my mentor Pulkit told me about a variable called `self._knownconfigs`. That was a turning point. The rest of the task happened really fast.

Coding period: week #9

navaneeths1998@gmail.com (navaneethsuresh) — Sat, 27 Jul 2019 05:54:02 +0000

This is a blog post about trying to clean up the code that I have written for a feature that had merged and marked as experimental.

What did I do this week?

For the past few weeks, I have been working on an interesting feature called `unshelve --continue`. My mentor was actively giving reviews on the old patch even after getting that merged. So, I had to send a follow-up stack following the merged patch. I started with a patch[1] to modify the help text on verbose mode to say the user why this feature is still experimental and it's problems. Then, I sent a new stack:

[2]: D6679 unshelve: store information about interactive mode in shelvedstate

[3]: D6687 unshelve: create a matcher only if required on creating unshelve ctx

[4]: D6694 unshelve: fix bug on a partial unshelve with --continue

[5]: D6697 cmdutil: add allowunfinished to prevent checkunfinished() on docommit()

[6]: D6685 unshelve: changes how date is set on interactive mode

[7]: D6686 unshelve: handle stripping changesets on interactive mode

[8]: D6683 unshelve: unify logic around creating an unshelve changeset

[9]: D6699 tests: add test for unshelve --interactive --keep

What is coming up next?

Since I have been working on `unshelve --interactive` for a long time, my priority should be this only. I will try to clean the code and modify the tests to account for all cases possible. If that is getting over on time, I will work on an interesting issue[10].

Did you get stuck anywhere?

As usual, I got problems in rebasing and evolving changesets, I had to make the DAG linear after every time after resolving enormous conflicts. But, I got used to it and it wasn't much a problem per se.

Coding period: week #8

navaneeths1998@gmail.com (navaneethsuresh) — Sun, 21 Jul 2019 06:03:25 +0000

This is a blog post about unfortunately getting a bug in the feature I thought I had written properly and fixing that.

What did I do this week?

I have been working on adding an interactive mode to unshelve for a really long time. It was not easy like I had thought. My mentor reminded me that my code won't work for unshelve which can lead to conflicts. So, I had to modify a function called `shelvecontinue()` to work on interactive mode. unshelve with conflicts is aborted and user had to do `hg unshelve --continue` to continue the unshelve after conflict resolution. I had to modify the function `shelvecontinue()` and the main function `dounshelve()`. It took me enough time to modify both the functions and to get the tests running successfully. Later, the patch[1] got merged. But, then my mentor informed me that I had to mark this feature as `EXPERIMENTAL` as unshelve interactive logic completely lies on `_rebaserestorecommit()` which will do a graft. So, the user will get conflicts even if there are conflicts in the changes other than those requested by the user. I sent two[2][3] follow up patches for simple fixes and those got merged. But, now working on the main fix.

What is coming up next?

I am trying to modify `_rebaserestorecommit()` so that when there are conflicts, the user should only get conflicts when there are only conflicts in the changes which are requested by the user. This may take time.

Did you get stuck anywhere?

Yes. While trying to modify `_rebaserestorecommit()` function so that there should be conflicts only when there are conflicts on changes requested by the user, I couldn't get this to work. `cmdutil.dorecord()` cannot be called directly. Since we are already committing the changes as `shelvectx` and not doing anything, we need to rethink how to interactively select changes from a stored commit. Now, we are doing a graft and changes are there in the repo. So, `cmdutil.dorecord()` will work successfully. I haven't got a solution for this till now. Will work the next week on this.

Coding period: week #7

navaneeths1998@gmail.com (navaneethsuresh) — Fri, 12 Jul 2019 17:15:16 +0000

This is a blog post about how I finished adding a new feature I have been working for the last two weeks.

What did I do this week?

I had sent this[1] patch last week and I finally got succeeded on adding an interactive mode to unshelve. There is already an interactive mode in shelve. The first step I took was to make unshelve selected files only. I had almost completed this in the last week itself. Unfortunately, I got sick this week. So, I wasn't able to work for more hours. However, I managed to finish my work this week. I had to create two changesets. One with changes requested by the user to unshelve at the given time and the latter is shelved for later. The first is done interactively by the user. They can select changes chunk by chunk. This received good reviews from the community.

What is coming up next?

My old patch on my project which does a lot of things is inactive for a while. I shall ask people to review that. Also, my mentor is actively giving new things outside the project sample space which the community has an interest. I shall try to work more on them. Since my college is opening this Monday, I will be concentrating to work on that within a short period of time.

Did you get stuck anywhere?

I thought we have to manually code everything in the interactive mode. I spent quite a lot of time investigating and try to code up that from scratch. Later, @av6 advised me to look at shelve interactive and it was much easier to implement. I learnt that quite often we need to think clearly rather than deeply. An interesting thing about a large codebase is that it might be hard to look up but, almost all of the things are implemented in it. We often have samples to look for motivation.

Coding period: week #6

navaneeths1998@gmail.com (navaneethsuresh) — Sat, 06 Jul 2019 16:18:00 +0000

This is a blog post about how I sent a patch to a new feature and fixed old patches of mine by sending follow-ups this week.

What did I do this week?

I sent a patch[1] to make `hg unshelve` accept files. It added a `--files` flag to perform unshelve on selected files. My end-goal is to add interactive mode to the command. This interactive mode should work the same as it works for `revert`, et al. It also has to support curses ui. I borrowed a function from `uncommit` extension called `_commitfiltered()`. This will generate another commit changeset with selected files only. I had to modify it a little. The original funcition was written to exclude files. I modified it to include the selected files. I sent another patch[2] as a follow-up to rename a function after shelve landed on core. I also had to rebase changes from this[3] patch on the top of the commit which made land shelve on core. This led to enormous conflicts and I had to resolve that.

What is coming up next?

This[4] patch is trying to add interactive mode to `unshelve` as an end-goal. Hopefully, I will complete this by next week. If time permits, I will also work on things on the project sample space.

Did you get stuck anywhere?

On adding a `--files` flag to `unshelve` it wasn't recognising the files selected. I spent quite a long time on this. Finally, I realized that I had to change one of the arguments in the function definition to a list. It was a string by default.

Coding period: week #5

navaneeths1998@gmail.com (navaneethsuresh) — Sat, 29 Jun 2019 07:22:55 +0000

This is a blog post about how I spent this week addressing reviews and started working on an interesting feature outside the sample space of the project.

What did I do this week?

I started getting reviews on a patch[1] that I had sent last week. I had to rebase my changes on the top of the tip of the default branch. I got enormous conflicts because, my project interfered with another GSoC project within my org. So, I solved the conflicts and ran tests. There were still some issues. I fixed those and amended the changes. I also sent a patch[2] to make shelve independent of rebase and it got merged. Then, I started working on adding interactive mode to unshelve which can make selected files unshelve from the stored shelve. Here[3] is the feature request of it.

What is coming up next?

I just started working on adding interactive mode to unshelve. I hope to complete this by next week. If time permits, I'll work on adding `--unresolved` flag to `hg commit` and test exchanging shelves. However, plans can get changed.

Did you get stuck anywhere?

When I was told to rebase my changes on top of tip of default, I got enormous conflicts. It wasn't easy for me to solve them. But, I started small and made progress. Some part of the rebase caused bugs. So, I made the changes on top of the rebase and amended those changes. Thus, all tests started running successfully.

Coding period: week #4

navaneeths1998@gmail.com (navaneethsuresh) — Fri, 21 Jun 2019 05:24:30 +0000

I had already done the things planned in this week last week itself. This is going to be a blog post about how I still utilized this time to do things which was worth investing time.

What did I do this week?

I was waiting for reviews for the first few days. I was contacting my mentor consistently and he told me that he'll pick something for me to work on from the sample space outside the project idea. We also had a video chat in which he told me things regarding my project which I can work on near future. As promised, he gave me a list of things to work on from which I can select whatever I like to work on. Here are those things:

Moving shelve extension to core.
Moving show extension to core.
Improvements around bookmarks.
Functionality to exchange shelves.

All of them were interesting to me. So, I decided to work on everything starting from the first one. It was relatively easy. All I had to do was to make sure that all tests are running successfully after moving shelve code from `hgext/shelve.py` to `mercurial/commands.py`. I sent a patch to phabricator regarding that. It can be found here[1].

What is coming up next?

Things coming up next week will mainly depend upon the reviews of my this week's patch. If there is time, I plan to extend the functionality of storing unresolved mergestate to multistep commands like `rebase`, `histedit`, etc.

Did you get stuck anywhere?

Some of the tests were failing when I moved shelve to core. One of the interesting ones was `test-check-module-imports.t`. I realized that I was importing `rebase` extension for `shelve` in `commands.py` and this `rebase` was itself depend on `commands`. This resulted in an import cycle. Technically, `shelve` will work. But, this `test-check-module-imports.t` will fail as it's not a good programming practice. I fixed this by only importing `rebase` whenever necessary inside the helper functions which shelve was depend upon. Other test fixes were relatively simple.

Coding period: week #3

navaneeths1998@gmail.com (navaneethsuresh) — Sat, 15 Jun 2019 04:40:43 +0000

This is a blog post about how I spent this week addressing the reviews of last week PRs and added a new storage method for the existing functionality.

What did I do this week?

I got more reviews on the two PRs that I had sent last week. I worked first three days solving those nit-picks in the reviews. Then, my mentor advised me to fold those two patches into one and update the description. I did that. Also, the very first PR that I sent on the very first day of coding period got merged but, broken two tests. I sent another patch to fix those broken tests and it got merged. Then, I worked on adding a storage method for the mergestate information in changeset extras. Until then, the usual mergestate information was just moved from `$HGRCPATH/merge` to `$HGRCPATH/merge-unresolved`. I had to write two functions in which one is an encoder and other is a decoder to store the list of merge records in extra mapping as it only parses strings. I haven't sent a new patch for this functionality. Instead, I amended it on top of the old patch as my mentor suggested. All work that I did was included here[1]. If you are interested in my work of this week only, then refer to this[2] changeset on my bitbucket hg clone. I also added an experimental config option to make sure that we have all tests running successfully for both storage methods at the same time successfully.

What is coming up next?

Now, I have also written storage in changeset level with the help of extra mapping. But, haven't received much reviews. The upcoming work will mostly based on reviews. However, this is a minimal implementation. I have used a storage format of mergestate which works for hg versions 3.7 and above. Also, I have omitted the storage of local version content of conflicted files. I'll try to explore more corner cases and work on that. I'll think upon transmitting these conflicted changesets across repositories.

Did you get stuck anywhere?

When storing mergestate records in changeset extras, I couldn't parse the information correctly. This was due to the absence of an encoder and a decoder. These mergestate records are a list of tuples. Since we can only read/write strings in extra, we needed to encode. I came to know this from my mentor and did code that functionality right away then. Also, two of my tests of my first merged patch were failing. I fixed that with the help of my mentor and another member.

Coding period: week #2

navaneeths1998@gmail.com (navaneethsuresh) — Fri, 07 Jun 2019 05:44:57 +0000

This is a blog post about how I spent this week implementing the main feature offered by the project with a simple hack.

What did I do this week?

I worked on adding a new flag `--unresolved` to both `hg shelve` and `hg unshelve` commands and have sent two PRs[1][2] to the core with an initial prototype of the project implementation. This changes the behavior of `hg shelve` and `hg unshelve` commands to handle unresolved merge-states. I have also written test cases and error handling for all corner cases possible. At first, I worked on my bitbucket clone of `hg` and was consistently pushing new changesets and seeking reviews from my mentor. After getting a nice prototype, I sent those two patches to Phabricator. This will slightly change the behavior of these commands in both UI and internal workflow. The user has to update to one of the merge parents of the unresolved shelve to unshelve for later. I tried to fix this internally. But, couldn't find out a solution. Things can get worse when I have to write traversal mechanism and write enough parsing. This may slow down hg. So, a perfect balance between performance and intuitive nature can be maintained. Since, `hg log -G` command can perfectly help the user to get the old merge parents, it's no big deal to update to one of those just before doing unshelve.

What is coming up next?

Now that I have written the UI of the project and got that to work. But, to reach the end-goal of the project I would have to store data in a more convenient way. In the current implementation, I made use of the already stored data about merge-state. I will wait some more time for reviews from the community. Until now, all of the reviews I have got were from my mentor only. So, the next step will depend upon community's interest of getting things done in a usual manner. I will try sending patches about storing data in changeset level.

Did you get stuck anywhere?

I haven't had an idea that a simple hack can make the project work. I tried to store unnecessary data in my implementation. Then, my mentor told me we are already storing these things in a different way and I can make use of that. I interacted with my mentor very much and got enough confidence to push new changesets consistently and make things work. Surprisingly, I stuck at a point of sending patches and got an error that my hg version was not compatible. But, it got resolved automatically. I had already contributed much to the core. Still couldn't figure out why it has happened. Also, my initial implementation modified the code of more commands. At that time, my mentor told me that we can skip modifying things at such a low level if I write a neat abstraction at the top level. And at last, I successfully isolated my code from other parts of the codebase and wrote enough comments on that to make the community more understand my code.

Coding period: week #1

navaneeths1998@gmail.com (navaneethsuresh) — Fri, 31 May 2019 06:30:55 +0000

This is going to be a blog post about how I spent this week coding up and thinking about implementing the initial feature of the project in a minimal manner.

What did I do this week?

I pushed two commits to my clone[1] of `hg` in Bitbucket. Since my project is about adding functionality to store an unresolved merge-state, there can be two approaches viz; storage in changeset level and storage as external textfiles. However, this project also adds a functionality to exchange unresolved commits between repositories. So, my end goal would be storing in changeset level. @marmoute had suggested some ideas on changeset level storage and realized me the things to consider when I go on implemented that (especially, performance issues). My mentor @pulkit25 insisted me on taking baby steps to the proposed implementation. He asked me to code up a minimal implementation of external textfile storage and push the code somewhere public. The final version of the code lies here[2]. After getting this done, I worked on adding another command to help the user get back to the old state and pushed the code here[3]. I bootstrapped both of the functionalities as extensions as my mentor told me to because, it can decrease the chances of main core codebase getting buggy and can isolate it from other functionalities. I had also written required tests for the code for every functionality. This will account for bugs on adding new functionalities and keep a good eye on breaking existing functionalities upon adding new ones.

What is coming up next?

I did commit a functionality to get the user back to the old state just before the merge. So, the next step would be to help them work with the old merge-state and continue their merge resolution. I will make the changes to the commit that I've made and wait for reviews from my mentor. If this goes well on time, I'll start planning about storing data in the changeset level with the help of my mentor and the hg community.

Did you get stuck anywhere?

There was an inflexion point in the very beginning in the start of the coding period. I wasn't confident about the implementation part as my proposed plan got changed by the suggestions of the community. But, with the help of my mentor I started making progress. With minimal patches as baby steps, I improved on my implementation and am getting closer to the expected results.

There was a point when I was confused in the development workflow. I was amending the changesets and the DAG was visible as expected when I viewed it locally. But, when I pushed to the remote repo in Bitbucket, it showed the old changesets which were outdated and created new heads. I asked my mentor about this. He explained me the mechanism that happens internally when I amend changesets. I basically needed obsmarkers. When I locally amend, hg strips the old changeset but, doesn't share that info with the server. So, when I pull again, I'll get the changeset back. What obsmarkers does is tell that the changeset is obsolete and prevent it from travelling. He pointed me to evolve extension[4] which made my life easier to edit history. However, evolve support is broken in Bitbucket right now. So, the obsolete changesets will keep on showing up. He told me to ignore this at the time being and focus more on the project implementation. My mentor @pulkit25 is really active and replies to me in the speed of light whenever I ping him. He realized me that this project is going to be tough but, having patience will make amazing end results.

Initial Welcome Blog Post

navaneeths1998@gmail.com (navaneethsuresh) — Sun, 26 May 2019 18:34:07 +0000

Hello everyone! I'm Navaneeth Suresh, a second-year undergraduate student at Indian Institute of Technology, Kharagpur. I'm working with Mercurial on adding a functionality to store an unresolved merge-state as my GSoC project.

What did I do this week?

There was a video conferencing session with my mentors of 45 min duration last week, we scheduled weekly meetings and discussed on how to have a kickstart with the project. I was advised by my mentors to send a mail in the mailing list so that I'll get to know the opinions of people on my proposal. They insisted on explaining the working of an open source community to me and told that only mentor-mentee interaction might not be enough during the GSoC period. As a result, I had sent a mail and got to know the opinions of various people on the project implementation. I had written a minimal RFC patch for the proposed implementation. However, changes were made in my proposed implementation method and I finalized the pipeline for the implementation with the help of the community by splitting the whole idea into even more subtasks. I realized the importance of starting small and seeking opinion of community during this period. Pros and cons for various methods to implement the project were discussed and people in the community voted for the approaches. I picked up the approach with most votes.

What is coming up next?

As soon as the coding period begins next week, I'll be writing code for initial feature of the project. I was told to bootstrap the implementation as an Extension so that it cannot interfere with the core package and turns out to be buggy. So, I'll be writing an extension to support storing unresolved merge-states as commits. This will involve giving attention to a couple of things and changing the default behavior of some parts of the codebase.

Did you get stuck anywhere?

When I was trying to visualize the changes caused by a command called `hg shelve` after studying its code and behavior, I was unable to proceed. I asked the community for a solution. One of the members suggested a solution which I had already tried and failed. But, then they make me realize that there is no need to visualize those changes now and told me that is not the correct starting point as I'll be facing some issues and told to start with a smaller subtask which less intervenes with most of the code. Then, I moved to thinking of a minimal solution to the smallest subtask possible.

That's all for this week. I'll be blogging about the things happen in the later phase of the GSoC period. Stay tuned!