Utkarsh's Blog

Weekly Check-In #5

Utkarsh
Published: 07/10/2021

This article covers the work assigned to me for the fifth week of my GSOC experience. This is last week before the first evaluation and I am very excited to conclude the first half and begin the next half with same grit and motivation.

What will I be doing?

This weeks my mentors want me to work on developing entry points for the scripts in the module. An "entry point" is typically a function (or other callable function-like object) that a developer or user of your Python package might want to use.

The most popular kind of entry point is the console_scripts entry point, which points to a function that you want made available as a command-line tool to whoever installs your package.

Though there is more than one way to define entry points, I will define them in setup.cfg file.

What problems did I face?

The first problem that I encountered as soon as I began this task was that not all scripts had a main() function defined in them. Now this is necessary because entry points access a function in the script that they make available to the used via the command-line directly.

So, first that is what I would like to focus on. While doing this I also have to keep in mind that I don't change too much code as it makes the review process very time-consuming for the mentors.
View Blog Post

Blog Post #4

Utkarsh
Published: 07/10/2021

This blog covers the work done by me in the fourth week of my GSOC experience. I spent this week fixing 7 skipped or xfailing tests and added coverage report to the the github workflow. Now the number of tests passing has reached up to 209. Also we can now view the code coverage due the tests in the coverage report.

What did I do?

As mentioned above I fixed some tests that were marked as xfailing or skipped and added support for code coverage. These works can be found in PRs #67 and #68.

Fixed Tests

The 7 tests that I fixed majorly had two issues. The first issue was related to how str object was dealt with in both Python versions(2 and 3). The second issue was that some tests used datetime object to verify values or certificates. This was fine until date was within the range of expiration of the ticket or value. But once those values and tickets expired, the tests would fail.

To solve the first issue I used the following imports that fixed it on both the Python versions:-
>>from past.builtins import basestring
>>from builtins import str

To solve the second issue I used pytest-freezegun module, that literally allows you to freeze time for the particular marked tests or test script. Using this module allowed the same request to be made every time to the servers therefore the above mentioned issues would never arise.

Added coverage report

The second task that I completed this week was to add coverage report to the workflow. This allowed one to view the detailed code coverage script wise and also the overall percentage. This was important because in later weeks my work is to increase the code coverage up to 85%. Adding coverage also helps in the way that it allows one to view the code that has been covered or not so that the tests can be developed in a better way.

Did I get stuck anywhere?

Yes, I definitely got stuck in the early part of the week when I was figuring out the reasons for the failing tests. It was a bit tricky to find the issue was because it was flaky. It was not a constant error that occurred every time I ran the tests.

The fact that tests had to make the same request every time for the assertions to be true was not obvious at first. It took a lot of debugging and constant help from my mentors to find the and solve the issue completely.

One other issue that caused me to waste time and effort was to re-record cassettes for the xfailing tests. As mentioned above they would work perfectly once and then after some time later would cause the tests to fail.

Results

Fixed Tests

Now with the above fixes, the number of passing tests rose from 202 to 209.

for python 2

$ pytest ============================= test session starts ============================= platform win32 -- Python 2.7.12, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 rootdir: C:\Users\shiva\OneDrive\Desktop\Pyafipaws_Utkarsh\pyafipws, inifile: pytest.ini plugins: html-1.22.1, metadata-1.11.0, vcr-1.0.2 collected 236 items tests\test_ws_sr_padron.py s.sx [ 1%] tests\test_wsaa.py ......... [ 5%] tests\test_wsbfev1.py .......... [ 9%] tests\test_wscdc.py s....... [ 13%] tests\test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx [ 26%] tests\test_wsfev1.py ...... [ 29%] tests\test_wsfev1_dummy.py . [ 29%] tests\test_wsfexv1.py .......... [ 33%] tests\test_wslsp.py ......................................x. [ 50%] tests\test_wsltv.py ............................x. [ 63%] tests\test_wslum.py ........................x. [ 74%] tests\test_wsmtx.py ................................. [ 88%] tests\test_wsremcarne.py ........................... [100%] ======= 209 passed, 5 skipped, 22 xfailed, 1 warnings in 21.75 seconds ========

for python 3

$ pytest ======================== test session starts ===================== platform win32 -- Python 3.9.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 rootdir: C:\Users\shiva\OneDrive\Desktop\Pyafipaws_Utkarsh\pyafipws, configfile: pytest.ini plugins: html-3.1.1, metadata-1.11.0, vcr-1.0.2 collected 236 items tests\test_ws_sr_padron.py s.sx [ 1%] tests\test_wsaa.py ......... [ 5%] tests\test_wsbfev1.py .......... [ 9%] tests\test_wscdc.py s....... [ 13%] tests\test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx [ 26%] tests\test_wsfev1.py ...... [ 29%] tests\test_wsfev1_dummy.py . [ 29%] tests\test_wsfexv1.py .......... [ 33%] tests\test_wslsp.py ......................................x. [ 50%] tests\test_wsltv.py ............................x. [ 63%] tests\test_wslum.py ........................x. [ 74%] tests\test_wsmtx.py ................................. [ 88%] tests\test_wsremcarne.py ........................... [100%] ================ 209 passed, 5 skipped, 22 xfailed, 3 warnings in 13.89s ===========

Added coverage

One can view the coverage report in the Artifacts section of the runs section of Actions page.
View Blog Post

Weekly Check-In #4

Utkarsh
Published: 07/07/2021

This is my weekly check-in as I head into the fourth week of my Gsoc experience. In this article I talk about the work that I will carry out and the approach that I will accept to do so.

What will I be doing?

My mentors have asked me to fix some more unit tests before I can move forward and work on python entry points. These are tests that were either marked as skipped or xfailed because of the unexpected results they were giving. The scripts that contained these skipped or xfailing tests were:-
  • test_wsaa.py
  • test_wsfev1.py
  • test_wsmtx.py
There were 7 such tests that needed to be fixed, taking the number of passing tests to 209.

Since some xfailing tests were fixed, their cassettes had to be re-recorded. Also my mentors had asked me to move all tests from test_wsaa_crypto.py to test_wsaa.py. This was done so that there was only a single test script for all the scripts that were tested.

Apart from the tests, my mentors had also asked me to add coverage report for the tests, so that we could have the code coverage in percentage for the scripts that were tested.

Did I get stuck anywhere?

My approach for the tests was simple, unmark the skipped and xfailing tests and fix the tests one test at a time. Soon I realized that main problems with the tests that I was supposed to fix was that they were dependent on validating or asserting certain values to variables that were time-dependent. In other words , the values that were tested would change with time .These values were generated by the servers depending on the time and date. This was the reason that tests were failing now but passed during the time they were created. Initially I was stuck while understanding the cause of errors. But my mentors were really helpful and explained the error in detail.
View Blog Post

Blog Post #3

Utkarsh
Published: 07/04/2021

This blog covers the work done by me in the 3ed week of my GSOC experience. My work during this time was focused on fixing version issues that were causing 11 tests to fail on python 2 environment, but not on python 3. All the work can be found in this PR.

What did I do?

The majority of my work was to find a solution to the issues that were causing different behaviors on two different environments, i.e. Python 2 and Python 3. The test scripts that were giving problems were:-
  • test_wsaa.py
  • test_wsbfev1.py
  • test_wsfev1.py
  • test_wsbfexv1.py
  • test_wslsp.py
  • test_wsltv.py
  • test_wslum.py
  • test_wsmtx.py

The main problem was the way in which string is treated on python 2 vs python 3. The str object in Python 3 is quite similar but not identical to the Python 2 unicode object. The major difference is the stricter type-checking of Py3’s str that enforces a distinction between unicode strings and byte-strings, such as when comparing, concatenating, joining, or replacing parts of strings.
Along with this I also added a pytest configuration file to register custom markers that I used to stop using fixtures in particular scripts and test methods.

Did I face any problems?

Yes, I faced a lot of problems finding the particular errors that were causing the test failures. Also I trouble finding the exact solution to the problem otherwise a different solution would lead to tests failing on python 3 environment as well. Apart from all this, my other point of focus was to make my solution robust. For example, at first I decided to handle the tests for different versions differently in the test method itself like this
if sys.version_info[0] == 3:
        assert isinstance(wsbfev1.CAE,str)
    elif sys.version_info[0] == 2:
        assert isinstance((wsbfev1.CAE),future.types.newstr)
But then it was correctly pointed out by my mentors was that this made the code look ugly was not a robust solution. Therefore I found a different solution. future module contains a newstr type that is a backport of the str object from Python 3. This inherits from the Python 2 unicode class but has customizations to improve compatibility with Python 3’s str object. We can use it as follows:
>>> from __future__ import unicode_literals
>>> from builtins import str
Using str in this way eliminated the error and was also a robust solution.

Results

Using the above mentioned solution fixed the 11 tests that were not working on python 2 environment to run and pass smoothly. With this 202 tests now passed on both the environments.

Python 2

$ pytest
============================= test session starts =============================
platform win32 -- Python 2.7.12, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: C:\Users\shiva\OneDrive\Desktop\Pyafipaws_Utkarsh\pyafipws, inifile: pytest.ini
plugins: html-1.22.1, metadata-1.11.0, vcr-1.0.2
collected 236 items

tests\test_ws_sr_padron.py s.sx                                          [  1%]
tests\test_wsaa.py ...X.                                                 [  3%]
tests\test_wsaa_crypto.py ....                                           [  5%]
tests\test_wsbfev1.py ..........                                         [  9%]
tests\test_wscdc.py s.......                                             [ 13%]
tests\test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests\test_wsfev1.py ...xxx                                              [ 29%]
tests\test_wsfev1_dummy.py .                                             [ 29%]
tests\test_wsfexv1.py ..........                                         [ 33%]
tests\test_wslsp.py ......................................x.             [ 50%]
tests\test_wsltv.py ............................x.                       [ 63%]
tests\test_wslum.py ........................x.                           [ 74%]
tests\test_wsmtx.py ..............x......x..........x                    [ 88%]
tests\test_wsremcarne.py ...........................                     [100%]

== 202 passed, 5 skipped, 28 xfailed, 1 xpassed, 1 warnings in 27.46 seconds ==

Python 3

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.9.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: C:\Users\shiva\OneDrive\Desktop\Pyafipaws_Utkarsh\pyafipws, configfile: pytest.ini
plugins: cov-2.12.1, html-3.1.1, metadata-1.11.0, vcr-1.0.2
collected 236 items

tests\test_ws_sr_padron.py s.sx                                          [  1%]
tests\test_wsaa.py ...X.                                                 [  3%]
tests\test_wsaa_crypto.py ....                                           [  5%]
tests\test_wsbfev1.py ..........                                         [  9%]
tests\test_wscdc.py s.......                                             [ 13%]
tests\test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests\test_wsfev1.py ...xxx                                              [ 29%]
tests\test_wsfev1_dummy.py .                                             [ 29%]
tests\test_wsfexv1.py ..........                                         [ 33%]
tests\test_wslsp.py ......................................x.             [ 50%]
tests\test_wsltv.py ............................x.                       [ 63%]
tests\test_wslum.py ........................x.                           [ 74%]
tests\test_wsmtx.py ..............x......x..........x                    [ 88%]
tests\test_wsremcarne.py ...........................                     [100%]

== 202 passed, 5 skipped, 28 xfailed, 1 xpassed, 1 warnings in 27.46 seconds ==
View Blog Post

Weekly Check-In #3

Utkarsh
Published: 06/28/2021

This week my work is try and get the tests working that were previously not working with python version 2. There are 11 such tests that pass only on python 3 and not on python 2.

What did I do this week?

This week I searched and fixed the bug that was causing 11 tests to fail on python 2 and pass without any issue on python 3. After much grinding I realized the issue was in the way python 2 handled strings. As the tests were written on a python 3 environment, when evaluated on a python 2 environment, they threw errors if a sequence of characters enclosed within curly brackets was compared to a string object(str). Python 2 has an unicode object which is quite similar to str in Python 3 but not identical. Due to this, issues arise in the way a string is compared in both python versions.

Did I get stuck anywhere

Not really, but finding the correct way in which strings are handled on different versions took some considerable research and patience. The problem was to write code that would make the tests run smoothly on both versions. For this I needed to know exactly what a function used in the tests returned. Whether it was a string or byte-string or some other data type. Also apart from these problems, while fixing some tests, I had to re record some cassettes to find the error.
View Blog Post