Utkarsh's Blog

Weekly Check-In #4

Utkarsh
Published: 07/07/2021

This is my weekly check-in as I head into the fourth week of my Gsoc experience. In this article I talk about the work that I will carry out and the approach that I will accept to do so.

What will I be doing?

My mentors have asked me to fix some more unit tests before I can move forward and work on python entry points. These are tests that were either marked as skipped or xfailed because of the unexpected results they were giving. The scripts that contained these skipped or xfailing tests were:-
  • test_wsaa.py
  • test_wsfev1.py
  • test_wsmtx.py
There were 7 such tests that needed to be fixed, taking the number of passing tests to 209.

Since some xfailing tests were fixed, their cassettes had to be re-recorded. Also my mentors had asked me to move all tests from test_wsaa_crypto.py to test_wsaa.py. This was done so that there was only a single test script for all the scripts that were tested.

Apart from the tests, my mentors had also asked me to add coverage report for the tests, so that we could have the code coverage in percentage for the scripts that were tested.

Did I get stuck anywhere?

My approach for the tests was simple, unmark the skipped and xfailing tests and fix the tests one test at a time. Soon I realized that main problems with the tests that I was supposed to fix was that they were dependent on validating or asserting certain values to variables that were time-dependent. In other words , the values that were tested would change with time .These values were generated by the servers depending on the time and date. This was the reason that tests were failing now but passed during the time they were created. Initially I was stuck while understanding the cause of errors. But my mentors were really helpful and explained the error in detail.
View Blog Post

Blog Post #3

Utkarsh
Published: 07/04/2021

This blog covers the work done by me in the 3ed week of my GSOC experience. My work during this time was focused on fixing version issues that were causing 11 tests to fail on python 2 environment, but not on python 3. All the work can be found in this PR.

What did I do?

The majority of my work was to find a solution to the issues that were causing different behaviors on two different environments, i.e. Python 2 and Python 3. The test scripts that were giving problems were:-
  • test_wsaa.py
  • test_wsbfev1.py
  • test_wsfev1.py
  • test_wsbfexv1.py
  • test_wslsp.py
  • test_wsltv.py
  • test_wslum.py
  • test_wsmtx.py

The main problem was the way in which string is treated on python 2 vs python 3. The str object in Python 3 is quite similar but not identical to the Python 2 unicode object. The major difference is the stricter type-checking of Py3’s str that enforces a distinction between unicode strings and byte-strings, such as when comparing, concatenating, joining, or replacing parts of strings.
Along with this I also added a pytest configuration file to register custom markers that I used to stop using fixtures in particular scripts and test methods.

Did I face any problems?

Yes, I faced a lot of problems finding the particular errors that were causing the test failures. Also I trouble finding the exact solution to the problem otherwise a different solution would lead to tests failing on python 3 environment as well. Apart from all this, my other point of focus was to make my solution robust. For example, at first I decided to handle the tests for different versions differently in the test method itself like this
if sys.version_info[0] == 3:
        assert isinstance(wsbfev1.CAE,str)
    elif sys.version_info[0] == 2:
        assert isinstance((wsbfev1.CAE),future.types.newstr)
But then it was correctly pointed out by my mentors was that this made the code look ugly was not a robust solution. Therefore I found a different solution. future module contains a newstr type that is a backport of the str object from Python 3. This inherits from the Python 2 unicode class but has customizations to improve compatibility with Python 3’s str object. We can use it as follows:
>>> from __future__ import unicode_literals
>>> from builtins import str
Using str in this way eliminated the error and was also a robust solution.

Results

Using the above mentioned solution fixed the 11 tests that were not working on python 2 environment to run and pass smoothly. With this 202 tests now passed on both the environments.

Python 2

$ pytest
============================= test session starts =============================
platform win32 -- Python 2.7.12, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: C:\Users\shiva\OneDrive\Desktop\Pyafipaws_Utkarsh\pyafipws, inifile: pytest.ini
plugins: html-1.22.1, metadata-1.11.0, vcr-1.0.2
collected 236 items

tests\test_ws_sr_padron.py s.sx                                          [  1%]
tests\test_wsaa.py ...X.                                                 [  3%]
tests\test_wsaa_crypto.py ....                                           [  5%]
tests\test_wsbfev1.py ..........                                         [  9%]
tests\test_wscdc.py s.......                                             [ 13%]
tests\test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests\test_wsfev1.py ...xxx                                              [ 29%]
tests\test_wsfev1_dummy.py .                                             [ 29%]
tests\test_wsfexv1.py ..........                                         [ 33%]
tests\test_wslsp.py ......................................x.             [ 50%]
tests\test_wsltv.py ............................x.                       [ 63%]
tests\test_wslum.py ........................x.                           [ 74%]
tests\test_wsmtx.py ..............x......x..........x                    [ 88%]
tests\test_wsremcarne.py ...........................                     [100%]

== 202 passed, 5 skipped, 28 xfailed, 1 xpassed, 1 warnings in 27.46 seconds ==

Python 3

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.9.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: C:\Users\shiva\OneDrive\Desktop\Pyafipaws_Utkarsh\pyafipws, configfile: pytest.ini
plugins: cov-2.12.1, html-3.1.1, metadata-1.11.0, vcr-1.0.2
collected 236 items

tests\test_ws_sr_padron.py s.sx                                          [  1%]
tests\test_wsaa.py ...X.                                                 [  3%]
tests\test_wsaa_crypto.py ....                                           [  5%]
tests\test_wsbfev1.py ..........                                         [  9%]
tests\test_wscdc.py s.......                                             [ 13%]
tests\test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests\test_wsfev1.py ...xxx                                              [ 29%]
tests\test_wsfev1_dummy.py .                                             [ 29%]
tests\test_wsfexv1.py ..........                                         [ 33%]
tests\test_wslsp.py ......................................x.             [ 50%]
tests\test_wsltv.py ............................x.                       [ 63%]
tests\test_wslum.py ........................x.                           [ 74%]
tests\test_wsmtx.py ..............x......x..........x                    [ 88%]
tests\test_wsremcarne.py ...........................                     [100%]

== 202 passed, 5 skipped, 28 xfailed, 1 xpassed, 1 warnings in 27.46 seconds ==
View Blog Post

Weekly Check-In #3

Utkarsh
Published: 06/28/2021

This week my work is try and get the tests working that were previously not working with python version 2. There are 11 such tests that pass only on python 3 and not on python 2.

What did I do this week?

This week I searched and fixed the bug that was causing 11 tests to fail on python 2 and pass without any issue on python 3. After much grinding I realized the issue was in the way python 2 handled strings. As the tests were written on a python 3 environment, when evaluated on a python 2 environment, they threw errors if a sequence of characters enclosed within curly brackets was compared to a string object(str). Python 2 has an unicode object which is quite similar to str in Python 3 but not identical. Due to this, issues arise in the way a string is compared in both python versions.

Did I get stuck anywhere

Not really, but finding the correct way in which strings are handled on different versions took some considerable research and patience. The problem was to write code that would make the tests run smoothly on both versions. For this I needed to know exactly what a function used in the tests returned. Whether it was a string or byte-string or some other data type. Also apart from these problems, while fixing some tests, I had to re record some cassettes to find the error.
View Blog Post

Blog Post #2

Utkarsh
Published: 06/28/2021

The second week of my GSOC experience mainly comprised of me fixing the tests that were merged into the main branch. The problems mainly arose due to the difference in main branch and the environment in which the tests were developed.

What did my work comprise of?

My work was mainly to get most of the tests working and to develop cassettes to record the traffic for faster retesting. I had to learn how to use pytest-vcr and to implement in such a way that was efficient and removed redundancies. There were 236 tests in total that covered 11 different scripts. The main reason for recording the traffic was that almost all the services offered require WSAA authentication and they require the wsaa.Autenticar() function to be called. The problem with this is that once this function is called, calling it immediately after causes an exception:- Falla SOAP: ns1:coe.alreadyAuthenticated: El CEE ya posee un TA valido para el acceso al WSN solicitado.
Another problem that I faced was the overwriting of the cassettes once after the recoding was done. This was seen in the exception :- CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('cassette.yaml') in your current record mode ('once').

How did I get about solving these problems?

I figured out that the reason for the CannotOverwriteExistingCassetteException was that there were many redundant requests and while replaying were causing the cassettes to be re-recorded. One way to handle this as to define a fixture that would handle the incoming requests.
I used the following fixture to fix this issue:-
@pytest.fixture(autouse=True)
def auth(request):
    if 'dontusefix' in request.keywords:
        return
    z=request.module.__obj__
    z.Cuit = CUIT
    wsaa=WSAA()
    ta = wsaa.Autenticar(request.module.__service__, CERT, PKEY)
    z.SetTicketAcceso(ta)
    z.Conectar(CACHE, request.module.__WSDL__)
    return z 

Also I had to record the traffic script by script and make cassettes. It was a tedious process but the result was worth it. Initially was running the pytest, it would take about 5 mins for all the tests to run and show the results of the test. Now with the cassettes in play, it was only a matter of seconds before all the tests ran. The cassettes also helped in solving the Already authenticated exception. This was because now when the scripts made request to the servers, the recorded response was played for them. This resulted in faster and efficient testing.

Final Result

Finally with the help of cassettes, I managed to get 202 tests passing on python 3 and 191 tests passing on python 2. The difference in the number of tests passing was due to some unicode errors on python 2. I plan to resolve these issues in the upcoming week so I can get maximum tests passing on both the versions. Here is a summary of tests on python 3:-

$ pytest
============================= test session starts ==============================
platform linux -- Python 3.9.5, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /home/runner/work/pyafipws/pyafipws
plugins: vcr-1.0.2, html-3.1.1, metadata-1.11.0
collected 236 items

tests/test_ws_sr_padron.py s.sx                                          [  1%]
tests/test_wsaa.py ...x.                                                 [  3%]
tests/test_wsaa_crypto.py ....                                           [  5%]
tests/test_wsbfev1.py ..........                                         [  9%]
tests/test_wscdc.py s.......                                             [ 13%]
tests/test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests/test_wsfev1.py ...xxx                                              [ 29%]
tests/test_wsfev1_dummy.py .                                             [ 29%]
tests/test_wsfexv1.py ..........                                         [ 33%]
tests/test_wslsp.py ......................................x.             [ 50%]
tests/test_wsltv.py ............................x.                       [ 63%]
tests/test_wslum.py ........................x.                           [ 74%]
tests/test_wsmtx.py ..............x......x..........x                    [ 88%]
tests/test_wsremcarne.py ...........................                     [100%]

-- Docs: https://docs.pytest.org/en/stable/warnings.html
- generated html file: file:///home/runner/work/pyafipws/pyafipws/report.html --
============ 202 passed, 5 skipped, 29 xfailed, 6 warnings in 6.68s ============

and with python 2:-
$ pytest
============================= test session starts ==============================
platform linux2 -- Python 2.7.18, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: /home/runner/work/pyafipws/pyafipws
plugins: vcr-1.0.2, html-1.22.1, metadata-1.11.0
collected 236 items

tests/test_ws_sr_padron.py s.sx                                          [  1%]
tests/test_wsaa.py ..sxs                                                 [  3%]
tests/test_wsaa_crypto.py ....                                           [  5%]
tests/test_wsbfev1.py .....s....                                         [  9%]
tests/test_wscdc.py s.......                                             [ 13%]
tests/test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests/test_wsfev1.py .ssxxx                                              [ 29%]
tests/test_wsfev1_dummy.py .                                             [ 29%]
tests/test_wsfexv1.py .....s...s                                         [ 33%]
tests/test_wslsp.py ......................................xs             [ 50%]
tests/test_wsltv.py ............................xs                       [ 63%]
tests/test_wslum.py ........................xs                           [ 74%]
tests/test_wsmtx.py ...........s..x......x..........x                    [ 88%]
tests/test_wsremcarne.py ...........................                     [100%]

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated html file: file:///home/runner/work/pyafipws/pyafipws/report.html --
======== 191 passed, 16 skipped, 29 xfailed, 2 warnings in 6.24 seconds ========
View Blog Post

Weekly Check-In #2

Utkarsh
Published: 06/21/2021

Hey everyone, this is me checking in after my second week in GSOC. This week I had to work on unit testing with pytest and pytest-vcr-- PR#63. My work included fixing the already developed tests for the new test suite and also generate cassettes for replaying the tests. This was done to countermeasure the situation if the AFIP servers ever went down.

What did you do this week?

This week’s work was comparatively tougher than the last week’s. I had to fix the tests that were designed in Gsoc 2019. These tests were for the major python scripts in this project. I mainly worked on the issues that arose from PR#59.
These issues were mainly due to the merging of the tests in the new main branch. I also had to work on generating cassettes for the tests as I recorded them using pytest-vcr. This was very useful and necessary because the tests continuously made requests to the server for authentication. Once authenticated the servers would return a message - Falla SOAP: ns1:coe.alreadyAuthenticated: El CEE ya posee un TA valido para el acceso al WSN solicitado. This was the case because the tickets generated were already authorized and we needed to wait sometime before trying to authenticate again. Since the tests needed to run without waiting in the Github wokflow, cassettes for all tests were generated.

Did you get stuck anywhere?

Yes, and badly. As this was my first time working with pytest-vcr, using the right decorators and fixtures was a nightmare. The main problem was that the tests were making a lot of redundant requests and while replaying were casing the error - CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('cassette.yaml') in your current record mode ('once').. Fixing this issue was most overwhelming to me because there were more that 120 tests and I had to fix errors in each.
View Blog Post