Utkarsh's Blog

Blog Post #2

Utkarsh
Published: 06/28/2021

The second week of my GSOC experience mainly comprised of me fixing the tests that were merged into the main branch. The problems mainly arose due to the difference in main branch and the environment in which the tests were developed.

What did my work comprise of?

My work was mainly to get most of the tests working and to develop cassettes to record the traffic for faster retesting. I had to learn how to use pytest-vcr and to implement in such a way that was efficient and removed redundancies. There were 236 tests in total that covered 11 different scripts. The main reason for recording the traffic was that almost all the services offered require WSAA authentication and they require the wsaa.Autenticar() function to be called. The problem with this is that once this function is called, calling it immediately after causes an exception:- Falla SOAP: ns1:coe.alreadyAuthenticated: El CEE ya posee un TA valido para el acceso al WSN solicitado.
Another problem that I faced was the overwriting of the cassettes once after the recoding was done. This was seen in the exception :- CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('cassette.yaml') in your current record mode ('once').

How did I get about solving these problems?

I figured out that the reason for the CannotOverwriteExistingCassetteException was that there were many redundant requests and while replaying were causing the cassettes to be re-recorded. One way to handle this as to define a fixture that would handle the incoming requests.
I used the following fixture to fix this issue:-
@pytest.fixture(autouse=True)
def auth(request):
    if 'dontusefix' in request.keywords:
        return
    z=request.module.__obj__
    z.Cuit = CUIT
    wsaa=WSAA()
    ta = wsaa.Autenticar(request.module.__service__, CERT, PKEY)
    z.SetTicketAcceso(ta)
    z.Conectar(CACHE, request.module.__WSDL__)
    return z 

Also I had to record the traffic script by script and make cassettes. It was a tedious process but the result was worth it. Initially was running the pytest, it would take about 5 mins for all the tests to run and show the results of the test. Now with the cassettes in play, it was only a matter of seconds before all the tests ran. The cassettes also helped in solving the Already authenticated exception. This was because now when the scripts made request to the servers, the recorded response was played for them. This resulted in faster and efficient testing.

Final Result

Finally with the help of cassettes, I managed to get 202 tests passing on python 3 and 191 tests passing on python 2. The difference in the number of tests passing was due to some unicode errors on python 2. I plan to resolve these issues in the upcoming week so I can get maximum tests passing on both the versions. Here is a summary of tests on python 3:-

$ pytest
============================= test session starts ==============================
platform linux -- Python 3.9.5, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /home/runner/work/pyafipws/pyafipws
plugins: vcr-1.0.2, html-3.1.1, metadata-1.11.0
collected 236 items

tests/test_ws_sr_padron.py s.sx                                          [  1%]
tests/test_wsaa.py ...x.                                                 [  3%]
tests/test_wsaa_crypto.py ....                                           [  5%]
tests/test_wsbfev1.py ..........                                         [  9%]
tests/test_wscdc.py s.......                                             [ 13%]
tests/test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests/test_wsfev1.py ...xxx                                              [ 29%]
tests/test_wsfev1_dummy.py .                                             [ 29%]
tests/test_wsfexv1.py ..........                                         [ 33%]
tests/test_wslsp.py ......................................x.             [ 50%]
tests/test_wsltv.py ............................x.                       [ 63%]
tests/test_wslum.py ........................x.                           [ 74%]
tests/test_wsmtx.py ..............x......x..........x                    [ 88%]
tests/test_wsremcarne.py ...........................                     [100%]

-- Docs: https://docs.pytest.org/en/stable/warnings.html
- generated html file: file:///home/runner/work/pyafipws/pyafipws/report.html --
============ 202 passed, 5 skipped, 29 xfailed, 6 warnings in 6.68s ============

and with python 2:-
$ pytest
============================= test session starts ==============================
platform linux2 -- Python 2.7.18, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: /home/runner/work/pyafipws/pyafipws
plugins: vcr-1.0.2, html-1.22.1, metadata-1.11.0
collected 236 items

tests/test_ws_sr_padron.py s.sx                                          [  1%]
tests/test_wsaa.py ..sxs                                                 [  3%]
tests/test_wsaa_crypto.py ....                                           [  5%]
tests/test_wsbfev1.py .....s....                                         [  9%]
tests/test_wscdc.py s.......                                             [ 13%]
tests/test_wsct.py ..........xx..xxxxxxxxxxssxxxxxx                      [ 26%]
tests/test_wsfev1.py .ssxxx                                              [ 29%]
tests/test_wsfev1_dummy.py .                                             [ 29%]
tests/test_wsfexv1.py .....s...s                                         [ 33%]
tests/test_wslsp.py ......................................xs             [ 50%]
tests/test_wsltv.py ............................xs                       [ 63%]
tests/test_wslum.py ........................xs                           [ 74%]
tests/test_wsmtx.py ...........s..x......x..........x                    [ 88%]
tests/test_wsremcarne.py ...........................                     [100%]

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated html file: file:///home/runner/work/pyafipws/pyafipws/report.html --
======== 191 passed, 16 skipped, 29 xfailed, 2 warnings in 6.24 seconds ========
View Blog Post

Weekly Check-In #2

Utkarsh
Published: 06/21/2021

Hey everyone, this is me checking in after my second week in GSOC. This week I had to work on unit testing with pytest and pytest-vcr-- PR#63. My work included fixing the already developed tests for the new test suite and also generate cassettes for replaying the tests. This was done to countermeasure the situation if the AFIP servers ever went down.

What did you do this week?

This week’s work was comparatively tougher than the last week’s. I had to fix the tests that were designed in Gsoc 2019. These tests were for the major python scripts in this project. I mainly worked on the issues that arose from PR#59.
These issues were mainly due to the merging of the tests in the new main branch. I also had to work on generating cassettes for the tests as I recorded them using pytest-vcr. This was very useful and necessary because the tests continuously made requests to the server for authentication. Once authenticated the servers would return a message - Falla SOAP: ns1:coe.alreadyAuthenticated: El CEE ya posee un TA valido para el acceso al WSN solicitado. This was the case because the tickets generated were already authorized and we needed to wait sometime before trying to authenticate again. Since the tests needed to run without waiting in the Github wokflow, cassettes for all tests were generated.

Did you get stuck anywhere?

Yes, and badly. As this was my first time working with pytest-vcr, using the right decorators and fixtures was a nightmare. The main problem was that the tests were making a lot of redundant requests and while replaying were casing the error - CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('cassette.yaml') in your current record mode ('once').. Fixing this issue was most overwhelming to me because there were more that 120 tests and I had to fix errors in each.
View Blog Post

Blog Post #1

Utkarsh
Published: 06/13/2021

In this blog I briefly talk about the work done and issues fixed in week 1 and what I will be doing for the following 2nd week.

What did I do?

The first week of my GSOC experience proved to be a great and fruitful as I successfully completed the week’s tasks under the guidance of my mentors. During the first week my task was to fix issues that arose due to the merging of PR #25 into the main branch. This PR replaced M2Cryto as the standard library for cryptographic recipes. These issues included support inline passing of certificate and key in wsaa.py script and passphrase support for encoding the private key. Along with these issues my PR-- PR#58 also added more unit tests for wsaa.py.
Apart for this particular PR, more 2 of my PR’s were merged which fixed minor bugs(PR #57) and added pytest-vcr support(PR #56) for dummy tests.

What will I be doing next?

Now for my 2nd week I have to work on testing wsaa.py script. Mainly my work will be to develop unit tests that adapt to the current test suite and improve the tests using fixtures and VCR. Although working on test-coverage is scheduled for later weeks, I will be developing the tests keeping the overall test-coverage in mind as the final test-coverage goal is set at 85%.
Needless to say I had lot of fun during the 1st week, mainly during developing tests as seeing them pass is the best feeling ever 😉. Also I am very grateful to my mentors who have made the environment very comfortable to work in. They are there to listen to my silliest queries and are very prompt in their replies. Looking forward to their guidance as I enter the 2nd week of my GSOC experience.
View Blog Post

Weekly Check-In #1

Utkarsh
Published: 06/08/2021

Hello! My name is Utkarsh. I am working this summer with Python Argentina for the project Pyafipws. During all GSOC period I will be posting each week here on my blog updates of my work, starting today! The community bonding period helped me to know my mentors and setup a working plan for the next weeks.I put the milestones mentioned on my proposal as GitHub milestones in my forked repo.I run few tests using pytest VCR.There were few errors in wsaa.py and wsfev1.py which I fixed it.I kept on practicing writing unit tests with VCR for Wsfev1.CompUltimoAutorizado,Wsfev1.CompConsultar methods.I will merge pull requests of previous gsoc in new branch with new features this week.There are some errors which I need to fix in wsfev1.py and also I will keep writing tests and run them.I was not able to generate TA.xml from wsaa.py in new branch and also there were few errors coming while running the tests.But my mentors are very helpful, they helped me a lot and that is the reason I was able to fix this issues.Hoping for great summer ahead!!
View Blog Post

Community Bonding period

Utkarsh
Published: 05/29/2021

Hey there! I am Utkarsh from india working on pyafipws project of python argentina(PSF).Community bonding period is going on now and I am busy in reading the documentation & preparing of a work plan.I am constantly communicating with the mentors and discussing about the ideas.Its really going to be a great summer ahead!!
View Blog Post