Utkarsh's Blog

Final Blog post GSoC'21

Utkarsh
Published: 08/20/2021

Hi everyone! It’s time for the last GSOC blog .I have learned a lot and faced many professional challenges. In the end it was really worth it. I hope GSOC experience will help me in the future and that my project will be useful to open source. You can check my final report at : https://github.com/PyAr/pyafipws/wiki/GSoC-2021:-Final-Summary

Thanks to both of my mentors for constantly helping me whenever I got stuck. Thank you for the constant help and advice; mentors that I would recommend to any student! It brings me great joy to end on a fulfilling note for contributing to Python and PyAfipWs project as part of Google Summer of Code 2021. I am happy and content with the work produced. I have previously worked on open source projects before, completing my tasks tasks for this project brought a level of satisfaction that cannot be matched. So that’s it. Thanks to all once again!

Hope to see you all in next GSoC or any other open source work.
View Blog Post

Weekly Check-In #10 : Wrapping up

Utkarsh
Published: 08/16/2021

Hey everyone! With this check-in we have come to the end of GSoC. Though some things are still left to be done, most of the work is completed and PRs have been merged. In this week I will be solving as many of these as possible issues this week. Most of the issues are from the PRs generated before are easier to solve. But there are some issues that need a more careful look into. I look forward to solving these problems and complete the project.

What issues are still not solved?

Out of all the left issues, this is the most important. It covers the testing of the ws_sr_padron.py script. The problem is internal and requires the attention of my mentors. However when this issue is resolved I can finish with writing tests for this script and increase the coverage to an acceptable percentage.

Another issue that needs to be looked at is regarding the pyfepdf script and pdf generation. The issue here was that different facturas.txt files were created for Python 2 and 3. Though I identified early that the error was with Unicode, I couldn't understand how to solve it. Solving this issue is important because without it we cannot check the completeness of the generated facturas.txt file(which is used to generate the pdf file).

Next issue, although not a major one, is present in many PRs and has to be solved for all of them. This issue deals with the location of CONFIG_FILE that contains the configuration of many scripts. I had directly use the original file present in the conf folder, which my mentors advised was a risky thing to do. Therefore, I had to modify the workflow file to make a copy of the rece.ini file in the working directory, so that the original is not modified in any way.
View Blog Post

Blog Post #9

Utkarsh
Published: 08/12/2021

In the penultimate week of my GSoC experience, I wrote unit tests and increased coverage for the scripts, namely, pyemail.py, pyqr.py, and pyi25. These scripts added email, QR, and barcode functionality respectively. My work for each of these scripts can be found in PR#94, PR#95, PR#96.

What did I do ?

As mentioned above, my work this week was to write unit tests for the scripts which responsible for email, QR , and barcode services. This week was mostly fun due to the fact that these scripts worked independently and the results were visual. By this I mean that you could see the QR and barcode generated by invoking the scripts and errors could be found more easily and fixed.

PyEmail

This script provided email functionality that could be used to send email to other users using a particular user's Email and password. It made use of smtplib library to make this possible. My work while developing unit tests was to mock this particular library using pytest-mock. This was necessary for testing as we couldn't otherwise check if the service was working properly or not. Therefore just checking if a particular function was invoked could help us to test the service. I wrote the following code to achieve this:-
def test_main_prueba(mocker):

    mocker.patch("smtplib.SMTP")
    sys.argv = [ ]
    sys.argv.append("/prueba")
    sys.argv.append("user@gmail.com")
    sys.argv.append("pass123")
    main()
    smtplib.SMTP.assert_called_with('smtp.gmail.com', 587)

One can easily see that I am using pytest-mock to mock the functionality of smtplib library and just checking if the SMTP fuction is invoked.

PyQR

This script provides QR image generation functionality in this project. Based on the data provided, it can be encoded into the QR image, which when scanned will provide the user with a link or url in this case. To test the script and check if it was working correctly, I compared the QR image generated during testing with a sample QR saved previously. To compare the images, I used Image and ImageChops library:-
def test_main_archivo():
    archivo = "prueba_qr.png"
    sys.argv = []
    sys.argv.append("--archivo")
    sys.argv.append(archivo)
    main()
    assert os.path.exists(archivo)
    #compare the image with a reference one
    if sys.version_info[0] < 3:
        ref = Image.open("tests/images/qr2.png")
    else:
        ref = Image.open("tests/images/qr.png")        
    test = Image.open(archivo)
    diff = ImageChops.difference(ref, test)
    assert diff.getbbox() is None

If one analyzed the code, they could see that I have tested the generated image with different images depending on the python version used. This is due to the fact that since strings are handled differently on both python versions, they lead to a different url being generated for both the python versions. But both URLs lead to the same page.

Pyi25

This script provided barcode functionality to the project and to test the generated barcode I used the same logic as above. ImageChops library is used to compare the pixel wise difference between two images.
def test_GenerarImagen():
    "Prueba de generación de imagen"
    barras = "2026756539302400161203034739042201105299"
    archivo = "prueba.png"
    pyi25.GenerarImagen(barras, archivo)
    assert os.path.exists(archivo)
    #compare the image with a reference one
    ref = Image.open("tests/images/prueba-cae-i25.png")
    test = Image.open(archivo)
    diff = ImageChops.difference(ref, test)
    assert diff.getbbox() is None

Also, two of the scripts, pyqr.py and pyi25.py provided an option to view the QR or barcode generated. Now on windows it's pretty straightforward. We just check if the image file is opened or not since on windows, the code uses os.startfile to open the images. But on linux eog service is required to be installed in order to view the image. So to check if this viewing functionality would work correctly on linux based systems, I mocked the os.system library and checked if it was invoked or not. This is sufficient because we can assume that if the user has eog image viewer installed, everything will work fine.
def test_main_mostrar(mocker):
    mocker.patch("os.system")
    sys.argv = []
    sys.argv.append("--mostrar")
    archivo = "prueba-cae-i25.png"
    main()
    if(sys.platform == 'linux2' or sys.platform == 'linux'):
        os.system.assert_called_with("eog " "%s" "" % archivo)
View Blog Post

Weekly Check-In #9

Utkarsh
Published: 08/08/2021

This blog marks the check-in for the 9th week of my GSoC experience. As this week is the penultimate week before the second evaluation, my work is to finish any previous left work and also to wrap up for the final submission. My mentors asked me to work on 3 scripts, namely, pyemail.py, pyi25.py and ,pyqr.py. My work was to develop unit tests for the scripts and also to increase their code coverage.

What work is to be done?

As mentioned above, this week, like the previous week had me working on unit tests and code coverage. The scripts that I was told to work on were different from the previous ones in the sense that these scripts were not dependent on other scripts for functioning , which made working with them a little less tiresome.
Also the unit tests had to be developed differently for them. Let me explain with an example:-
The script pyemail.py can be used for email services and requires the valid email and password of the user in order to work correctly. Since this is not possible while testing, my mentors suggested me to use pytest-mock, which be used to mock the functionality of modules while testing. In this example that module was smtplib.

Did I get stuck anywhere?

Yes, initially I had no idea to test such scripts that require the authentication of user's personal credentials in order to function. I was helped greatly by my mentors who informed me about pytest-mock and how it can be used to check if a certain function was being called or not. As you can understand this was enough in this case.
View Blog Post

Blog Post #8

Utkarsh
Published: 08/03/2021

In this blog I talk about my work done in the 8th week of my GSOC experience. This week my mentors had me working on 4 scripts, namely rece1.py, recem.py, recex1.py and receb1.py.

What did I do?

Testing and code coverage are two of the most important requirements of any project. Therefore much of my work lately can be seen focused around these two topics. Last week I had already written unit tests and increased the code coverage of 12 most important scripts. Each represented a web service and it was necessary that the code did not have unnecessary problems.

This week I worked on the above mentioned scripts and my work done for each of them can be found in the following links:-

What problems did I face ?

Most of my work this week comprised of finding minute errors that were causing failure of scripts. One the common errors in all 4 scripts was - 'entrada.txt not found error. All the 4 scripts required this file for working properly and it was generated with the help of /prueba command line option.

Another common error was in the function - depurar_xml(). This function was responsible for creating request.xml and response.xml files. Due to the difference in ways in which strings are handled in Python 2 and Python 3, this function required some modification in order to generate correct ouputs.
Similarly there were many instances where unicode errors were causing problems.

One other major problem that I faced was the presence of the sys.exit(0) statement that was present in all 4 scripts along with the command line options. Normally these statements seem harmless, but if pytest encounters them while testing, it leads to failure. Same goes with the input() function.
Example:-
        if "/dummy" in sys.argv:
            print("Consultando estado de servidores...")
            ws.Dummy()
            print("AppServerStatus", ws.AppServerStatus)
            print("DbServerStatus", ws.DbServerStatus)
            print("AuthServerStatus", ws.AuthServerStatus)
            sys.exit(0)

Writing a test for the /dummy command line option, would give an error when run.

Same goes with statements like these:-
        if "/consultarcaea" in sys.argv:
            periodo = input("Periodo: ")
            orden = input("Orden: ")

Here the error is due to the input() method, as pytest doesn't allow them. To make the above command line option work one has to read the command line arguments.
Something like this:-
            if len(sys.argv) > sys.argv.index("/consultarcaea") + 1:
                periodo = sys.argv[sys.argv.index("/consultarcaea") + 1]
                orden = sys.argv[sys.argv.index("/consultarcaea") + 2]
            else:
                periodo = input("Periodo (año-mes, ej 201108): ")
                orden = input("Orden (quincena, 1 u 2): ")


Overall I had fun while solving these problems. Watching little green dots appear after the pytest command offer an another level of satisfaction and content 😉.
View Blog Post
DJDT

Versions

Time

Settings from gsoc.settings

Headers

Request

SQL queries from 1 connection

Static files (2312 found, 3 used)

Templates (28 rendered)

Cache calls from 1 backend

Signals

Log messages