TedLawson's Blog

Week 11 Review

TedLawson
Published: 08/15/2023

Accomplishments:

This week I had one PR merged, and created an additional 3 more PRs relating to the test suite cleanup and test coverage increases. Starting with the merged PR - 

- Setting for number format in archive tab #1719 - https://github.com/borgbase/vorta/pull/1719 - In this PR I created a setting in the Misc Tab that allows users to decide if they want their units of size in the archiver tab to be displayed in one fixed unit (such as all in KB or all in GB), or if they would rather have them displayed in dynamic units based on what is most appropriate for the archive size. This setting is important because some users prefer to have all files displayed in the same unit so they can easily compare archives to each other. Some prefer to have their archive sizes in a more natural display with dynamic units, so I implemented this option. After creating the setting and adding functionality to it, I also added tests for the new setting as well as cleaned up and parametrized the existing utils tests. Getting this PR merged is my first UI related change for Vorta, and I am proud to have it complete!

Additionally, I created 3 new PRs relating to increasing the code coverage for vorta and cleaning up existing tests:

- PR: tests/unit/test_repo - Clean up, parametrize, and increase test coverage - https://github.com/borgbase/vorta/pull/1771
- PR: tests/unit/test_source: cleanup, parametrize, and increase test coverage - https://github.com/borgbase/vorta/pull/1772
- PR: tests/unit/test_import_export: increased test coverage - https://github.com/borgbase/vorta/pull/1774
Each of these PR's did a combination of cleaning up existing tests by reducing code duplication and increasing readability, as well as adding new tests to increase Vortas overall test coverage.

Finally, I attending the GSoC contributor final evaluation meeting for one hour on Tuesday morning where I listened to the GSoC admin discuss what we need to do to prepare for the final weeks of the program. I also scheduled and attended a meeting with my mentor, Manu, where we discussed progress on the test coverage project thus far, and where to go from here.

Challenges:

Things went well this week, and I did not identify any real challenges outside of just continuing to learn how to test some of the QT widgets and tables. Mostly it was a very smooth learning experience. I am continuing to learn how to identify what needs to be tested and how to approach testing in a way that adds meaningful value to the test suite.

The week ahead:

This week I believe I will be able to finish the Vorta test suite fixes. I am mostly done with the unit tests and can begin on the integration tests after that. If I am able to finish the test suite fixes this week then I will be going into next week ready for the UI projects that were part of my original proposal.

View Blog Post

Week 10 Review

TedLawson
Published: 08/07/2023

Accomplishments:

This week I continued work on the GSoC project involving Vorta's test suite. This project has 3 main goals:

- Clean up the Vorta test suite 

- Reduce amount of duplicated code through use of parametrization

- Increase code coverage from ~70% to ~80%

 

I submitted three draft PRs this week to begin working on this:

- PR: utils unit tests: parametrize and increase test coverage #1768

- https://github.com/borgbase/vorta/pull/1768

- Description: It is possible to reduce code duplication in the utils unit tests by using parametrization. Additionally, there are a some utils functions that still needed testing. In this PR, I aim to solve both of these issues.

- PR: archive unit tests: reduce code duplication and increase test coverage #1769

- https://github.com/borgbase/vorta/pull/1769

- Description: I can reduce code duplication in the archive unit tests by using a fixture to handle the repeated setup. Additionally, there are a some `archive_tab` methods that still needed testing. In this PR, I aim to solve both of these issues.

- PR: diff unit tests: increase test coverage #1770

https://github.com/borgbase/vorta/pull/1770

- Description: There are some areas of opportunity in the `diff_result` module for increased test coverage of some class methods and functions. The `test_diff` module is already well parametrized and does not have a notable amount of code duplication, so here I would like to focus mainly on increasing code coverage.

Challenges:

The biggest hurdle I faced this week was my lessening my knowledge gap when it comes to working with PyQt. As I began writing unit tests for functions and methods, I found myself reading a lot of the documentation in order to understand how to properly test the different Qt elements. For example, when I was testing how to double-click on an archive name in the archive tab, I had to find out how to get the QPoint (a set of x-y coordinates) that I wanted to double click. To do this, I had to retrieve the item from the table, get the coordinates of the four sides of the rectangle that made up the item, and then double click the "center" of those points. I am still getting used to tables and and models, so the more experience I can get testing these UI elements for proper functionality the better.

The week ahead:

In the next week I have a GSoC meeting Tuesday with the coordinators of the program. During this meeting they will outline the final weeks of the program and how to begin getting a portfolio of all of our contributions together for display on the GSoC website after the program is over. I will continue to work my way through the rest of the test files and reduce code duplication, clean up tests where possible, and add additional functionality. I plan on continuing to do one PR for each test module as this keeps things very clean and will hopefully result in quicker approvals and merging. 

View Blog Post

Week 9 Review

TedLawson
Published: 08/01/2023

Accomplishments:

This week, I finished up the Borg test suite changes and got 3 PR's merged into Borg master. They are as follows:

PR: `testsuite/platform.py` conversion to pytest + remove `BaseTestCase` - https://github.com/borgbackup/borg/pull/7743 - This is one of the last remaining files in the test suite to have quite a bit of unittest and BaseTestCase usages, so I split it into its own PR. Here I converted all unittest to pytest, removed BaseTestCase entirely, and renamed some tests to follow the naming conventions of the classes they are being pulled from. Additionally, I split the `platform.py` test file into three subfiles: `platform_darwin`, `platform_linux`, and `platform_posix`. The original `platform` test file is now used to host tests that run on all platforms, and carries a collection of common pytest.mark.skip definitions that can be used by the other platforms. 

PR: `platform_freebsd.py` dummy test file - https://github.com/borgbackup/borg/pull/7748 - Created a dummy file for ACL testing FreeBSD. Relates to issue #7745. - Created at the request of one of my mentors, Thomas, who recommended starting a dummy file with tests that are skipped due to not being implemented yet. These tests will be implemented at a future date.

PR: Remove BaseTestCase from `testsuite/` - https://github.com/borgbackup/borg/pull/7742 - Now that the `ArchiverBaseTestCase` has been removed from `testsuite/archiver/__init__.py`, I now removed `BaseTestCase` from the `testsuite/__init__.py`. After @ThomasWaldmann's recent update to the `BaseTestCase`[here](https://github.com/borgbackup/borg/pull/7736), all that remained were a few assert methods that seem superfluous and could be replaced with the built-in python asserts. - This PR removed all remaining unittest and BaseTestCase from the testsuite folder, outside of the Borg selftest which relies on unittest.

Challenges:

This week I did not hit many roadblock - I am comfortable with the Borg testsutie at this point that I made most of these changes quickly and without too much assistance needed to get them merged quickly. As I move on to Vorta's test suite I imagine this background will come in very handy.

The week ahead:

I've talked about moving into Vorta's test suite a lot over the last few weeks, and then there was always something else that needed to be done with Borg! I can say with a lot of confidence that Borg's test suite is where we set out for it to be and I am proud of the work that has been accomplished there, with the massive amount of help from Thomas and the other contributors along the way. It feels good to have everything merged in and the test suite in a. really good place now. I have begin writing tests for the VOrta test suite and keeping them locally on my machine until the integration tests PR that is being worked on by Jetchirag is merged into master. So far I have completed some utils tests and plan to work on Scheduler and Notifications next, as there is some quick gains to be made here. 

View Blog Post

Week 8 Review

TedLawson
Published: 07/27/2023

Accomplishments:

Last week I was able to get the Archiver folder changes merged into Borg, which was a huge victory that took weeks to finish. This PR included over 6k lines of code, and completely removed the ArchiverBaseTestCase and all elements of unittest from the archiver test suite. After getting this merged, I started a new PR that worked on improving the merge by removing some redundant code and ensuring more consistency across the test suite, which was a big goal of the outline for this project. That PR was a little over a thousand lines and also got merged, meaning the archiver test folder is officially complete! On Friday I had a meeting with Manu to discuss some of the issues I was still having with testing on live Borg binaries and also to talk about work on Vorta's test suite.

Challenges:

This past week was one of many accomplishments, and I cannot overstate how big of a person victory it was for me to get the Archiver folder changes merged into Borg. However, I hit a serious roadblock when it came to testing on a Borg.exe binary. I was under the impression that to test on an '.exe' file I needed to have a virtual windows machine running on my Mac and ssh into it to run the tests, and I was having difficulties setting this up because the Borg vargrantfile was intended to be used with virtualbox, which does not support apple silicon arm64 architecture. Thankfully, my mentor Manu showed me Friday how simple it was to run tests on a Borg binary, and that the vagrant stuff was all a bit of a red herring that I was misunderstanding. After compiling a binary and getting the test suite to utilize it, I was finally able to see all 700+ tests pass without skipping the 200+ binary archiver ones. Previously, I relied on the GitHub CI to run these tests, and now I was able to test them myself.

The week ahead:
As I am writing this post a bit late,  Week 9 is already in full swing. I have spent some time working with the other GSoC contributors assigned to Borg and Vorta and checked out their branches locally to run and test the changes they have been making. I will continue to work with them and assist in ways that add value to their projects. Additionally, I am working to remove the last of the Unittest and BaseTestCase elements from the Borg `testsuite/` folder, a much smaller task now that the `testsuite/archiver/` folder is already complete. Finally, I am excited to continue working to improve Vorta's test coverage. I have already shown Manu a couple small tests and they looked good to him, so I am ready to get more tests started for Vorta, while also giving time for Jetchirag to finish his test suite changes. 

View Blog Post

Week 7 Review

TedLawson
Published: 07/18/2023

Accomplishments:

This week saw the merging of two PR's, one was the archiver test file which I parameterized and removed the unittest elements. The other was the repository test file which I converted from unittest to pytest and created fixtures to replace the setup and teardown methods. These are two files that I had finished a while ago but took some time to get them to a place where they were ready to be merged, and I am very happy to see them finally complete! I spent most of the week thereafter going through the archiver commands folder of tests that I had completed the previous week. My goal was to do a bit a of a quality check and ensure that I did not break any functionality of the tests in the process of converting them away from unittest to pytest. Additionally, I gathered feedback my from mentor Thomas and implemented several fixes based on his review. The archiver commands folder is now in a really good place, and I will continue to follow feedback from Thomas until that PR is able to be merged.

Challenges:

I definitely found myself hitting a couple roadblocks this week when trying to address feedback from Thomas. The first was building a binary for my tests to run on locally. I followed the instructions from the Borg documentation and took at look at the Vagrantfile, and was able to get the `Borg.exe` file created, however whenever I tried to run the tests with the binary I would get an error that the file was not found. I put a pause on this for now, but will do more research to overcome this issue and finally be able to run the tests on a binary.
My second roadblock came when I tried to pass `pytest_generate_tests`a list of `kinds` of tests that needed to be generated. Some test files only need local archivers, however some run each test on a combination of local archiver, remote archiver, and binary archiver, and the idea was to pass a list of `kinds` needed to `pytest_generate_tests` after it was imported to each test file. Ultimately, however, I ran into issues because it does not appear that this is a function that be called and passed a list the way other functions can be. My mentor recommended using a lambda function Orr `partial` to make it happen, but I was unsuccessful and will continue to work on this.

The week ahead:

As things get buttoned up with the Borg test suite, I am ready to move on to my next project: Improving Vorta's Test Coverage. This week I will create a coverage report and submit my findings and outline a proposed solution in a GitHub FR. I will then receive feedback and begin working on adding unit tests and parameterization where needed with the goal of improving Vorta's test coverage from its current standing of 72% to a goal of ~80%.

View Blog Post