GSoC: Week 9: ConfigParser()

Niraj-Kamdar
Published: 07/26/2020

What did I do this week?

I have done research on various configuration file formats and compiled outcomes of it in a issue:  Discussion: Configuration file format. Some users recommended INI files because it is very old and still popular among masses but  INI file does not have any built-in type support and It also lacks formal specification. It parses everything as string. So, we have to process data parsed by configparser to convert it into something usable.
Our example data can be parsed as following dictionary:

{
    "checker": {
        "runs": "[curl,binutils]",  # This has to be transformed into list 
        "skips": "[python,bzip2]"
    },
    "input": {
        "directory": "test/assets",
        "input_file": "test/csv/triage.csv"
    },
}

So, parsing INI file won't be as easy as TOML or YAML which supports complex datatypes by default. It is also not easy to parse other datatypes like integer, float etc.

TOML is very similar to INI file and TOML also supports complex data types by default.

{
    'checker': {
        'runs': ['curl', 'binutils'],  # this is correctly parsed as list
        'skips': ['python', 'bzip2']
    },
    'input': {
        'directory': 'test/assets',
        'input_file': 'test/csv/triage.csv'
    },
}

I concluded that TOML and YAML are both very easy to read and write by both machine and human. So, we should use one of them. We have discussed which format to use in meeting and my mentors had various opinions on it. Summary of our discussion was: "The top contenders among our team seem to be TOML (readable, familar to python folk and close enough to INI for skill transfer for windows folk) and YAML (which might be a better fit for the dev-ops community that we hope will be among the biggest users of cve-bin-tool)."

Since Parsers for both formats produce similar python structures, I have created ConfigParser class which can parse both YAML and TOML file format. I have also added basic tests for it. I have also changed architecture of main function of cli.py to add support for config files and I also made sure that option given from terminal get preference over config option. I am also going to add tests for this. I have also fixed quiet mode bugs.

What am I doing this week? 

I am going to write tests for config files in test_cli.py and since I have completed almost all work related to InputEngine, I think it's good time to document it. 

Have I got stuck anywhere?

Yes, I need my Quiet mode bug fix PR merged since I have changed TestCLI in it and I need latest TestCLI for testing ConfigParser.

 

DJDT

Versions

Time

Settings from gsoc.settings

Headers

Request

SQL queries from 1 connection

Static files (2312 found, 3 used)

Templates (11 rendered)

Cache calls from 1 backend

Signals

Log messages