ConfigurationFiles.md

Configuration Files

The behaviour of SaQC is completely controlled by a text based configuration file.

Format

SaQC expects its configuration files to be semicolon-separated text files with a fixed header. Each row of the configuration file lists one variable and one or several test functions, which will be evaluated to procduce a result for the given variable.

Header names

The header names are basically fixed, but if you really insist in custom configuration headers have a look here.

Name	Data Type	Description	Optional
varname	string	name of a variable	no
test	function notation	test function	no
plot	boolean (True/False)	plot the test's result	yes

Test function notation

The notation of test functions follows the function call notation of Python and many other programming languages and looks like this:

flagRange(min=0, max=100)

Here the function flagRange is called and the values 0 and 100 are passed to the parameters min and max respectively. As we (currently) value readablity of the configuration more than conciseness of the extrension language, only keyword arguments are supported. That means that the notation flagRange(0, 100) is not a valid replacement for the above example.

Examples

Single Test

Every row lists one test per variable, if you want to call multiple tests on a specific variable (and you probably want to), list them in separate rows

varname | test
#-------|----------------------------------
x       | flagMissing()
x       | flagRange(min=0, max=100)
x       | constants_flagBasic(window="3h")
y       | flagRange(min=-10, max=40)

Multiple Tests

A row lists multiple tests for a specific variable in separate columns. All test columns need to share the common prefix test.

varname ; test_1                     ; test_2                    ; test_3
#-------;----------------------------;---------------------------;---------------------------------
x       ; flagMissing()              ; flagRange(min=0, max=100) ; constants_flagBasic(window="3h")
y       ; flagRange(min=-10, max=40) ;                           ;

The evaluation of such a configuration is in columns-major order, so the given example is identical to the following:

varname ; test_1                     
#-------;---------------------------------
x       ; flagMissing()
y       ; flagRange(min=-10, max=40)
x       ; flagRange(min=0, max=100)
x       ; constants_flagBasic(window="3h")