Newer
Older
# Generic Functions
Generic Functions provide a possibility to implement simple quality checks
directly within the configuration using a simple, Python based extension
language.
## Specification
Generic funtions are used in the same manner as their
[non-generic counterparts](docs/FunctionDescriptions.md). The basic
signature looks like that:
```sh
flagGeneric(func=<expression>, flag=<flagging_constant>)
```
where `<expression>` is composed of the [supported constructs](#supported-constructs)
and `<flag_constant>` is either one of the predefined
[flagging constants](docs/ParameterDescriptions.md#flagging-constants) or any value supported
by the flagger in use.
## Examples
### Simple comparisons
#### Task
Flag all values of variable `x` when variable `y` falls below a certain threashold
#### Configuration file
| varname | test |
|---------|---------------------------|
| `x` | `flagGeneric(func=y < 0)` |
### Calculations
#### Task
Flag all values of variable `x` that exceed 3 standard deviations of variable `y`
#### Configuration file
| varname | test |
|---------|---------------------------------------|
| `x` | `flagGeneric(func=this > std(y) * 3)` |
### Special functions
#### Task
Flag variable `x` where variable `y` is flagged and variable `x` has missing values
#### Configuration file
| varname | test |
|---------|-------------------------------------------------------|
| `x` | `flagGeneric(func=this > isflagged(y) & ismissing(z)` |
## Variable References
All variables of the processed dataset are available within generic functions, so
arbitrary cross references are possible. The variable of intereset
is furthermore available with the special reference `this`, so the second
[example](#calculations) could be rewritten as:
| varname | test |
|---------|------------------------------------|
| `x` | `flagGeneric(func=x > std(y) * 3)` |
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
When referencing other variables, their flags will be respected during evaluation
of the generic expression. So, in the example above only previously
unflagged values of `x` and `y` are used within the expression `x > std(y)*3`.
## Supported constructs
### Operators
#### Comparsions
The following comparison operators are available:
| Operator | Description |
|----------|----------------------------------------------------------------------------------------------------|
| `==` | `True` if the values of the operands are equal |
| `!=` | `True` if the values of the operands are not equal |
| `>` | `True` if the values of the left operand are greater than the values of the right operand |
| `<` | `True` if the values of the left operand are smaller than the values of the right operand |
| `>=` | `True` if the values of the left operand are greater or equal than the values of the right operand |
| `<=` | `True` if the values of the left operand are smaller or equal than the values of the right operand |
#### Arithmetics
The following arithmetic operators are supported:
| Operator | Description |
|----------|----------------|
| `+` | addition |
| `-` | substraction |
| `*` | multiplication |
| `/` | division |
| `**` | exponantion |
| `%` | modulus |
#### Bitwise
The bitwise operators also act as logical operators in comparison chains
| Operator | Description |
|----------|-------------------|
| `&` | binary and |
| | | binary or |
| `^` | binary xor |
| `~` | binary complement |
### Functions
All functions expect a [variable reference](#variable-references)
as the only non-keyword argument (see [here](#special-functions))
| Name | Description |
|-------------|-----------------------------------|
| `abs` | absolute values of a variable |
| `max` | maximum value of a variable |
| `min` | minimum value of a variable |
| `mean` | mean value of a variable |
| `sum` | sum of a variable |
| `std` | standard deviation of a variable |
| `len` | the number of values for variable |
| `ismissing` | check for missing values |
| `isflagged` | check for flags |
### Constants
Generic functions support the same constants as normal functions, a detailed
list is available [here](docs/ParameterDescriptions.md#constants).