Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
SaQC
Manage
Activity
Members
Labels
Plan
Issues
36
Issue boards
Milestones
Wiki
Code
Merge requests
8
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
rdm-software
SaQC
Commits
bbc5f55e
Commit
bbc5f55e
authored
5 years ago
by
David Schäfer
Browse files
Options
Downloads
Patches
Plain Diff
Update ConfigurationFiles.md
parent
b6c56739
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Pipeline
#3245
passed with stage
in 7 minutes and 41 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
docs/ConfigurationFiles.md
+25
-25
25 additions, 25 deletions
docs/ConfigurationFiles.md
with
25 additions
and
25 deletions
docs/ConfigurationFiles.md
+
25
−
25
View file @
bbc5f55e
...
...
@@ -24,12 +24,12 @@ configuration headers have a look [here](saqc/core/config.py).
The notation of test functions follows the function call notation of Python and
many other programming languages and looks like this:
```
r
ange(min=0, max=100)
flagR
ange(min=0, max=100)
```
Here the function
`
r
ange`
is called and the values
`0`
and
`100`
are passed
Here the function
`
flagR
ange`
is called and the values
`0`
and
`100`
are passed
to the parameters
`min`
and
`max`
respectively. As we (currently) value readablity
of the configuration more than conciseness of the extrension language, only
keyword arguments are supported. That means that the notation
`
r
ange(0, 100)`
keyword arguments are supported. That means that the notation
`
flagR
ange(0, 100)`
is not a valid replacement for the above example.
## Examples
...
...
@@ -37,21 +37,21 @@ is not a valid replacement for the above example.
Every row lists one test per variable, if you want to call multiple tests on
a specific variable (and you probably want to), list them in separate rows
| varname | test |
|---------|--------------------------|
|
`x`
|
`
m
issing()`
|
|
`x`
|
`
r
ange(min=0, max=100)`
|
|
`x`
|
`constant(window="3h")`
|
|
`y`
|
`
r
ange(min=-10, max=40)`
|
| varname | test
|
|---------|--------------------------
-----------
|
|
`x`
|
`
flagM
issing()`
|
|
`x`
|
`
flagR
ange(min=0, max=100)`
|
|
`x`
|
`constant
s_flagBasic
(window="3h")`
|
|
`y`
|
`
flagR
ange(min=-10, max=40)`
|
### Multiple Tests
A row lists multiple tests for a specific variable in separate columns. All test
columns need to share the common prefix
`test`
.
| varname | test_1 | test_2 | test_3 |
|---------|--------------------------
|
-------------------------|-------------------------|
|
`x`
|
`
m
issing()`
|
`
r
ange(min=0, max=100)`
|
`constant(window="3h")`
|
|
`y`
|
`
r
ange(min=-10, max=40)`
|
|
|
| varname | test_1
| test_2
| test_3
|
|---------|--------------------------
----|----
-------------------------|-------------------------
-----------
|
|
`x`
|
`
flagM
issing()`
|
`
flagR
ange(min=0, max=100)`
|
`constant
s_flagBasic
(window="3h")`
|
|
`y`
|
`
flagR
ange(min=-10, max=40)`
|
|
|
### Plotting
As the process of finding a good quality check setup is somewhat experimental, SaQC
...
...
@@ -61,12 +61,12 @@ to `True` whenever you want to see the result of the evaluation. These plots are
meant to provide a quick and easy visual evaluation of the test setup and not to
yield 'publication-ready' results
| varname | test | plot |
|---------|--------------------------|-------|
|
`x`
|
`
m
issing()`
| |
|
`x`
|
`
r
ange(min=0, max=100)`
| False |
|
`x`
|
`constant(window="3h")`
| True |
|
`y`
|
`
r
ange(min=-10, max=40)`
| |
| varname | test
| plot |
|---------|--------------------------
-----------
|-------|
|
`x`
|
`
flagM
issing()`
| |
|
`x`
|
`
flagR
ange(min=0, max=100)`
| False |
|
`x`
|
`constant
s_flagBasic
(window="3h")`
| True |
|
`y`
|
`
flagR
ange(min=-10, max=40)`
| |
### Regular Expressions
Some of the most basic tests (e.g. checks for missing values or range tests) but
...
...
@@ -80,8 +80,8 @@ within the `varname` column. Please not that a `varname` needs to be quoted
| varname | test |
|--------------|--------------------------------------|
|
`'.*'`
|
`harm
onize
_shift2Grid(freq="15Min")`
|
|
`'(x \| y)'`
|
`
m
issing()`
|
|
`'.*'`
|
`harm_shift2Grid(freq="15Min")`
|
|
`'(x \| y)'`
|
`
flagM
issing()`
|
#### Bring it to a file
As mentioned above SaQC, expectd the configuration to be a table-like,
...
...
@@ -90,8 +90,8 @@ semicolon-separated text file. So the configuration from the
```
varname;test;plot
x;
m
issing();
x;
r
ange(min=0, max=100);False
x;constant(window="3h");True
y;
r
ange(min=-10, max=40);
x;
flagM
issing();
x;
flagR
ange(min=0, max=100);False
x;constant
s_flaBasic
(window="3h");True
y;
flagR
ange(min=-10, max=40);
```
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment