Skip to content
Snippets Groups Projects
Commit a45ad76a authored by Peter Lünenschloß's avatar Peter Lünenschloß
Browse files

bucks

parent 814cc3d5
No related branches found
No related tags found
7 merge requests!685Release 2.4,!684Release 2.4,!567Release 2.2.1,!566Release 2.2,!501Release 2.1,!372fix doctest snippets,!369Current documentation
......@@ -13,6 +13,32 @@
:width: 184 px
:target: https://www.ufz.de/index.php?de=45348
.. |sacRaw| image:: /ressources/images/Representative/RawData.png
:height: 150 px
:width: 288 px
.. |sacFlagged| image:: /ressources/images/Representative/DifferentFlags.png
:height: 150 px
:width: 288 px
.. |sacMV| image:: /ressources/images/Representative/MultivarSAC.png
:height: 150 px
:width: 288 px
.. |sacProc| image:: /ressources/images/Representative/ProcessingDrift.png
:height: 165 px
:width: 316 px
.. |pyLogo| image:: /ressources/images/Representative/pythonLogo.png
:height: 108 px
:width: 105 px
.. |csvConfig| image:: /ressources/images/Representative/csvConfig.png
:height: 100 px
:width: 176 px
.. image:: /ressources/images/Representative/gitlabLogo.png
:height: 100 px
:width: 110 px
......@@ -33,8 +59,35 @@ even for experts. SaQC is both, a Python framework and a command line applicatio
addresses the exploratory nature of quality control by offering a
continuously growing number of quality check routines through a flexible
and simple configuration system.
Below its user interface, SaQC is highly customizable and extensible.
A modular structure and well-defined interfaces make it easy to extend
the system with custom quality checks. Furthermore, even core components like
the flagging scheme are exchangeable.
--------
Features
--------
.. list-table::
* - |pyLogo| |csvConfig|
* SaQC is readily available from PyPI
* use SaQC as a commandline application and configure your labeling pipelines via plain .csv files
* or use the SaQC python API, enabling integration in your python processing script
* - |sacRaw|
- * easily load data from multiple sources, concatenating them in a SaQC object
* preprocess your data, by aligning it to equidistant frequency grids
* - |sacFlagged|
- * ally basic plausibillity checks, as well as
* more complex, univariat flagging Functions
* automatically keep track of labeling history and label significance
* - |sacProc|
- * modifyyour data, by interpolations, corrections and transformations
* calculate data products, such as residues or outlier scores
* automatically keep track of labeling history and label significance
* - |sacMV|
- * apply multivariate flagging function
varname ; test
#----------;---------------------------------------
level ; outliers.flagRange(min=0)
water_temp ; outliers.flagRange(min=-.5)
sak254 ; outliers.flagRange(min=0, max=70)
turbidity ; outliers.flagRange(min=0, max=140)
# interpolation resampling
level ; resample.linear(freq='15min')
water_temp ; resample.linear(freq='15min')
sak254 ; resample.linear(freq='15min')
turbidity ; resample.linear(freq='15min')
# Drift Correction
sak254 ; drift.correctDrift(target='sak254_corr', maintenance_field='maint', model=expDriftModel)
turbidity ; drift.correctDrift(target='turbidity_corr', maintenance_field='maint', model=expDriftModel)
# flag maintenance intervals
sak254 ; flagtools.flagManual(mdata='maint')
sak254_corr ; flagtools.flagManual(mdata='maint')
turbidity ; flagtools.flagManual(mdata='maint')
turbidity_corr ; flagtools.flagManual(mdata='maint')
# linearly fill singleton gaps
level ; interpolation.interpolateInvalid('linear', limit=2)
water_temp ; interpolation.interpolateInvalid('linear', limit=2)
sak254 ; interpolation.interpolateInvalid('linear', limit=2)
turbidity ; interpolation.interpolateInvalid('linear', limit=2)
# transform variables
level ; transformation.transform(target='norm_level', func='zNorm', freq='20D')
sac_corr ; transformation.transform(target='norm_SAK', func='zNorm', freq='20D')
turbidity_corr ; transformation.transform(target='norm_turbidity_corr', func='zNorm', freq='20D')
water_temp ; transformation.transform('norm_water_temp', func='zNorm', freq='20D')
# nearest neighbor scoring
varname ; test
#---------------;---------------------------------------
level ; flagRange(min=0)
water_temp ; flagRange(min=-.5)
sak254 ; flagRange(min=0, max=70)
turbidity ; flagRange(min=0, max=140)
level ; linear(freq='15min')
water_temp ; linear(freq='15min')
sak254 ; linear(freq='15min')
turbidity ; linear(freq='15min')
sak254 ; correctDrift(target='sak254_corr', maintenance_field='maint', model=expDriftModel)
turbidity ; correctDrift(target='turbidity_corr', maintenance_field='maint', model=expDriftModel)
sak254 ; flagManual(mdata='maint')
sak254_corr ; flagManual(mdata='maint')
turbidity ; flagManual(mdata='maint')
turbidity_corr ; flagManual(mdata='maint')
level ; interpolateInvalid('linear', limit=2)
water_temp ; interpolateInvalid('linear', limit=2)
sak254 ; interpolateInvalid('linear', limit=2)
turbidity ; interpolateInvalid('linear', limit=2)
level ; transform(target='norm_level', func='zNorm', freq='20D')
sac_corr ; transform(target='norm_SAK', func='zNorm', freq='20D')
turbidity_corr ; transform(target='norm_turbidity_corr', func='zNorm', freq='20D')
water_temp ; transform('norm_water_temp', func='zNorm', freq='20D')
norm_level, norm_sac_corr, norm_turbidity_corr, norm_water_temp ; scores.assignKNNScore(targe='kNN_scores', freq='20D')
kNN_scores ; outliers.flagByStray(freq='20D')
# project stray results
level, sac_corr, turbidity_corr, water_temp;generic.projectFlags(source='kNNscores', 'match')
# undo interpolation resampling
level ; generic.projectFlags(source='level_original', 'inverse_interpolation')
......
sphinx-doc/ressources/images/Representative/csvConfig.png

66.5 KiB

sphinx-doc/ressources/images/Representative/pythonLogo.png

13.6 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment