Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
SaQC
Manage
Activity
Members
Labels
Plan
Issues
36
Issue boards
Milestones
Wiki
Code
Merge requests
8
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
rdm-software
SaQC
Commits
5bca7181
Commit
5bca7181
authored
3 years ago
by
Peter Lünenschloß
Browse files
Options
Downloads
Patches
Plain Diff
clearing
parent
7a2effac
No related branches found
No related tags found
6 merge requests
!685
Release 2.4
,
!684
Release 2.4
,
!567
Release 2.2.1
,
!566
Release 2.2
,
!501
Release 2.1
,
!372
fix doctest snippets
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
sphinx-doc/cook_books/DataRegularisation.rst
+1
-0
1 addition, 0 deletions
sphinx-doc/cook_books/DataRegularisation.rst
sphinx-doc/getting_started/TutorialAPI.rst
+38
-8
38 additions, 8 deletions
sphinx-doc/getting_started/TutorialAPI.rst
with
39 additions
and
8 deletions
sphinx-doc/cook_books/DataRegularisation.rst
+
1
−
0
View file @
5bca7181
...
...
@@ -442,6 +442,7 @@ it can be of advantage, to flag data before regularisation in order to effective
from the resulting regularly sampled data set. Lets see an example for the *SoilMoisture* data set.
.. testsetup::
qc = qc.drop('SoilMoisture')
>>> qc = qc.linear('SoilMoisture', target='SoilMoisture_linear', freq='10min')
...
...
This diff is collapsed.
Click to expand it.
sphinx-doc/getting_started/TutorialAPI.rst
+
38
−
8
View file @
5bca7181
.. testsetup::
import numpy as np
import pandas as pd
from saqc import SaQC
# we need some dummy data
values = np.random.randint(low=0, high=100, size=100)
dates = pd.date_range(start="2020-01-01", periods=len(values), freq="D")
data = pd.DataFrame({"a": values}, index=dates)
# let's insert some constant values ...
data.iloc[30:40] = values.mean()
# ... and an outlier
data.iloc[70] = 175
# initialize saqc
qc = SaQC(data=data, scheme="simple")
# execute some tests
qc = (qc
.flagConstants("a", thresh=0.1, window="4D")
.flagByGrubbs("a", window="10D")
.flagRange("a", min=20, max=80))
# retrieve the data as a pandas.DataFrame
qc.data
# retrieve the flags as a pandas.DataFrame
qc.flags
Python API
==========
...
...
@@ -33,7 +63,7 @@ of an appropriate flagging scheme. For reasons of simplicity, we'll use the ``Si
the following examples. However, as the flagging schemes are mostly interchangable, replacing the ``SimpleScheme``
with something more elaborate, is in fact a one line change. So let's start with:
.. code
-block
:: python
..
test
code:: python
import numpy as np
import pandas as pd
...
...
@@ -58,9 +88,9 @@ Moving on - Quality control your data
The ``qc`` variable now serves as the base for all further processing steps. As mentioned above, all
available functions appear as methods of the ``SaQC`` class, so we can add a tests to our suite with:
.. code
-block
:: python
..
test
code:: python
qc.flagRange("a", min=20, max=80)
qc =
qc.flagRange("a", min=20, max=80)
:py:func:`flagRange <Functions.saqc.flagRange>` is the easiest of all functions and simply marks all values
smaller than ``min`` and larger than ``max``. This feature by itself wouldn't be worth the trouble of getting
...
...
@@ -73,7 +103,7 @@ structures, so we only create shallow copies). Setting up more complex quality c
the additional methods :py:func:`flagConstants <Functions.saqc.flagConstants>` and
:py:func:`flagByGrubbs <Functions.saqc.flagByGrubbs>`) is therefore simply a matter of method chaining.
.. code
-block
:: python
..
test
code:: python
# execute some tests
qc = (qc
...
...
@@ -90,7 +120,7 @@ we have seen above, calling quality checks does however not immediately return t
associated flags, but rather an new ``SaQC`` object. The actual execution products are accessible through a
number of different attributes, of which you likely might want to use the following:
.. code
-block
:: python
..
test
code:: python
# retrieve the data as a pandas.DataFrame
qc.data
...
...
@@ -103,7 +133,7 @@ Putting it together - The complete workflow
-------------------------------------------
The snippet below provides you with a compete example from the things we have seen so far.
..
code-block:: python
..
testcode::
import numpy as np
import pandas as pd
...
...
@@ -142,8 +172,8 @@ We provide an elaborated plotting method to generate and show or write matplotli
the example :ref:`above <getting_started/TutorialAPI:putting it together - the complete workflow>` above
simply call:
.. code
-block
:: python
..
test
code:: python
qc.plot("a")
qc =
qc.plot("a")
.. image:: /ressources/images/tutorial_api_1.png
This diff is collapsed.
Click to expand it.
David Schäfer
@schaefed
mentioned in commit
63b55c6d
·
2 years ago
mentioned in commit
63b55c6d
mentioned in commit 63b55c6d7dadb0e612b23a897f292d5ffc14cb52
Toggle commit list
David Schäfer
@schaefed
mentioned in commit
684dc8a0
·
2 years ago
mentioned in commit
684dc8a0
mentioned in commit 684dc8a0515470d644fc85fee95d07661c8dd572
Toggle commit list
David Schäfer
@schaefed
mentioned in commit
8f7a90e4
·
1 year ago
mentioned in commit
8f7a90e4
mentioned in commit 8f7a90e4aed61c79a9dc8d67541a46beba0907e8
Toggle commit list
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment