Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
SaQC
Manage
Activity
Members
Labels
Plan
Issues
35
Issue boards
Milestones
Wiki
Code
Merge requests
7
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
rdm-software
SaQC
Commits
087a3bef
Commit
087a3bef
authored
3 years ago
by
David Schäfer
Browse files
Options
Downloads
Patches
Plain Diff
fixed stuff
parent
85fb44f3
No related branches found
Branches containing commit
No related tags found
Tags containing commit
7 merge requests
!685
Release 2.4
,
!684
Release 2.4
,
!567
Release 2.2.1
,
!566
Release 2.2
,
!501
Release 2.1
,
!372
fix doctest snippets
,
!369
Current documentation
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
sphinx-doc/getting_started/TutorialAPI.rst
+27
-20
27 additions, 20 deletions
sphinx-doc/getting_started/TutorialAPI.rst
with
27 additions
and
20 deletions
sphinx-doc/getting_started/TutorialAPI.rst
+
27
−
20
View file @
087a3bef
...
@@ -5,6 +5,7 @@ General structure
...
@@ -5,6 +5,7 @@ General structure
-----------------
-----------------
The Python API of SaQC consists of three distinct components:
The Python API of SaQC consists of three distinct components:
1. the core class ``SaQC``
1. the core class ``SaQC``
2. a number of flagging schemes
2. a number of flagging schemes
3. a collection of functions
3. a collection of functions
...
@@ -12,13 +13,14 @@ The Python API of SaQC consists of three distinct components:
...
@@ -12,13 +13,14 @@ The Python API of SaQC consists of three distinct components:
One and two are implemented as distinct classes, the core object is called ``SaQC`` and we currently
One and two are implemented as distinct classes, the core object is called ``SaQC`` and we currently
provide three flagging schemes, namely:
provide three flagging schemes, namely:
1. ``FloatTranslators``: Provides the quality flags ``-np.inf`` and ``[0..255]``. ``-np.inf`` denotes the
1. ``FloatScheme``: Provides the quality flags ``-np.inf`` and ``[0..255]``.
absence of quality flags and ``255`` denotes, that the associated data value is considered to be bad.
``-np.inf`` denotes the absence of quality flags and ``255`` denotes, that
the absence of a flags, ``1`` the presence of flag (i.e. one of the tests provided a positive result)
the associated data value is considered to be bad. the absence of a flags,
``1`` the presence of flag (i.e. one of the tests provided a positive result)
and ``0`` acts as an indicator for an tested-to-be-not-bad value.
and ``0`` acts as an indicator for an tested-to-be-not-bad value.
2. ``Simple
Translator
``: Provides three distinct quality labels, namely ``UNFLAGGED``, ``BAD``, ``OKAY``.
2. ``Simple
Scheme
``: Provides three distinct quality labels, namely ``UNFLAGGED``, ``BAD``, ``OKAY``.
3. ``Dmp
Translator
``: Provides the four distinct flags ``NIL``, ``OK``, ``DOUBTFUL``, ``BAD``, whereas each
3. ``Dmp
Scheme
``: Provides the four distinct flags ``NIL``, ``OK``, ``DOUBTFUL``, ``BAD``, whereas each
flag is extended by information about the generating function and optional comments
flag is extended by information about the generating function and optional comments
.
The third component, the actual test functions, appear as methods of ``SaQC``.
The third component, the actual test functions, appear as methods of ``SaQC``.
...
@@ -27,8 +29,8 @@ Getting started - Put something in
...
@@ -27,8 +29,8 @@ Getting started - Put something in
----------------------------------
----------------------------------
The definition of a ``SaQC`` test suite starts with some data as a ``pandas.DataFrame`` and the selection
The definition of a ``SaQC`` test suite starts with some data as a ``pandas.DataFrame`` and the selection
of an appropriate flagging scheme. For reasons of simplicity, we'll use the ``Simple
Translator
`` throughout
of an appropriate flagging scheme. For reasons of simplicity, we'll use the ``Simple
Scheme
`` throughout
the following examples. However, as the flagging schemes are interchangable, replacing the ``Simple
Translator
``
the following examples. However, as the flagging schemes are
mostly
interchangable, replacing the ``Simple
Scheme
``
with something more elaborate, is in fact a one line change. So let's start with:
with something more elaborate, is in fact a one line change. So let's start with:
.. code-block:: python
.. code-block:: python
...
@@ -60,22 +62,23 @@ available functions appear as methods of the ``SaQC`` class, so we can add a te
...
@@ -60,22 +62,23 @@ available functions appear as methods of the ``SaQC`` class, so we can add a te
qc.flagRange("a", min=20, max=80)
qc.flagRange("a", min=20, max=80)
`
`flagRange
`
` is the easiest of all functions and simply marks all values
smaller than ``min`` and larger
:py:func:
`flagRange
<Functions.saqc.flagRange>
` is the easiest of all functions and simply marks all values
than ``max``. This feature by itself wouldn't be worth the trouble of getting
into ``SaQC``, but it serves
smaller than ``min`` and larger
than ``max``. This feature by itself wouldn't be worth the trouble of getting
as a simple example. All functions expect the name of a column in
``data`` as the first positional argum
en
t
into ``SaQC``, but it serves
as a simple example. All functions expect the name of a column in
the giv
en
(called ``field``). The function ``flagRange`` (like all other
functions for that matter) is then called on
``data`` as their first positional argument
(called ``field``). The function ``flagRange`` (like all other
the given ``field`` (only).
functions for that matter) is then called on
the given ``field`` (only).
Each call to a ``SaQC`` method returns a new object (all itermediate objects share the main internal data
Each call to a ``SaQC`` method returns a new object (all itermediate objects share the main internal data
structures, so we only create shallow copies). Setting up more complex quality control suites is therefore
structures, so we only create shallow copies). Setting up more complex quality control suites (here by calling
simply a matter of method chaining.
the additional methods :py:func:`flagConstants <Functions.saqc.flagConstants>` and
:py:func:`flagByGrubbs <Functions.saqc.flagByGrubbs>`) is therefore simply a matter of method chaining.
.. code-block:: python
.. code-block:: python
# execute some tests
# execute some tests
qc = (qc
qc = (qc
.flagConstants("a", thresh=0.1, window=
"4D"
)
.flagConstants("a", thresh=0.1, window=
4
)
.flagByGrubbs("a", window=
"
10
D"
)
.flagByGrubbs("a", window=10)
.flagRange("a", min=20, max=80))
.flagRange("a", min=20, max=80))
...
@@ -135,8 +138,12 @@ The snippet below provides you with a compete example from the things we have se
...
@@ -135,8 +138,12 @@ The snippet below provides you with a compete example from the things we have se
Can I get something visual, please?
Can I get something visual, please?
-----------------------------------
-----------------------------------
Yes, you can. We provide an elaborated plotting method to generate and show or write matplotlib figures.
We provide an elaborated plotting method to generate and show or write matplotlib figures. Building on
Building on the example :ref:`above <getting_started/TutorialAPI:putting it together - a complete workflow>`
the example :ref:`above <getting_started/TutorialAPI:putting it together - the complete workflow>` above
the calling the method ``qc.plot("a")`` will generate a plot like the following:
simply call:
.. code-block:: python
qc.plot("a")
.. image:: /ressources/images/tutorial_api_1.png
.. image:: /ressources/images/tutorial_api_1.png
This diff is collapsed.
Click to expand it.
David Schäfer
@schaefed
mentioned in commit
63b55c6d
·
2 years ago
mentioned in commit
63b55c6d
mentioned in commit 63b55c6d7dadb0e612b23a897f292d5ffc14cb52
Toggle commit list
David Schäfer
@schaefed
mentioned in commit
684dc8a0
·
2 years ago
mentioned in commit
684dc8a0
mentioned in commit 684dc8a0515470d644fc85fee95d07661c8dd572
Toggle commit list
David Schäfer
@schaefed
mentioned in commit
8f7a90e4
·
1 year ago
mentioned in commit
8f7a90e4
mentioned in commit 8f7a90e4aed61c79a9dc8d67541a46beba0907e8
Toggle commit list
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment