OutlierDetection.rst



Outlier Detection and Flagging
The tutorial aims to introduce the usage of saqc methods in order to detect outliers in an uni-variate set up.
The tutorial guides through the following steps:

We checkout and load the example data set. Subsequently, we initialise an :py:class:`SaQC <saqc.core.core.SaQC>` object.
:ref:`Preparation <cook_books/OutlierDetection:Preparation>`
:ref:`Data <cook_books/OutlierDetection:Data>`
:ref:`Initialisation <cook_books/OutlierDetection:Initialisation>`


We will see how to apply different smoothing methods and models to the data in order to obtain usefull residue
variables.
:ref:`Modelling <cook_books/OutlierDetection:Modelling>`
:ref:`Rolling Mean <cook_books/OutlierDetection:Rolling Mean>`
:ref:`Rolling Median <cook_books/OutlierDetection:Rolling Median>`
:ref:`Polynomial Fit <cook_books/OutlierDetection:Polynomial Fit>`
:ref:`Custom Models <cook_books/OutlierDetection:Custom Models>`


:ref:`Evaluation and Visualisation <cook_books/OutlierDetection:Visualisation>`


We will see how we can obtain residues and scores from the calculated model curves.
:ref:`Residues and Scores <cook_books/OutlierDetection:Residues and Scores>`
:ref:`Residues <cook_books/OutlierDetection:Residues>`
:ref:`Scores <cook_books/OutlierDetection:Scores>`
:ref:`Optimization by Decomposition <cook_books/OutlierDetection:Optimization by Decomposition>`


Finally, we will see how to derive flags from the scores itself and impose additional conditions, functioning as
correctives.
:ref:`Setting and Correcting Flags <cook_books/OutlierDetection:Setting and Correcting Flags>`
:ref:`Flagging the Scores <cook_books/OutlierDetection:Flagging the Scores>`
Additional Conditions ("unflagging")
:ref:`Including Multiple Conditions <cook_books/OutlierDetection:Including Multiple Conditions>`


Preparation

Data
The example data set
is selected to be small, comprehendable and its single anomalous outlier
can be identified easily visually:

It can be downloaded from the saqc git repository.