Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
SaQC
Manage
Activity
Members
Labels
Plan
Issues
35
Issue boards
Milestones
Wiki
Code
Merge requests
7
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
rdm-software
SaQC
Commits
9632503e
Commit
9632503e
authored
3 years ago
by
Peter Lünenschloß
Browse files
Options
Downloads
Patches
Plain Diff
Update OutlierDetection.md
parent
0b5a89b7
No related branches found
No related tags found
7 merge requests
!685
Release 2.4
,
!684
Release 2.4
,
!567
Release 2.2.1
,
!566
Release 2.2
,
!501
Release 2.1
,
!372
fix doctest snippets
,
!369
Current documentation
Pipeline
#24909
passed with stage
in 1 minute and 48 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
sphinx-doc/cook_books_md/OutlierDetection.md
+33
-1
33 additions, 1 deletion
sphinx-doc/cook_books_md/OutlierDetection.md
with
33 additions
and
1 deletion
sphinx-doc/cook_books_md/OutlierDetection.md
+
33
−
1
View file @
9632503e
...
...
@@ -167,14 +167,46 @@ Also, as a result, all the values are centered and we dont have to care about wi
i_saqc
=
i_saqc
.
processGeneric
(
fields
=
[
'
incidents_residues
'
,
'
incidents_mean
'
,
'
incidents_std
'
],
target
=
'
incidents_scores
'
,
func
=
lambda
x
,
y
,
z
:
abs
((
x
-
y
)
/
z
))
```
## Setting Flags
Lets evaluate the residues calculation and have a look at the resulting scores:
```
python
i_saqc
=
i_saqc
.
evaluate
()
i_saqc
.
show
(
'
incidents_scores
'
)
```
## Setting Flag und unsetting Flags
We can now implement the common rule of thumb, that any Z-score value above 3, may indicate an outlierish data point, by:
```
python
i_saqc
=
i_saqc
.
flagRange
(
'
incidents_scores
'
,
max
=
3
).
evaluate
()
```
Now flags have been calculated for the scores:
```
python
i_saqc
.
show
(
'
incidents_scores
'
)
```
We now could project those flags onto our original incidents timeseries:
```
python
i_saqc
=
i_saqc
.
flagGeneric
(
field
=
[
'
incidents_scores
'
],
target
=
'
incidents
'
,
func
=
lambda
x
:
isFlagged
(
x
))
```
Note, that we could have skipped the range flagging step, by including the lowpassing in our generic expression:
```
python
i_saqc
=
i_saqc
.
flagGeneric
(
field
=
[
'
incidents_scores
'
],
target
=
'
incidents
'
,
func
=
lambda
x
:
x
>
3
)
```
Lets check the result:
```
python
i_saqc
=
i_saqc
.
evaluate
i_saqc
.
show
(
'
incidents
'
)
```
Obveously, there are some flags set, that relate to
This diff is collapsed.
Click to expand it.
David Schäfer
@schaefed
mentioned in commit
63b55c6d
·
2 years ago
mentioned in commit
63b55c6d
mentioned in commit 63b55c6d7dadb0e612b23a897f292d5ffc14cb52
Toggle commit list
David Schäfer
@schaefed
mentioned in commit
684dc8a0
·
2 years ago
mentioned in commit
684dc8a0
mentioned in commit 684dc8a0515470d644fc85fee95d07661c8dd572
Toggle commit list
David Schäfer
@schaefed
mentioned in commit
8f7a90e4
·
1 year ago
mentioned in commit
8f7a90e4
mentioned in commit 8f7a90e4aed61c79a9dc8d67541a46beba0907e8
Toggle commit list
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment