Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
SaQC
Manage
Activity
Members
Labels
Plan
Issues
36
Issue boards
Milestones
Wiki
Code
Merge requests
8
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
rdm-software
SaQC
Commits
54848a50
Commit
54848a50
authored
4 years ago
by
Bert Palm
🎇
Browse files
Options
Downloads
Patches
Plain Diff
bugfix.. and long insitu explanation
parent
f188a38c
No related branches found
Branches containing commit
No related tags found
Tags containing commit
3 merge requests
!193
Release 1.4
,
!188
Release 1.4
,
!166
Rolling rework fixes
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
saqc/lib/rolling.py
+15
-7
15 additions, 7 deletions
saqc/lib/rolling.py
test/lib/test_rolling.py
+12
-6
12 additions, 6 deletions
test/lib/test_rolling.py
with
27 additions
and
13 deletions
saqc/lib/rolling.py
+
15
−
7
View file @
54848a50
...
...
@@ -371,10 +371,18 @@ def customRoller(obj, window, min_periods=None, # aka minimum non-nan values
indexer
=
_VariableWindowDirectionIndexer
if
x
.
is_freq_type
else
_FixedWindowDirectionIndexer
indexer
=
indexer
(
index_array
=
x
.
_on
.
asi8
,
window_size
=
x
.
window
,
**
ours
)
# center offset is calculated from min_periods if a indexer is passed to rolling().
# if instead a normal (dt or num) window is passed, it is used for offset calculation.
# also if we pass min_periods == None or 0, all values will Nan in the result even if
# start[i]<end[i] as expected. So we cannot pass `center` to rolling. Instead we manually do the centering
# in the Indexer. To calculate min_periods (!) including NaN count (!) we need to pass min_periods, but
# ensure that it is not None nor 0.
return
obj
.
rolling
(
indexer
,
min_periods
=
indexer
.
min_periods
,
**
theirs
)
# Centering is fully done in our own indexers. So we do not pass center to rolling(). Especially because
# we also allow centering on dt-based indexes. Also centering would fail in forward windows, because of
# pandas internal centering magic (append nans at the end of array, later cut values from beginning of the
# result).
# min_periods is also quite tricky. Especially if None is passed. For dt-based windows min_periods defaults to 1
# and is set during rolling setup (-> if r=obj.rolling() is called). For numeric windows instead, it keeps None
# during setup and defaults to indexer.window_size if a rolling-method is called (-> if r.sum()). Thats a bit
# odd and quite hard to find. So we are good if we pass the already calculated x.min_periods as this will just
# hold the correct initialised or not initialised value. (It gets even trickier if one evaluates which value is
# actually passed to the function that actually thrown them out; i leave that to the reader to find out. start
# @ pandas.core.window.rolling:_Window._apply)
# Lastly, it is necessary to pass min_periods at all (!) and do not set it to a fix value (1, 0, None,...). This
# is, because we cannot throw out values by ourself in the indexer, because min_periods also evaluates NA values
# in its count and we have no control over the actual values, just their indexes.
return
obj
.
rolling
(
indexer
,
min_periods
=
x
.
min_periods
,
**
theirs
)
This diff is collapsed.
Click to expand it.
test/lib/test_rolling.py
+
12
−
6
View file @
54848a50
...
...
@@ -91,12 +91,15 @@ def runtest_for_kw_combi(s, kws, func='sum'):
return
getattr
(
roller
,
'
apply
'
)(
func
)
if
forward
:
expR
=
pd
.
Series
(
reversed
(
s
),
reversed
(
s
.
index
)).
rolling
(
**
kws
)
resR
=
customRoller
(
s
,
forward
=
True
,
**
kws
)
try
:
expR
=
pd
.
Series
(
reversed
(
s
),
reversed
(
s
.
index
)).
rolling
(
**
kws
)
expected
=
calc
(
expR
)[::
-
1
]
except
Exception
:
pytest
.
skip
(
"
pandas faild
"
)
pytest
.
skip
(
"
pandas failed
"
)
return
resR
=
customRoller
(
s
,
forward
=
True
,
**
kws
)
result
=
calc
(
resR
)
success
=
check_series
(
result
,
expected
)
...
...
@@ -104,12 +107,15 @@ def runtest_for_kw_combi(s, kws, func='sum'):
print_diff
(
s
,
result
,
expected
)
assert
False
,
f
"
forward=True !!
{
kws
}
"
else
:
expR
=
s
.
rolling
(
**
kws
)
resR
=
customRoller
(
s
,
**
kws
)
try
:
expR
=
s
.
rolling
(
**
kws
)
expected
=
calc
(
expR
)
except
Exception
:
pytest
.
skip
(
"
pandas faild
"
)
pytest
.
skip
(
"
pandas failed
"
)
return
resR
=
customRoller
(
s
,
**
kws
)
result
=
calc
(
resR
)
success
=
check_series
(
result
,
expected
)
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment