Reimplement rolling
The error not always appears - had no time since to investigate.
It appears when using the doc ressource data (path in snippet). Since this is the workshop data set, it disquallifies .roll
from being used in the workshop - wich is really bad, since the generic alias that applies pandas roll dircetly is most likely syntactically too complex for people that have to first understand what a rolling window function is....
The error applears when .roll
is re applied on its result.
I couldnt reproduce with less complex data, so the snippet uses the doc ressource directly.
i got the error in other situations too, that are harder to reproduce
Error disappears when data is harmonized
snippet
import saqc
import pandas as pd
# using doc ressource here:
path = '/docs/resources/data/SoilMoisture.csv'
data = pd.read_csv(path)
data = data.set_index('Unnamed: 0')
data.index = pd.DatetimeIndex(data.index)
qc = saqc.SaQC(data)
qc = qc.roll('SoilMoisture', window='1D', func=np.mean)
qc = qc.roll('SoilMoisture', window='1D', func=np.mean)
error msg:
Traceback (most recent call last):
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3369, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-41-116553a83be1>", line 1, in <cell line: 1>
qc = qc.roll('SoilMoisture', window='1D', func=np.mean)
File "/home/luenensc/PyPojects/saqc/saqc/core/register.py", line 397, in inner
out = func(out, **kwargs)
File "/home/luenensc/PyPojects/saqc/saqc/funcs/rolling.py", line 67, in roll
self._data, self._flags = _roll(
File "/home/luenensc/PyPojects/saqc/saqc/funcs/rolling.py", line 128, in _roll
centers.rolling(window, closed="both")
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 1661, in apply
return super().apply(
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 1192, in apply
return self._apply(
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 545, in _apply
return self._apply_blockwise(homogeneous_func, name)
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 419, in _apply_blockwise
return self._apply_series(homogeneous_func, name)
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 408, in _apply_series
result = homogeneous_func(values)
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 537, in homogeneous_func
result = calc(values)
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 531, in calc
return func(x, start, end, min_periods)
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/window/rolling.py", line 1217, in apply_func
return window_func(values, begin, end, min_periods)
File "pandas/_libs/window/aggregations.pyx", line 1173, in pandas._libs.window.aggregations.roll_apply
File "/home/luenensc/PyPojects/saqc/saqc/funcs/rolling.py", line 124, in center_func
pos = x.index[int(len(x) - y[x.index[-1]])]
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/indexes/extension.py", line 279, in __getitem__
result = self._data[key]
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/arrays/datetimelike.py", line 322, in __getitem__
result = super().__getitem__(key)
File "/home/luenensc/PyPojects/venv_saqc_py38/lib/python3.8/site-packages/pandas/core/arrays/_mixins.py", line 194, in __getitem__
result = self._ndarray[key]
IndexError: index 130 is out of bounds for axis 0 with size 130
Edited by Peter Lünenschloß