Skip to content
Snippets Groups Projects

Inter limit fix

Merged Peter Lünenschloß requested to merge interLimitFix into develop
All threads resolved!
2 files
+ 16
43
Compare changes
  • Side-by-side
  • Inline
Files
2
+ 6
4
@@ -314,11 +314,11 @@ def interpolateNANs(
:param method: String. Method keyword designating interpolation method to use.
:param order: Integer. If your desired interpolation method needs an order to be passed -
here you pass it.
:param inter_limit: Integer. Default = 2. Limit up to which consecutive nan - values in the data get
:param inter_limit: Integer. Default = 2. Number up to which consecutive nan - values in the data get
replaced by interpolation.
Its default value suits an interpolation that only will apply to points of an
inserted frequency grid. (regularization by interpolation)
Gaps wider than "limit" will NOT be interpolated at all.
Gaps of size "limit" or greater will NOT be interpolated at all.
:param downgrade_interpolation: Boolean. Default False. If True:
If a data chunk not contains enough values for interpolation of the order "order",
the highest order possible will be selected for that chunks interpolation.
@@ -330,11 +330,13 @@ def interpolateNANs(
if inter_limit is None:
# if there is actually no limit set to the gaps to-be interpolated, generate a dummy mask for the gaps
gap_mask = pd.Series(True, index=data.index, name=data.name)
elif inter_limit < 2:
return data
else:
# if there is a limit to the gaps to be interpolated, generate a mask that evaluates to False at the right side
# of each too-large gap with a rolling.sum combo
gap_mask = data.isna().rolling(inter_limit, min_periods=0).sum() != inter_limit
if inter_limit < 20:
if inter_limit == 2:
# for the common case of inter_limit=2 (default "harmonisation"), we efficiently bag propagate the False
# value to fill the whole too-large gap by a shift and a conjunction.
gap_mask &= gap_mask & gap_mask.shift(-1, fill_value=True)
@@ -363,7 +365,7 @@ def interpolateNANs(
)
else:
# if the method that is inerpolated with depends on not only the left and right border points of any gap,
# if the method that is interpolated with depends on not only the left and right border points of any gap,
# but includes more points, it has to be applied on any data chunk seperated by the too-big gaps individually.
# So we use the gap_mask to group the data into chunks and perform the interpolation on every chunk seperatly
# with the .transform method of the grouper.
Loading