Skip to content

Dataprocessing features

Peter Lünenschloß requested to merge dataprocessing_features into develop

Quite a lot minor movements/changes and some data processing tools

  1. introduced module proc_functions, containing:
  • proc_interpolateMissing - a registered function for interpolation
  • proc_resample - a resampler for timeseries, mimicing pandas .resample method, including parameters for "nan_control"
  • proc_transform - function to apply timeseries transformations on the data
  1. trimmed down the harm_functions module:
  • removed from_merged/to_merged
  • removed downsample (replaced by proc_functions.proc_resample)
  • moved/renamed its _interpolate method to ts_operators.interpolateNANs, since it is used now not only in harmonization context
  1. extended ts_operators module:
  • interpolateNANs is a usual timeseries operator now and the strings '"linear"' and '"polynomial"', associated with wrappers of interpolateNANs can be passed as arguments to resampling/transformation procedures
  • extended transformation string syntax by the $ sign to allow parameter passing. passing "linear$3" refers to linear interpolation of up to 3 values and so on. (See details in doku -> doku comes soo)
  1. added test_module for processing functions
  • its so boring to write tests

Merge request reports