FunctionDescriptions.md

Implemented Quality Check Functions

Index of the main documentation of the implemented functions, their purpose and parametrization.

Index

• Miscellaneous
  • range
  • seasonalRange
  • isolated
  • missing
  • clear
  • force
• Spike Detection
  • spikes_basic
  • spikes_simpleMad
  • spikes_slidingZscore
  • spikes_spektrumBased
• Constant Detection
  • constant
  • constants_varianceBased
• Break Detection
  • breaks_spektrumBased
• Time Series Harmonization
  • harmonize_shift2Grid
  • harmonize_aggregate2Grid
  • harmonize_linear2Grid
  • harmonize_interpolate2Grid
  • harmonize_downsample
  • harmonize
  • deharmonize
• Soil Moisture
  • soilMoisture_spikes
  • soilMoisture_breaks
  • soilMoisture_plateaus
  • soilMoisture_byFrost
  • soilMoisture_byPrecipitation
• Machine Learning
  • machinelearning

Machine Learning

machinelearning

machinelearning(references, window_values, window_flags, path)

parameter	data type	default value	description
references	string or list of strings		the fieldnames of the data series that should be used as reference variables
window_values	integer		Window size that is used to derive the gradients of both the field- and reference-series inside the moving window
window_flags	integer		Window size that is used to count the surrounding automatic flags that have been set before
path	string		Path to the respective model object, i.e. its name and the respective value of the grouping variable. e.g. "models/model_0.2.pkl"

This Function uses pre-trained machine-learning model objects for flagging. This requires training a model by use of the training script provided. For flagging, inputs to the model are the data of the variable of interest, data of reference variables and the automatic flags that were assigned by other tests inside SaQC. Therefore, this function should be defined last in the config-file, i.e. it should be the last test that is executed. Internally, context information for each point is gathered in form of moving windows. The size of the moving windows for counting of the surrounding automatic flags and for calculation of gradients in the data is specified by the user during model training. For the model to work, the parameters 'references', 'window_values' and 'window_flags' have to be set to the same values as during training. For a more detailed description of the modeling aproach see the training script.