Skip to content

foundational timeseries model Regressor

Peter Lünenschloß requested to merge TTM into develop

Adding a regression function based on IBMs TinyTimeMixer foundational timeseries model.

Although the model is wrapped in a way, that it can be finetuned/trained through the saqc interface, its most interesting application might be the zero shot application, where the foundational model is applied without any training (and thus no additional parameterisation) like so (using the turbidity data from the doc ressources):

data = pd.read_csv('/resources/data/hydro_data.csv', parse_dates=['Timestamp'], index_col=0, nrows=10000)
data = data.resample('15min').bfill() # quick and dirty introducing regular sampling rate
data = data.fillna(method='pad',axis=0) # making sure no NaNS are present

qc = qc.fitTinyTimeMixer(['sac254_raw'], observed=['level_raw', 'water_temp_raw'],
                         target=['sac254_ttm'])
qc = qc.processGeneric(['sac254_raw','sac254_ttm'], func=lambda x,y:x-y, target='residue')
qc = qc.flagZScore('residue')
qc = qc.transferFlags('residue', target='sac254_raw')
qc.plot('sac254_raw')

wich already yields surprisingly good results! (Notably, there hadnt to be set any specific parameters.)

sacflagged

Unfortunately the additional packages needed (tsfm, pytorch, transformers) do not fit into the storage space of the CI container (getting not enough device storage in break report), so i could not add proper tests and had to conditionally import the required packages. I guess this is not optimal - especially regarding the fact that the functionality wont be tested. But i m not sure what to do about the storage limit. Can we elevate that?

Edited by Peter Lünenschloß

Merge request reports