Skip to content

new Flagger and Stuff

Bert Palm requested to merge (removed):develop into develop

this is currently actively developed under https://git.ufz.de/rdm/saqc/

ChangeLog

  • rm all old Flagger (Simple/Dmp/Positional/...)
  • in docstrings: saqc.flagger.BaseFlagger becomes saqc.flagger.Flagger
  • rm flagger from SaQC init (because its always the same, an initialized flagger object can still be passed via flags-paramter)
  • moved all masking magic from core to register (may we should find a better name/place for this?)
  • masking is now directly invoked on a call to a function (via @register decorator)
  • rm force-support
  • replaced all occurence of flagger.BAD/DOUBT/etc with plain BAD/DOUBT/etc., which live in saqc.common.py
  • to_mask is now scalar and not listlike anymore and use > instead of == for comparison (see #160 (closed))
  • we now pass cleared flags with each call to a saqc-function
  • flagger.getFlags(field) -> flagger[field]
  • flagger.setFlags(field, mask, flag) -> flagger[mask, field] = flag
  • flagger.initFlags(data=data) -> initFlagsLike(data) a function in saqc.flagger.flags
  • flagger.initFlags(flags=flags) -> Flagger(flags) flags is existing Flagger/Dataframe/DictOfSeries with flag values
  • rm flagger.merge()
  • rm flagger.slice()
  • flag=SOME_EDAULT now is exposed to all relevant functions
  • signatures in function are now identical with Saqc-class functions
  • fuzzy tests only run on scheduled pipelines (if setup in Gitlab->CI/CD->Schedules (very easy))

Followup todo's (in new MR's)

  • make every TODO in code an issue and add a reference to the issue in TODO text
  • clean up core.py
  • rewrite none-fuzzy pytests for masking
  • provide flagging schemes / translations (eg. Dmp, Simple, Positional, etc.)
Edited by Bert Palm

Merge request reports