set flagged values to nan?

We currently replace all flagged values by np.nan before they are passed to the test functions. Is this a good idea?

A little more context:

This was necessary to propagate flags through the evaluator. The idea was that a call like:

flagGeneric(func=this < var1 + var2)

would return an aggregate of the result of the < condition and the flags of var1 and var2, i.e. this < var1 + var2 can only be True, where var1 and var2 are unflagged.

This restriction could be lifted with a function in evaluator.py like:

def _dslAggregateFunc(func, flagger, data):
    flags = flagger.isFlagged(data.name)
    return func(data.mask(flags))

and the following changes to initLocalEnv:

return {
        ...
        "mean": partial(_dslAggregateFunc, np.nanmean, flagger),
        "sum": partial(_dslAggregateFunc, np.nansum, flagger),
        "std": partial(_dslAggregateFunc, np.nanstd, flagger),
        ...
    }

Any options on that @palmb and @luenensc ?

Edited by David Schäfer