No xarray!

A trial to switch back to numpy arrays for performance. We do not even really use any xarray features currently.

For the cascading reservior example, which also does some calculations, this results in approx. 3x speedup.

Todo

• Get rid of xarray
• Rename to_xarray() (renamed to prepare())
• Rename xdata args
• Update the book
• Update the homepage

Benchmarks

When comparing columns, keep in mind that pushing xarray data basically requires creating the underlying numpy data first. So the most frequent use case is comparing first vs. last column. However, even compared to pushing xarray data we get quite some speedup for larger data.

Full runs

benchmark	before (numpy)	before (xarray)	after
run 365, 64x32	178 ms	29 ms	32 ms
run 365, 1024x512	289 ms	134 ms	127 ms
run 365, 2048x1024	862 ms	740 ms	481 ms
run 365, 64x32, copy	220 ms	57 ms	62 ms
run 365, 1024x512, copy	600 ms	426 ms	270 ms
run 365, 2048x1024, copy	2395 ms	2234 ms	915 ms
run 365, 64x32, units	-	109 ms	49 ms
run 365, 1024x512, units	-	483 ms	254 ms
run 365, 2048x1024, units	-	2044 ms	1138 ms

Push & pull

benchmark	before (numpy)	before (xarray)	after
push&pull 512x256	460 us	74 us	80 us
push&pull 1024x512	490 us	74 us	80 us
push&pull 2048x1024	475 us	74 us	80 us
push&pull 512x256, units	880 us	501 us	157 us
push&pull 1024x512, units	1370 us	1000 us	297 us
push&pull 2048x1024, units	3940 us	3570 us	1870 us

Tool functions

benchmark	before (xarray)	after
get_magnitude	6 us	0.4 us
check	13 us	6 us
to_xarray	18 us	22 us 12 us with !237 (merged)
to_units (no-op)	8 us	2.3 us
to_units 512x256	297 us	62 us
to_units 2048x1024	2180 us	1928 us (?)
get_units	3.5 us	1.1 us
is_quantified	3.6 us	0.2 us
strip_time	135 us	5.7 us

to_xarray can be further optimized by removing prod from the data dim calculation (see !237 (merged))

src/finam/data/cf_units.py

+"""Module to provide unit support via pint approximating UDUNITS/CF.
+
+Source: cf-xarray (https://github.com/xarray-contrib/cf-xarray/blob/main/cf_xarray/units.py)
+License: Apache License 2.0 (https://github.com/xarray-contrib/cf-xarray/blob/main/LICENSE)
+"""
+import functools
+import re
+import warnings
+
+import pint
+
+# pylint: disable-next=unused-import
+from pint import DimensionalityError, UndefinedUnitError, UnitStrippedWarning
+
+# from `xclim`'s unit support module with permission of the maintainers
+try:
+
+    @pint.register_unit_format("cf")  # pylint: disable-next=unused-argument
+    def short_formatter(unit, registry, **options):
+        """Return a CF-compliant unit string from a `pint` unit.
+        Parameters
+        ----------
+        unit : pint.UnitContainer
+            Input unit.
+        registry : pint.UnitRegistry
+            the associated registry
+        **options
+            Additional options (may be ignored)
+        Returns
+        -------
+        out : str
+            Units following CF-Convention, using symbols.
+        """
+
+        # convert UnitContainer back to Unit
+        unit = registry.Unit(unit)
+        # Print units using abbreviations (millimeter -> mm)
+        s = f"{unit:~D}"
+
+        # Search and replace patterns
+        pat = r"(?P<inverse>(?:1 )?/ )?(?P<unit>\w+)(?: \*\* (?P<pow>\d))?"
+
+        def repl(m):
+            i, u, p = m.groups()
+            p = p or (1 if i else "")
+            neg = "-" if i else ""
+
+            return f"{u}{neg}{p}"
+
+        out, _n = re.subn(pat, repl, s)
+
+        # Remove multiplications
+        out = out.replace(" * ", " ")
+        # Delta degrees:
+        out = out.replace("Δ°", "delta_deg")
+        return out.replace("percent", "%")
+
+except ImportError:
+    pass
+
+# ------
+# Reused with modification from MetPy under the terms of the BSD 3-Clause License.
+# Copyright (c) 2015,2017,2019 MetPy Developers.
+# Create registry, with preprocessors for UDUNITS-style powers (m2 s-2) and percent signs
+units = pint.UnitRegistry(
+    autoconvert_offset_to_baseunit=True,
+    preprocessors=[
+        functools.partial(
+            re.compile(
+                r"(?<=[A-Za-z])(?![A-Za-z])(?<![0-9\-][eE])(?<![0-9\-])(?=[0-9\-])"
+            ).sub,
+            "**",
+        ),
+        lambda string: string.replace("%", "percent"),
+    ],
+    force_ndarray_like=True,
+)
+
+units.define("percent = 0.01 = %")
+
+# Define commonly encountered units (both CF and non-CF) not defined by pint
+units.define("@alias meter = gpm")
+# ----- end block copied from metpy
+
+# -----
+# The following redefinitions were copied from xclim under the terms of their Apache-2 license
+# In pint, the default symbol for year is "a" which is not CF-compliant (stands for "are")
+units.define("year = 365.25 * day = yr")
+
+# Define commonly encountered units not defined by pint
+units.define("@alias degC = C = deg_C = Celsius = degrees_Celsius")
+units.define("@alias degK = deg_K")
+units.define("@alias day = d")
+units.define("@alias hour = h")  # Not the Planck constant...
+units.define(
+    "degrees_north = degree = degrees_north = degrees_N = degreesN = degree_north = degree_N = degreeN"
+)
+units.define(
+    "degrees_east = degree = degrees_east = degrees_E = degreesE = degree_east = degree_E = degreeE"
+)
+units.define("[speed] = [length] / [time]")
+# ----- end block copied from xclim
+
+# Add other specific aliases (by cf_xarray developers)
+units.define("practical_salinity_unit = [] = psu = PSU")
+
+# Enable pint's built-in matplotlib support
+try:
+    units.setup_matplotlib()
+except ImportError:
+    warnings.warn(
+        "Import(s) unavailable to set up matplotlib support...skipping this portion "
+        "of the setup."
+    )
+# end of vendored code from MetPy
+
+# Set as application registry
+pint.set_application_registry(units)