No xarray!

A trial to switch back to numpy arrays for performance. We do not even really use any xarray features currently.

For the cascading reservior example, which also does some calculations, this results in approx. 3x speedup.

Todo

• Get rid of xarray
• Rename to_xarray() (renamed to prepare())
• Rename xdata args
• Update the book
• Update the homepage

Benchmarks

When comparing columns, keep in mind that pushing xarray data basically requires creating the underlying numpy data first. So the most frequent use case is comparing first vs. last column. However, even compared to pushing xarray data we get quite some speedup for larger data.

Full runs

benchmark	before (numpy)	before (xarray)	after
run 365, 64x32	178 ms	29 ms	32 ms
run 365, 1024x512	289 ms	134 ms	127 ms
run 365, 2048x1024	862 ms	740 ms	481 ms
run 365, 64x32, copy	220 ms	57 ms	62 ms
run 365, 1024x512, copy	600 ms	426 ms	270 ms
run 365, 2048x1024, copy	2395 ms	2234 ms	915 ms
run 365, 64x32, units	-	109 ms	49 ms
run 365, 1024x512, units	-	483 ms	254 ms
run 365, 2048x1024, units	-	2044 ms	1138 ms

Push & pull

benchmark	before (numpy)	before (xarray)	after
push&pull 512x256	460 us	74 us	80 us
push&pull 1024x512	490 us	74 us	80 us
push&pull 2048x1024	475 us	74 us	80 us
push&pull 512x256, units	880 us	501 us	157 us
push&pull 1024x512, units	1370 us	1000 us	297 us
push&pull 2048x1024, units	3940 us	3570 us	1870 us

Tool functions

benchmark	before (xarray)	after
get_magnitude	6 us	0.4 us
check	13 us	6 us
to_xarray	18 us	22 us 12 us with !237 (merged)
to_units (no-op)	8 us	2.3 us
to_units 512x256	297 us	62 us
to_units 2048x1024	2180 us	1928 us (?)
get_units	3.5 us	1.1 us
is_quantified	3.6 us	0.2 us
strip_time	135 us	5.7 us

to_xarray can be further optimized by removing prod from the data dim calculation (see !237 (merged))

docs/source/finam-book/development/adapters.rst

@@ -72,7 +72,7 @@ File ``src/scale.py``:
 
     comp.run(end_time=datetime(2000, 1, 2))
 
-    print(fm.data.strip_data(consumer.data["Input"]))
+    print(consumer.data["Input"][0, ...])
 
 .. testoutput:: scale-adapter
     :hide:
@@ -171,10 +171,7 @@ In :meth:`.Adapter._source_updated`, we need to store incoming data:
             data = self.pull_data(time, self)
 
             self.old_data = self.new_data
-            self.new_data = (
-                time,
-                fm.data.strip_data(data)
-            )
+            self.new_data = (time, data)
 
         def _get_data(self, time, target):
             pass
@@ -203,15 +200,12 @@ In :meth:`.Adapter._get_data`, we can now do the interpolation whenever data is
             data = self.pull_data(time, self)
 
             self.old_data = self.new_data
-            self.new_data = (
-                time,
-                fm.data.strip_data(data)
-            )
+            self.new_data = (time, data)
 
         def _get_data(self, time, _target):
             if self.old_data is None:
                 if self.new_data is None:
-                    return None
+                    raise fm.FinamNoDataError("No data available.")
                 else:
                     return self.new_data[1]
 
@@ -246,7 +240,7 @@ In :meth:`.Adapter._get_data`, we can now do the interpolation whenever data is
 
     comp.run(end_time=datetime(2000, 1, 15))
 
-    print(fm.data.strip_data(consumer.data["Input"]))
+    print(consumer.data["Input"][0, ...])
 
 .. testoutput:: time-adapter
     :hide: