mHM merge requestshttps://git.ufz.de/mhm/mhm/-/merge_requests2020-04-24T10:33:53+02:00https://git.ufz.de/mhm/mhm/-/merge_requests/32Add support for the NAG Fortran compiler v6.22020-04-24T10:33:53+02:00Sebastian MüllerAdd support for the NAG Fortran compiler v6.2This merge request adds support for the NAG Fortran compiler v6.2 on EVE.
The following this were added/modfied:
- `moduleLoadScripts/eve.nag62` - loading all needed modules (use `source eve.nag62`)
- `CMakeLists.txt` - added `-fpp` fla...This merge request adds support for the NAG Fortran compiler v6.2 on EVE.
The following this were added/modfied:
- `moduleLoadScripts/eve.nag62` - loading all needed modules (use `source eve.nag62`)
- `CMakeLists.txt` - added `-fpp` flag for NAGs pre-processor and added compiler flags for NAG (RELEASE/DEBUG)
- `.gitlab-ci.yml` - added compilation with NAG; valgrind check for NAG; check-cases with NAG
- `check/run_mhm_checks.py` - added new option to skip certain cases (motivated by NAG, since RNG produces overflows and fails in DEBUG)5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/42[FIX] messages about written mhm fluxes AND eve NAG workflow2020-08-31T16:23:16+02:00Sebastian Müller[FIX] messages about written mhm fluxes AND eve NAG workflowThere was a mismatch between the selected fluxes in `mhm_output.nml` and the displayed messages about the fluxes to be written.
See #81
In addition the module versions in the eve NAG loadscript were fixed in order to get a successfu...There was a mismatch between the selected fluxes in `mhm_output.nml` and the displayed messages about the fluxes to be written.
See #81
In addition the module versions in the eve NAG loadscript were fixed in order to get a successful CI run. See #142 5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/45Bugfix setup routing resolution2020-09-15T14:03:04+02:00Sebastian MüllerBugfix setup routing resolutionFixing wrongly matched IDs from L1 to L11 when routing resolution (L11) is finer than L1 resolution.Fixing wrongly matched IDs from L1 to L11 when routing resolution (L11) is finer than L1 resolution.5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/46The length in net_startup is only cut in case there are less then 2 lengths2020-09-15T16:02:08+02:00Maren KaluzaThe length in net_startup is only cut in case there are less then 2 lengthsWhen the network is setup for small river networks the mRM setup failed. The reason for that was that the cell distances were cut after the 60th percentile and set to a fix value. But for small setups the array has too few values. Now, w...When the network is setup for small river networks the mRM setup failed. The reason for that was that the cell distances were cut after the 60th percentile and set to a fix value. But for small setups the array has too few values. Now, when calling the percentile subroutine another if clause tests if there are enough values.5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/44Online Documentation with GitLab pages2020-09-17T15:34:14+02:00Sebastian MüllerOnline Documentation with GitLab pagesThis MR enables the online documentation of mHM based on the present Doxgen documentation:
- switch to select the documented branch (master/develop/v5.11/...) was added to the webpage to switch between versions of mHM.
- the documentat...This MR enables the online documentation of mHM based on the present Doxgen documentation:
- switch to select the documented branch (master/develop/v5.11/...) was added to the webpage to switch between versions of mHM.
- the documentation can later be accessed by: https://mhm.pages.ufz.de/mhm
- by default, the documentation will point to the `master` version (which is not present yet)
- test version can be accessed by: https://muellese.pages.ufz.de/mhm/doc_update (you can only switch to `develop` there, since `master` is not present)
- the `lib` folder was excluded in the documentation, since it will get its own one
- the documentation will be build automatically during CI runs in the `build` stage
- along with the online documentation, a pdf version will be build and provided as an artifact by the Gitlab-runner5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/47corrected unit attributes for lat lon variables2020-09-24T13:39:53+02:00Stephan Thobercorrected unit attributes for lat lon variables5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/37River temperature module2020-10-01T11:00:36+02:00Sebastian MüllerRiver temperature moduleThis MR introduces a first version of a **river temperature routing process** in an OOP fashioned way.
# Abstract
River temperature determines a wide variety of properties of water and is of great interest as model
output in the fie...This MR introduces a first version of a **river temperature routing process** in an OOP fashioned way.
# Abstract
River temperature determines a wide variety of properties of water and is of great interest as model
output in the field of hydrological modeling.
In this merge request, we propose a suitable implementation, to enable modeling river temperature in mHM [Samaniego et al., 2010, Kumar et al., 2013].
Based on a literature survey, we decided to use a physical based model build up on the work of [Wanders et al., 2019, Beek et al., 2012, Foreman et al., 2001], since its formulation best fits to the structure of mHM.
A reformulation of the governing equations was necessary, to reuse routines of the routing module mRM [Thober et al., 2019], so a numerical solution based on a Muskingum-Cunge discretization [Chow et al., 1988, Thober et al., 2019] could be formulated.
This implementation does not include ice layer at the moment and is not evaluated yet, so we mark it as **EXPERIMENTAL**.
## Governing equations
1. Surface water energy balance equation:
```math
\frac{\partial\left(h\cdot T_{w}\right)}{\partial t}+\frac{\partial\left(q\cdot T_{w}\right)}{\partial A} =\underbrace{\frac{Q^{*}-H-LE}{C_{v}}}_{=:T_{IO}}+\frac{\partial\left(q_{s}\cdot T_{s}\right)}{\partial A}
```
2. Net Radiation:
```math
Q^{*}=S_{I}\cdot\left(1-\alpha_{w}\right)+L_{I}-L_{O}
```
3. Boltzmann equation ([see](https://energyeducation.ca/encyclopedia/Stefan-Boltzmann_law)):
```math
L_{O}=\varepsilon\cdot\sigma\cdot\left(T_{w}\right)^{4}
```
4. Sensible heat flux formula:
```math
H=k_{H}\cdot\left(T_{w}-T_{a}\right)
```
5. Latent heat formula:
```math
LE=\lambda_{v}\cdot ET_{0}
```
6. Lateral flux sum:
```math
q_{s}=q_{dr}+q_{i}+q_{b}
```
7. Temperature runoff components:
- From [Wanders et al., 2019]: (in use)
```math
T_{s}=\frac{q_{dr}}{q_{s}}\cdot\max\left(T_{i},T_{a}-1.5\right)+\frac{q_{i}}{q_{s}}\cdot\max\left(T_{i},T_{a}\right)+\frac{q_{b}}{q_{s}}\cdot\max\left(T_{i}+5,\overline{T_{a}}\right)
```
- From [Beek et al., 2012]: (alternative)
```math
T_{s}=\frac{q_{dr}}{q_{s}}\cdot T_{a}+\frac{q_{i}}{q_{s}}\cdot T_{a}+\frac{q_{b}}{q_{s}}\cdot\overline{T_{a}}
```
## TODOs
- [x] add a test case
- [x] final clean up
- [x] documentation5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/50Online Coverage Report2020-10-03T12:16:58+02:00Sebastian MüllerOnline Coverage ReportThis MR introduces automatic coverage calculation, which is added as a separate page to the auto-generated online documentation.
![coverage](/uploads/5be173f8f28889d2cf072a8775ba7537/coverage.png)
This uses `gcov` from `gcc7.3`. Co...This MR introduces automatic coverage calculation, which is added as a separate page to the auto-generated online documentation.
![coverage](/uploads/5be173f8f28889d2cf072a8775ba7537/coverage.png)
This uses `gcov` from `gcc7.3`. Coverage was added as a target in the CMakeLists.txt and in order to execute coverage calculation you have to do the following:
```bash
mkdir build && cd build
cmake -DCMAKE_WITH_COVERAGE:STRING=ON ..
make mhm_coverage_CI
```
As an addition, the `mhm` executable can now be called with a passed directory path, where `mhm` should be executed in.
This feature is likely to change in the future.5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/48[EVE] add intel 2020 load scripts and CI workflow2020-10-05T13:48:26+02:00Sebastian Müller[EVE] add intel 2020 load scripts and CI workflowWith this MR, the new EasyBuild workflow for the Intel-compiler chain is added:
- Intel C Compiler v2020.1
- Intel Fortran Compiler v2020.1
- OpenMPI v4.0.3
- Intel MKL v2020.1
Both, normal and MPI workflow are added.With this MR, the new EasyBuild workflow for the Intel-compiler chain is added:
- Intel C Compiler v2020.1
- Intel Fortran Compiler v2020.1
- OpenMPI v4.0.3
- Intel MKL v2020.1
Both, normal and MPI workflow are added.5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/51Versioning2020-10-05T13:50:06+02:00Sebastian MüllerVersioningThis MR introduces two addition files:
- `version.txt` holding the current version of mHM (now: `5.11.0-dev0`)
- `version_date.txt` holding the latest release date (now: `Jun 2019` of version 5.10)
Versioning is following [semantic ...This MR introduces two addition files:
- `version.txt` holding the current version of mHM (now: `5.11.0-dev0`)
- `version_date.txt` holding the latest release date (now: `Jun 2019` of version 5.10)
Versioning is following [semantic versions](http://semver.org). When a `-devX` is appended to the version, it is treated as a development version, and the version date will be set to the current date. otherwise, the date from the given file will be used.
This has the advantage, that users see the development version they are using in the mhm output.
In addition, these files are used in the documentation to state the version of mHM, that the documentation belongs to.
There were some old versions written at different places in the code-base. This should be cleaned up now.
Last but not least, the UFZ logos were updated to the latest versions.5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/34Restart ids2020-10-05T16:32:59+02:00Maren KaluzaRestart idsChanged the way, restart files are read. Now in the namelist we define the pathes to the files instead the pathes to a directory.
Example:
**before**:
dir\_restartout(1:6) = 'output\_b1/b1\_', 'output\_b1/b2\_',...
**after**:...Changed the way, restart files are read. Now in the namelist we define the pathes to the files instead the pathes to a directory.
Example:
**before**:
dir\_restartout(1:6) = 'output\_b1/b1\_', 'output\_b1/b2\_',...
**after**:
mhm\_file\_restartout(1:6) = 'output\_b1/b1\_mHM\_restart\_001.nc', 'output\_b1/b2\_mHM\_restart\_002.nc',...
mrm\_file\_restartout(1:6) = 'output\_b1/b1\_mRM\_restart\_001.nc', 'output\_b1/b2\_mRM\_restart\_002.nc',...
**Why**:
- in case you write restart files for 10 domains, the restartfiles are appended with
numbers 1 to 10 (domain index). If you want to restart only with domain 2 and
6, you need to rename the files.
- more flexibility. You can easily restart from files from another run
- The appended number is limiting the number of domains (i.e. < 1000 domains in case of 3 digits)5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/17Fortran-Lib: remove NR references2020-10-05T16:35:23+02:00Sebastian MüllerFortran-Lib: remove NR referencesWith this merge request all Numerical recipes references are sweeped out as discussed in #26 :
* [x] remove references in ``mo_moments`` for common sense routines
* [x] remove NR-types from ``mo_kind``
* [x] add minimal ``mo_corr`` cont...With this merge request all Numerical recipes references are sweeped out as discussed in #26 :
* [x] remove references in ``mo_moments`` for common sense routines
* [x] remove NR-types from ``mo_kind``
* [x] add minimal ``mo_corr`` containing only ``autocorr`` for now
* [x] add minimal ``mo_boxcox`` containing only ``boxcox`` for now5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/12Average opti over cells2020-10-05T16:35:37+02:00Maren KaluzaAverage opti over cellsSwitched to cell wise kge of et and tws in opti_function 33, avereaged over all summands in the end. Also now twsa is created from simulated tws, since we compare the data with twsa data. For the same reason to reduce confusion the vari...Switched to cell wise kge of et and tws in opti_function 33, avereaged over all summands in the end. Also now twsa is created from simulated tws, since we compare the data with twsa data. For the same reason to reduce confusion the variable L1_twsObs was renamed L1_twsaObs, since the input data consists of the anomaly.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/13CI: add valgrind checks for test domain2020-10-05T16:35:56+02:00Sebastian MüllerCI: add valgrind checks for test domainIn this merge request an addition to CI is proposed:
- another CI script is added to compile mhm in debug mode
- a mhm run for the test domain is checked with the valgrind tool ``memcheck`` to check for memory errors
- a mhm run for the ...In this merge request an addition to CI is proposed:
- another CI script is added to compile mhm in debug mode
- a mhm run for the test domain is checked with the valgrind tool ``memcheck`` to check for memory errors
- a mhm run for the test domain is checked with the valgrind tool ``massif`` to analyze memory consumption
- ``massif`` output is stored as artifact ``massif.out.0`` from job ``valgrind-mem-use``
- gfortran compiled executable is stored as artifact ``mhm`` from job ``compile-gfortran83``5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/14CI: add intel compiler to test case runs2020-10-05T16:36:05+02:00Sebastian MüllerCI: add intel compiler to test case runsIn this merge request, the intel compiler is added to the test case runs.In this merge request, the intel compiler is added to the test case runs.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/11First implementation of continuous integration2020-10-05T16:36:36+02:00Sebastian MüllerFirst implementation of continuous integrationIn this merge request a first implementation of continuous integration is proposed.
This includes the following:
* use Gitlab runner on EVE (there can be multiple runners registered)
* at the moment only the GNU compiler is used
* cmake...In this merge request a first implementation of continuous integration is proposed.
This includes the following:
* use Gitlab runner on EVE (there can be multiple runners registered)
* at the moment only the GNU compiler is used
* cmake compilation is checked with simple run of test domain
* check scripts are run for:
* debug/release
* sequential/parallel
Some CI related scripts were added in a new folder called ``CI-scripts`` and the configuration file for the gitlab-runner called ``.gitlab-ci.yml`` was added to the base folder.
You can add a gitlab-runner to your own fork following this instruction:
* https://wiki.ufz.de/eve/index.php/Gitlab_Runner
But you have to enable ``custom build dir`` by adding ``enabled = true`` to ``[runners.custom_build_dir]`` in the gitlab-runner config file (usually under ``.gitlab-runner/config.toml`` in your home dir on EVE).
Example:
```
[[runners]]
name = "frontend1 muellese"
url = "https://git.ufz.de"
token = ...
executor = "shell"
[runners.custom_build_dir]
enabled = true
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
```5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/10Optidata management2020-10-05T16:36:48+02:00Maren KaluzaOptidata managementThis feature branch introduces a new data type for simulated gridded optidata. The writeoutCounter moves from the observed datatype to the simulated datatype. In general it is very similar to the datatype for the observed gridded optidat...This feature branch introduces a new data type for simulated gridded optidata. The writeoutCounter moves from the observed datatype to the simulated datatype. In general it is very similar to the datatype for the observed gridded optidata.
In addition it contains some procedures that improve readability and reduce sources of errors. A bugs was removed where the average_counter was incremented dependent on how many optidata where simulated. This is now part of the new type.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/9tws netCDF format2020-10-05T16:37:08+02:00Maren Kaluzatws netCDF formatIn this feature branch the tws input was changed from ascii files to netCDF format. It is still averaged over the domain. This issue will be addressed in the next feature branch together with et.
Before, there had to be a subroutine whe...In this feature branch the tws input was changed from ascii files to netCDF format. It is still averaged over the domain. This issue will be addressed in the next feature branch together with et.
Before, there had to be a subroutine where the tws data was appended with dummy data in case there were domains without tws. Essentially, the issue was there with the et data, as well, but didn't cause any problem if et data was read first, meaning, if the first domains to be read were all the et domains. The reason is the following:
- only if a domain has et, the subroutine to read et is called. It appends the global variable L1_et
- however later when accessing the data, s1 and e1 from level1 for the given domain is used. It is assumed that L1_et has the same shape as any other L1 data.
With this branch there is a new datatype used for tws, and if we can agree on that I will implement it for the other opti data, as well. It is called optidata and it stores
- the observed L1 data
- the mask for that L1 data
- the directory where the observed data is stored
- the timestep for which we want to create the simulated data
- the writeout counter for the simulated data, meaning the current timestep with respect to the simulated data (before, there was a variable writeout_counter that was used for all simulated data and if there was more than one there was a collision)
L1_tws is then an array of this type with nDomain dimensions. It is domain specific and there is no append strategy anymore. Another advantage is, that there are no unnecessary fill values anymore, if the shape of the arrays time X s1:e1 do not fit together. As far as I looked through the code it was never accessed as one whole array anyway.
We could discuss if the letter two attributes (timestepInput, writeOutCounter) should be part of that datatype. This is a minor change that I could reverse if wanted.
The disadvantage: in mhm_eval the whole datatype L1_tws is loaded where also the observed data is stored, which is not used in mhm_eval. If Fortran compiles reasonable there should not be an issue with copied data, though.
The advantages: data, that belongs together is packed together. This could reduce errors in further implementations. Also with this type we could easily extend the code in a way that we have a different timeStepInput for each domain even if it is all tws data.
Another possibility would be to have an own datatype for meta data for the simulated data with these two variables. It could be easily changed and is not the main part of this merge request anyway.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/8Add new moduleLoadScripts for Eve gfortran 8.32020-10-05T16:37:22+02:00Robert SchweppeAdd new moduleLoadScripts for Eve gfortran 8.3it includes a serial and OpenMPI-parallel versionit includes a serial and OpenMPI-parallel version5.11https://git.ufz.de/mhm/mhm/-/merge_requests/7Separate obj func2020-10-05T16:37:40+02:00Maren KaluzaSeparate obj funcIn this branch a new objective function has been developed. It can get outflow, ET and TWS data in the objective function from separated mHM calls. Therefore an index array had to be provided and is an optional parameter passed to every ...In this branch a new objective function has been developed. It can get outflow, ET and TWS data in the objective function from separated mHM calls. Therefore an index array had to be provided and is an optional parameter passed to every eval subroutine now. With this index array only the domains with the corresponding indices are simulated in that run.
The new objective function creates a disjunct partition of all indices, then calls mHM several times for each seperate subset of indices. Then it calculates an objective from all the simulated data. It is not a useful objective function in terms of science yet, but it is a useful sceleton.5.11Stephan ThoberStephan Thober