mHM merge requestshttps://git.ufz.de/mhm/mhm/-/merge_requests2020-10-05T16:32:59+02:00https://git.ufz.de/mhm/mhm/-/merge_requests/34Restart ids2020-10-05T16:32:59+02:00Maren KaluzaRestart idsChanged the way, restart files are read. Now in the namelist we define the pathes to the files instead the pathes to a directory.
Example:
**before**:
dir\_restartout(1:6) = 'output\_b1/b1\_', 'output\_b1/b2\_',...
**after**:...Changed the way, restart files are read. Now in the namelist we define the pathes to the files instead the pathes to a directory.
Example:
**before**:
dir\_restartout(1:6) = 'output\_b1/b1\_', 'output\_b1/b2\_',...
**after**:
mhm\_file\_restartout(1:6) = 'output\_b1/b1\_mHM\_restart\_001.nc', 'output\_b1/b2\_mHM\_restart\_002.nc',...
mrm\_file\_restartout(1:6) = 'output\_b1/b1\_mRM\_restart\_001.nc', 'output\_b1/b2\_mRM\_restart\_002.nc',...
**Why**:
- in case you write restart files for 10 domains, the restartfiles are appended with
numbers 1 to 10 (domain index). If you want to restart only with domain 2 and
6, you need to rename the files.
- more flexibility. You can easily restart from files from another run
- The appended number is limiting the number of domains (i.e. < 1000 domains in case of 3 digits)5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/32Add support for the NAG Fortran compiler v6.22020-04-24T10:33:53+02:00Sebastian MüllerAdd support for the NAG Fortran compiler v6.2This merge request adds support for the NAG Fortran compiler v6.2 on EVE.
The following this were added/modfied:
- `moduleLoadScripts/eve.nag62` - loading all needed modules (use `source eve.nag62`)
- `CMakeLists.txt` - added `-fpp` fla...This merge request adds support for the NAG Fortran compiler v6.2 on EVE.
The following this were added/modfied:
- `moduleLoadScripts/eve.nag62` - loading all needed modules (use `source eve.nag62`)
- `CMakeLists.txt` - added `-fpp` flag for NAGs pre-processor and added compiler flags for NAG (RELEASE/DEBUG)
- `.gitlab-ci.yml` - added compilation with NAG; valgrind check for NAG; check-cases with NAG
- `check/run_mhm_checks.py` - added new option to skip certain cases (motivated by NAG, since RNG produces overflows and fails in DEBUG)5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/31mHM netCDF output coordinate system:2020-12-01T09:16:33+01:00Pallav Kumar Shresthapallav-kumar.shrestha@ufz.demHM netCDF output coordinate system:- Due to lat and lon being stored as 2D array irrespective of iFlag_cordinate_sys, CDO and other operators read the mHM output as curvilinear/ projected even if it was otherwise.
- The lat lon are now stored as 1D vectors in case of lat ...- Due to lat and lon being stored as 2D array irrespective of iFlag_cordinate_sys, CDO and other operators read the mHM output as curvilinear/ projected even if it was otherwise.
- The lat lon are now stored as 1D vectors in case of lat lon system. This saves external postprocessing needed previously.
Fixes #985.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/30BUGFIX: domainID not set correctly for mRM if restart is activated2020-10-05T16:41:03+02:00Stephan ThoberBUGFIX: domainID not set correctly for mRM if restart is activated5.11https://git.ufz.de/mhm/mhm/-/merge_requests/28CI update and automated checking2024-02-26T09:59:05+01:00Sebastian MüllerCI update and automated checkingWith this `Merge Request` a couple of changes in the CI/Checking workflow are proposed:
- the CI-run divides into 4 stages now:
1. Info about ENV-VARS
2. Building of multiple executables (GCC73, GCC83, Intel18, with/without OpenMP s...With this `Merge Request` a couple of changes in the CI/Checking workflow are proposed:
- the CI-run divides into 4 stages now:
1. Info about ENV-VARS
2. Building of multiple executables (GCC73, GCC83, Intel18, with/without OpenMP support, with/without MPI, always release + debug) -> 18 artifacts
3. Valgrind test: The GCC73, GCC83 and Intel18 serial-debug artifacts run the example domain to check for memory consumption/leaks
4. Check cases: All artifacts from stage 2 are used to run all mHM check cases
- a new checking suite `run_mhm_checks.py` is provided to run the mHM check cases with a given mHM executable (see stage 4)
- a description can be found in `checks/README.md`
- the old `check_mhm_cases` is not working anymore since it depends on the old module system (it may works, if you load `chspython` and `cdo` by hand)
- the `moduleLoadScripts` folder has been updated to work with the new easybuild toolchains on EVE
# Problems
- The MPI checks revealed, that `case_05` and `case_07` fail, when mHM is ran by MPI. In case of Intel + debug + MPI, also the case_04 fails.
@kaluza : Is that a problem?
- one CI run now lasts for 25 min.
@thober Is that a problem?5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/26Update INSTALL.md2020-10-05T16:41:41+02:00Friedrich BoeingUpdate INSTALL.mddescription how to compile mHM with easybuilddescription how to compile mHM with easybuild5.11https://git.ufz.de/mhm/mhm/-/merge_requests/25Issue #492020-10-05T16:41:59+02:00Stephan ThoberIssue #495.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/24added notes on writting nml files in documentation2020-10-08T20:29:22+02:00Stephan Thoberadded notes on writting nml files in documentation5.11https://git.ufz.de/mhm/mhm/-/merge_requests/21BUG FIX: allowing higher routing resolution than hydrology2020-10-05T16:40:07+02:00Stephan ThoberBUG FIX: allowing higher routing resolution than hydrologyBug description:
Using a higher routing resolution than hydrology resolution may
cause segmentation faults because mapping from L1id on L11 is not
working correctly
Bug solution:
using directly Ids of corresponding low resolution grid ...Bug description:
Using a higher routing resolution than hydrology resolution may
cause segmentation faults because mapping from L1id on L11 is not
working correctly
Bug solution:
using directly Ids of corresponding low resolution grid cells in subroutine L11_L1_mapping5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/17Fortran-Lib: remove NR references2020-10-05T16:35:23+02:00Sebastian MüllerFortran-Lib: remove NR referencesWith this merge request all Numerical recipes references are sweeped out as discussed in #26 :
* [x] remove references in ``mo_moments`` for common sense routines
* [x] remove NR-types from ``mo_kind``
* [x] add minimal ``mo_corr`` cont...With this merge request all Numerical recipes references are sweeped out as discussed in #26 :
* [x] remove references in ``mo_moments`` for common sense routines
* [x] remove NR-types from ``mo_kind``
* [x] add minimal ``mo_corr`` containing only ``autocorr`` for now
* [x] add minimal ``mo_boxcox`` containing only ``boxcox`` for now5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/16Datetime2022-11-22T15:31:30+01:00Maren KaluzaDatetime# new datatype `datetimeinfo`
Hosts year, month, day, hour.
Also has prev_year and so on private. These are needed to get is_new_year,...
Everything is updated on increment. With this lots of code is not distributed and doubled over m...# new datatype `datetimeinfo`
Hosts year, month, day, hour.
Also has prev_year and so on private. These are needed to get is_new_year,...
Everything is updated on increment. With this lots of code is not distributed and doubled over mrm_eval, mhm_eval and mrm_write_fluxes.
datetimeinfo also contains iLAI and yID since these integers are fully derived by datetimeinfo and used often.5.11Sebastian MüllerSebastian Müllerhttps://git.ufz.de/mhm/mhm/-/merge_requests/15Restart: read L1_jarvis_thresh_c1 to restart file for process id 2 AND 32020-10-05T16:39:28+02:00Sebastian MüllerRestart: read L1_jarvis_thresh_c1 to restart file for process id 2 AND 3This merge request addresses issue #29 .This merge request addresses issue #29 .5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/14CI: add intel compiler to test case runs2020-10-05T16:36:05+02:00Sebastian MüllerCI: add intel compiler to test case runsIn this merge request, the intel compiler is added to the test case runs.In this merge request, the intel compiler is added to the test case runs.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/13CI: add valgrind checks for test domain2020-10-05T16:35:56+02:00Sebastian MüllerCI: add valgrind checks for test domainIn this merge request an addition to CI is proposed:
- another CI script is added to compile mhm in debug mode
- a mhm run for the test domain is checked with the valgrind tool ``memcheck`` to check for memory errors
- a mhm run for the ...In this merge request an addition to CI is proposed:
- another CI script is added to compile mhm in debug mode
- a mhm run for the test domain is checked with the valgrind tool ``memcheck`` to check for memory errors
- a mhm run for the test domain is checked with the valgrind tool ``massif`` to analyze memory consumption
- ``massif`` output is stored as artifact ``massif.out.0`` from job ``valgrind-mem-use``
- gfortran compiled executable is stored as artifact ``mhm`` from job ``compile-gfortran83``5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/12Average opti over cells2020-10-05T16:35:37+02:00Maren KaluzaAverage opti over cellsSwitched to cell wise kge of et and tws in opti_function 33, avereaged over all summands in the end. Also now twsa is created from simulated tws, since we compare the data with twsa data. For the same reason to reduce confusion the vari...Switched to cell wise kge of et and tws in opti_function 33, avereaged over all summands in the end. Also now twsa is created from simulated tws, since we compare the data with twsa data. For the same reason to reduce confusion the variable L1_twsObs was renamed L1_twsaObs, since the input data consists of the anomaly.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/11First implementation of continuous integration2020-10-05T16:36:36+02:00Sebastian MüllerFirst implementation of continuous integrationIn this merge request a first implementation of continuous integration is proposed.
This includes the following:
* use Gitlab runner on EVE (there can be multiple runners registered)
* at the moment only the GNU compiler is used
* cmake...In this merge request a first implementation of continuous integration is proposed.
This includes the following:
* use Gitlab runner on EVE (there can be multiple runners registered)
* at the moment only the GNU compiler is used
* cmake compilation is checked with simple run of test domain
* check scripts are run for:
* debug/release
* sequential/parallel
Some CI related scripts were added in a new folder called ``CI-scripts`` and the configuration file for the gitlab-runner called ``.gitlab-ci.yml`` was added to the base folder.
You can add a gitlab-runner to your own fork following this instruction:
* https://wiki.ufz.de/eve/index.php/Gitlab_Runner
But you have to enable ``custom build dir`` by adding ``enabled = true`` to ``[runners.custom_build_dir]`` in the gitlab-runner config file (usually under ``.gitlab-runner/config.toml`` in your home dir on EVE).
Example:
```
[[runners]]
name = "frontend1 muellese"
url = "https://git.ufz.de"
token = ...
executor = "shell"
[runners.custom_build_dir]
enabled = true
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
```5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/10Optidata management2020-10-05T16:36:48+02:00Maren KaluzaOptidata managementThis feature branch introduces a new data type for simulated gridded optidata. The writeoutCounter moves from the observed datatype to the simulated datatype. In general it is very similar to the datatype for the observed gridded optidat...This feature branch introduces a new data type for simulated gridded optidata. The writeoutCounter moves from the observed datatype to the simulated datatype. In general it is very similar to the datatype for the observed gridded optidata.
In addition it contains some procedures that improve readability and reduce sources of errors. A bugs was removed where the average_counter was incremented dependent on how many optidata where simulated. This is now part of the new type.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/9tws netCDF format2020-10-05T16:37:08+02:00Maren Kaluzatws netCDF formatIn this feature branch the tws input was changed from ascii files to netCDF format. It is still averaged over the domain. This issue will be addressed in the next feature branch together with et.
Before, there had to be a subroutine whe...In this feature branch the tws input was changed from ascii files to netCDF format. It is still averaged over the domain. This issue will be addressed in the next feature branch together with et.
Before, there had to be a subroutine where the tws data was appended with dummy data in case there were domains without tws. Essentially, the issue was there with the et data, as well, but didn't cause any problem if et data was read first, meaning, if the first domains to be read were all the et domains. The reason is the following:
- only if a domain has et, the subroutine to read et is called. It appends the global variable L1_et
- however later when accessing the data, s1 and e1 from level1 for the given domain is used. It is assumed that L1_et has the same shape as any other L1 data.
With this branch there is a new datatype used for tws, and if we can agree on that I will implement it for the other opti data, as well. It is called optidata and it stores
- the observed L1 data
- the mask for that L1 data
- the directory where the observed data is stored
- the timestep for which we want to create the simulated data
- the writeout counter for the simulated data, meaning the current timestep with respect to the simulated data (before, there was a variable writeout_counter that was used for all simulated data and if there was more than one there was a collision)
L1_tws is then an array of this type with nDomain dimensions. It is domain specific and there is no append strategy anymore. Another advantage is, that there are no unnecessary fill values anymore, if the shape of the arrays time X s1:e1 do not fit together. As far as I looked through the code it was never accessed as one whole array anyway.
We could discuss if the letter two attributes (timestepInput, writeOutCounter) should be part of that datatype. This is a minor change that I could reverse if wanted.
The disadvantage: in mhm_eval the whole datatype L1_tws is loaded where also the observed data is stored, which is not used in mhm_eval. If Fortran compiles reasonable there should not be an issue with copied data, though.
The advantages: data, that belongs together is packed together. This could reduce errors in further implementations. Also with this type we could easily extend the code in a way that we have a different timeStepInput for each domain even if it is all tws data.
Another possibility would be to have an own datatype for meta data for the simulated data with these two variables. It could be easily changed and is not the main part of this merge request anyway.5.11Stephan ThoberStephan Thoberhttps://git.ufz.de/mhm/mhm/-/merge_requests/8Add new moduleLoadScripts for Eve gfortran 8.32020-10-05T16:37:22+02:00Robert SchweppeAdd new moduleLoadScripts for Eve gfortran 8.3it includes a serial and OpenMPI-parallel versionit includes a serial and OpenMPI-parallel version5.11https://git.ufz.de/mhm/mhm/-/merge_requests/7Separate obj func2020-10-05T16:37:40+02:00Maren KaluzaSeparate obj funcIn this branch a new objective function has been developed. It can get outflow, ET and TWS data in the objective function from separated mHM calls. Therefore an index array had to be provided and is an optional parameter passed to every ...In this branch a new objective function has been developed. It can get outflow, ET and TWS data in the objective function from separated mHM calls. Therefore an index array had to be provided and is an optional parameter passed to every eval subroutine now. With this index array only the domains with the corresponding indices are simulated in that run.
The new objective function creates a disjunct partition of all indices, then calls mHM several times for each seperate subset of indices. Then it calculates an objective from all the simulated data. It is not a useful objective function in terms of science yet, but it is a useful sceleton.5.11Stephan ThoberStephan Thober