processCase(3) and processCase(5) in mhm v5.10_fixed -- Linux: forrtl: severe (174): SIGSEGV, segmentation fault occurred -- Cygwin: Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
For screenshots and easy to follow the issue, please refer to the attached pdf file.
Error messages in CYGWIN:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0 0xffffffffffffffff in ???
Segmentation fault (core dumped)
Note: The following floating-point exceptions are signalling: IEEE_DIVIDE_BY_ZERO IEEE_OVERFLOW_FLAG IEEE_UNDERFLOW_FLAG IEEE_DENORMAL
Error Messages in Linux Cluster:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Attn: Dear @ottor and Dear @thober
------- Email Correspondence--------
Dear Robert,
Thank you very much for the quick and kind reply. I hope I provide sufficient details below about both systems: cygwin and linux that I am using. I am quite new in Linux environment.
p.s. Thanks Olda, Gitlab email is added in Cc. Hopefully the issue will be reported there soon.
Cygwin x64 details: setup version 2.897 (64 bit)
OS: Windows 10
Compiler: GNU 7.3.0
Installed using:
- make system=cygwin compiler=gnu (also tested openmp=true and release=debug versions)
- also tested cmake installation
NETCDF-FORTRAN DETAILS:
Linux cluster details:
[mdemirel@sariyer ~ ]$ uname --all Linux sariyer 3.10.0-327.36.1.el7.x86_64 #1 (closed) SMP Sun Sep 18 13:04:29 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[mdemirel@sariyer ~ ]$ cat /proc/version Linux version 3.10.0-327.36.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) ) #1 (closed) SMP Sun Sep 18 13:04:29 UTC 2016
[mdemirel@sariyer ~ ]$ cat /etc/centos-release CentOS Linux release 7.2.1511 (Core)
[mdemirel@sariyer ~ ]$ cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core)
mHM is compiled using these modules
Currently Loaded Module files:
- uhem-bin 4) NetCDF/netcdf-c-4.4.1.1-shared-intel-2018.3
- intel/parallel_studio_xe_2018.3.051 5) NetCDF/netcdf-fortran-4.4.4-shared-netcdf-c-4.4.1.1-shared-intel-2018.3
- HDF5/hdf5-1.8.9-shared-intel-2018.3 6) cmake/3.9.1-intel-2018.3
In addition to ifort I also tested gfortran using gcc version 7.5.0 (GCC)
[mdemirel@sariyer slaveBEHV1 ]$ echo $FC gfortran [mdemirel@sariyer slaveBEHV1 ]$ gfortran -v Using built-in specs. COLLECT_GCC=gfortran COLLECT_LTO_WRAPPER=/okyanus/progs/gcc/GCC-7.5.0/libexec/gcc/x86_64-pc-linux-gnu/7.5.0/lto-wrapper Target: x86_64-pc-linux-gnu Configured with: ../gcc-7.5.0/configure --prefix=/okyanus/progs/gcc/GCC-7.5.0 --enable-languages=c,c++,fortran,go Thread model: posix gcc version 7.5.0 (GCC)
Kind Regards, Cüneyd
From: Robert Schweppe robert.schweppe@ufz.de Sent: Monday, January 20, 2020 11:30 AM To: Mehmet Cüneyd Demirel demirelmc@itu.edu.tr Cc: 'Oldrich Rakovec' oldrich.rakovec@ufz.de; rohini.kumar@ufz.de; luis.samaniego@ufz.de; 'Simon Stisen' sst@geus.dk; 'Julian Koch' juko@geus.dk Subject: Re: processCase(3) and processCase(5) in mhm v5.10 _fixed
Dear Cüneyd, thank you for your report. We will have a look at it and come back to you once we need further information or have a fix. Could you please tell us the versions of the Fortran compilers, netcdf-fortran libraries and the operating systems that you have tested? Thank you, Robert
Am 18.01.20 um 21:09 schrieb Mehmet Cüneyd Demirel: Dear Robert,
I am Cüneyd Demirel. I have contributed to mhm v5.7 and 5.8 on processCase(3) and processCase(5) with my colleagues from GEUS and Mathias Zink.
I see that you contributed refactoring and reformatting of these processes.
Did you test mhm v5.10_fixed in large and multiple basins with these options?
processCase(3) = 2 or 3 processCase(5) = -1 processCase(8) = 1
We had no issues in running 6 EU basins (listed below) together with v5.8; however, we are getting segmentation fault in both Linux and Cygwin versions.
I have trying/testing many scenarios (stack size etc) but I couldn’t diagnose the issue (screen shots are below). The model runs at coarse resolutions e.g. when I reduce L11 and L1 from 2km to 8km (Elbe model from UFZ).
I would be very happy if you can run Elbe basin L1 and L11 both at 2km (L0=500m) in UFZ side. I can share the setup and nml files if you need.
processCase(3) = 2 or 3 processCase(5) = -1 processCase(8) = 1
p.s. Even this combination causes segmentation fault. processCase(3) = 1 Feddes processCase(5) = -1 LAI correction processCase(8) = 1 Muskingum
Kind Regards, Cüneyd
These flags didn’t help either.
EXTRA_LDFLAGS += -Wl,--stack,16777216 EXTRA_CFLAGS += -Wl,--stack,16777216
I also tested the code (compiled with ifort) in a super computer/cluster in our university and mhm also failed with this error “forrtl: severe (174): SIGSEGV, segmentation fault occurred”
L0 scale Elbe Main Meuse Moselle Neckar Vienne TOTAL ncols 960 336 432 384 336 432 2880 nrows 864 384 624 624 384 384 3264
!> soil moisture
!> 2 - Jarvis equation for ET reduction, multi-layer infiltration capacity approach, Brooks-Corey like
!> 3 - Jarvis equation for ET reduction and FC dependency on root fraction coefficient
processCase(3) = 3
!> potential evapotranspiration (PET)
!> -1 - PET is input, LAI driven correction
processCase(5) = -1
!> routing
!> 0 - deactivated
!> 1 - Muskingum approach
!> 2 - adaptive timestep
processCase(8) = 1
History
! Rohini Kumar Mar 2016 - changes for handling multiple soil database options
! Zink M. & Demirel M.C. Mar 2017 - Added Jarvis soil water stress function at SM process(3)
! Demirel M.C. & S. Stisen Apr 2017 - Added FC dependency on root fraction coefficient at SM process(3)
! Robert Schweppe Dec 2017 - added loop over LCscenes inside MPR, renamed variables rewrite
! Robert Schweppe Jun 2018 - refactoring and reformatting