# Parallel hdf5 and netcdf

**URL:** <https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419>\
**Category:** Building Abinit\
**Created:** [October 3, 2023, 8:03pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419 "2023-10-03T20:03:18Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![samabdel](https://discourse.abinit.org/letter_avatar_proxy/v4/letter/s/ba8739/32.png) [@samabdel](https://discourse.abinit.org/u/samabdel)\
**Post date:** [October 3, 2023, 8:03pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/1 "2023-10-03T20:03:18Z")

</div>

## hello every one, here is what I get when compiling abinit 9.10.1: Core build parameters

- C compiler : gnu version 12.3

- Fortran compiler : gnu version 12.3

- architecture : intel core\_i3 (64 bits)

- debugging : basic

- optimizations : standard

- OpenMP enabled : no (collapse: ignored)

- MPI enabled : yes (flavor: auto)

- MPI in-place : no

- MPI-IO enabled : yes

- GPU enabled : no (flavor: none)

- LibXML2 enabled : no

- LibPSML enabled : no

- XMLF90 enabled : no

- HDF5 enabled : yes (MPI support: no)

- NetCDF enabled : yes (MPI support: no)

- NetCDF-F enabled : yes (MPI support: no)

- FFT flavor : fftw3 (libs: user-defined)

- LINALG flavor : netlib (libs: user-defined)

- SCALAPACK enabled : no

- ELPA enabled : no

- MAGMA enabled : unknown (magma version \>= 1.5 ? )

- FCFLAGS : -g -ffree-line-length-none -fallow-argument-mismatch

- NVCC\_CFLAGS :

- CPATH :

- Build workflow : monolith  
with:

# MPI settings

with\_mpi=“yes”  
enable\_mpi\_io=“yes”

# linear algebra settings

with\_linalg\_flavor=“netlib”  
LINALG\_LIBS=“-L/usr/lib/x86\_64-linux-gnu -llapack -lblas”

# mandatory libraries

with\_libxc=“yes”

# LibXC

# Website: [Best Australia Online Casinos – Complete Guide to Playing at AU Online Casinos](http://www.tddft.org/programs/octopus/wiki/index.php/Libxc)

# Trigger and install prefix for LibXC (default is unset)

# 

with\_libxc=“/usr/lib/x86\_64-linux-gnu/”

# C preprocessor flags for LibXC (default is unset)

# 

LIBXC\_CPPFLAGS=“-I/usr/include”

# C flags for LibXC (default is unset)

# 

LIBXC\_CFLAGS=“-I/usr/include”

# Fortran flags for LibXC (default is unset)

# 

LIBXC\_FCFLAGS=“-I/usr/include”

# Link flags for the LibXC library (default is unset)

# 

LIBXC\_FCFLAGS=“-I/usr/include”

# Link flags for the LibXC library (default is unset)

# 

#LIBXC\_LDFLAGS=“”

# Library flags for LibXC (default is unset)

# 

LIBXC\_LIBS=“-L/usr/lib/x86\_64-linux-gnu -lxcf90 -lxc”  
with\_libxc=“yes”  
with\_hdf5=“yes”  
with\_netcdf=“yes”  
with\_netcdf\_fortran=“yes”

# fast fourier settings

with\_fft\_flavor=“fftw3”  
FFTW3\_CPPFLAGS=“-I/usr/include”  
FFTW3\_FCFLAGS=“-I/usr/include”  
FFTW3\_LIBS=“-L/usr/lib/x86\_64-linux-gnu -lfftw3 -lfftw3f”

# XML2 library (used by multibinit)

## #with\_libxml2=“yes” #LIBXML2\_FCFLAGS=“-I/usr/lib/x86\_64-linux-gnu/” #LIBXML2\_LIBS=“-L/usr/include/libxml2/libxml/ -lxmlf90”

as you can see both hdf5 netcdf and netcdf-f are not recognized by abinit as parallel libraries…should I recompile them all from source or there is another workaround in abinit setting?

---

<div class="post-metadata">

**Author:** ![gingras.ol](https://discourse.abinit.org/user_avatar/discourse.abinit.org/gingras.ol/32/275_2.png) [@gingras.ol](https://discourse.abinit.org/u/gingras.ol)\
**Post date:** [October 3, 2023, 8:22pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/2 "2023-10-03T20:22:02Z")

</div>

Hello,

What C and fortran compiler are you using? They should also be MPI. You should try setting

> CC=mpicc  
> FC=“mpif90”  
> F77=“mpif77”

Otherwise, please provide the `config.log` file generated during the `configure` execution.

Best,  
Olivier

---

<div class="post-metadata">

**Author:** ![samabdel](https://discourse.abinit.org/letter_avatar_proxy/v4/letter/s/ba8739/32.png) [@samabdel](https://discourse.abinit.org/u/samabdel)\
**Post date:** [October 4, 2023, 4:19pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/3 "2023-10-04T16:19:44Z")

</div>

Dear Olivier,  
I set the compilers as you recommended but I still have the same problem…  
Please find attached the config.log file.  
I had a look at it, and for hdf5 for example, abinit is linking to the serial one !!!  
/usr/lib/x86\_64-linux-gnu/hdf5/serial  
so why it is not linking to the parallel one??? or this is because it does not exist?  
many many thanks,  
Samir

---

<div class="post-metadata">

**Author:** ![samabdel](https://discourse.abinit.org/letter_avatar_proxy/v4/letter/s/ba8739/32.png) [@samabdel](https://discourse.abinit.org/u/samabdel)\
**Post date:** [October 4, 2023, 4:21pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/4 "2023-10-04T16:21:24Z")

</div>

[config.log](https://discourse.abinit.org/uploads/short-url/g3FZ2ViudqkmP9fHqiG73dsbiPG.log) (191.3 KB)

---

<div class="post-metadata">

**Author:** ![fgoudreault](https://discourse.abinit.org/user_avatar/discourse.abinit.org/fgoudreault/32/104_2.png) [@fgoudreault](https://discourse.abinit.org/u/fgoudreault)\
**Post date:** [October 4, 2023, 5:05pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/5 "2023-10-04T17:05:30Z")

</div>

Hi @samabdel! How did you install the hdf5 and netcdf libraries? Do you use precompiled binairies?

Cheers

Félix

---

<div class="post-metadata">

**Author:** ![samabdel](https://discourse.abinit.org/letter_avatar_proxy/v4/letter/s/ba8739/32.png) [@samabdel](https://discourse.abinit.org/u/samabdel)\
**Post date:** [October 4, 2023, 5:22pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/6 "2023-10-04T17:22:31Z")

</div>

Hi,  
yes I use those already installed in Ubuntu.  
Thanks

---

<div class="post-metadata">

**Author:** ![gingras.ol](https://discourse.abinit.org/user_avatar/discourse.abinit.org/gingras.ol/32/275_2.png) [@gingras.ol](https://discourse.abinit.org/u/gingras.ol)\
**Post date:** [October 4, 2023, 5:41pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/7 "2023-10-04T17:41:11Z")

</div>

Hi,

Looking at your `config.log` file, the HDF5 configuration is performed at line 2218. You are not directing the installation to the specific folder you want. Probably it’s finding the serial version before the parallel? Same as for NETCDF on line 2264. Try to set

> with\_hdf5=“$PATH\_TO\_HDF5\_PARAL”  
> with\_netcdf=“$PATH\_TO\_NETCDF\_PARAL”

Best,  
Olivier

---

<div class="post-metadata">

**Author:** ![samabdel](https://discourse.abinit.org/letter_avatar_proxy/v4/letter/s/ba8739/32.png) [@samabdel](https://discourse.abinit.org/u/samabdel)\
**Post date:** [October 4, 2023, 6:22pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/8 "2023-10-04T18:22:59Z")

</div>

Hi,  
After having set HDF5 library path to the correct one…abinit is complaining now about the hdf5 C compiler h5pcc???  
[config.log](https://discourse.abinit.org/uploads/short-url/552FusrFo3joJe1xBJ6WeKsoRuu.log) (186.8 KB)

---

<div class="post-metadata">

**Author:** ![gingras.ol](https://discourse.abinit.org/user_avatar/discourse.abinit.org/gingras.ol/32/275_2.png) [@gingras.ol](https://discourse.abinit.org/u/gingras.ol)\
**Post date:** [October 4, 2023, 6:41pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/9 "2023-10-04T18:41:34Z")

</div>

Indeed, you have

> configure:40206: WARNING: could not find the HDF5 C compiler  
> Please set the H5CC environment variable to a valid HDF5  
> C compiler (usually called ‘h5cc’ or ‘h5pcc’), so that  
> essential information about your HDF5 installation will be  
> exposed to the build system of ABINIT.

Try setting `HDF5CC="$PATH_TO_HDF5_PARAL/bin/h5cc"` manually?  
Olivier

---

<div class="post-metadata">

**Author:** ![samabdel](https://discourse.abinit.org/letter_avatar_proxy/v4/letter/s/ba8739/32.png) [@samabdel](https://discourse.abinit.org/u/samabdel)\
**Post date:** [October 5, 2023, 1:21pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/10 "2023-10-05T13:21:41Z")

</div>

It is H5CC which should be set/defined…not HDF5CC.  
I already set it but for some reason abinit is not taking it? don’t know why?

---

<div class="post-metadata">

**Author:** ![gingras.ol](https://discourse.abinit.org/user_avatar/discourse.abinit.org/gingras.ol/32/275_2.png) [@gingras.ol](https://discourse.abinit.org/u/gingras.ol)\
**Post date:** [October 6, 2023, 1:03am UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/11 "2023-10-06T01:03:34Z")

</div>

Sorry for the typo. Indeed it’s not finding it. Are you sure you are giving it correctly and that it is there and working?  
Olivier

---

<div class="post-metadata">

**Author:** ![fgoudreault](https://discourse.abinit.org/user_avatar/discourse.abinit.org/fgoudreault/32/104_2.png) [@fgoudreault](https://discourse.abinit.org/u/fgoudreault)\
**Post date:** [October 6, 2023, 7:30pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/12 "2023-10-06T19:30:37Z")

</div>

@Samabdel, It is possible that the hdf5 library downloaded using apt is not compiled in parallel. Did you install the `libhdf5-mpi-dev` and `libnetcdf-mpi-dev` using apt?

---

<div class="post-metadata">

**Author:** ![gingras.ol](https://discourse.abinit.org/user_avatar/discourse.abinit.org/gingras.ol/32/275_2.png) [@gingras.ol](https://discourse.abinit.org/u/gingras.ol)\
**Post date:** [October 11, 2023, 8:56pm UTC](https://discourse.abinit.org/t/parallel-hdf5-and-netcdf/3419/13 "2023-10-11T20:56:19Z")

</div>

@samabdel what’s the status of this installation?