Hello everyone and I am happy to be on the abinit forum. I have built on google cloud console abinit-10.2.7 with intel oneapi (mpi and openmp) with elpa, libxc, wannier90, libpaw, hdf5, netcdf-fortran and c and it runs quite well. I next wish to build on Dolly Sods hpc at WVU and take advantage of its A100 nvidea gpu. I see the link describing a related build (https://github.com/abinit/abinit/blob/master/doc/build/GPU_NvidiaA100%2BEPYC7543.ac9). If anyone can advise further, I would really appreciate it. If I understand things, either elpa or magna might be beneficial. I intend to investigate nonlinear electron-phonon coupling aspects of susceptibility and Born effective mass. Cheers, Thomas
Hi Thomas, and welcome!
I might not have your specific hardware, but this is how we compiled a GPU-enabled Abinit on our cluster. Credits to @pomax
We first loaded the modules:
module load CCconfig
module load StdEnv/2023
module load gentoo/2023
module load gcccore/.13.3
module load nvhpc/25.1
module load cuda/12.6
module load imkl/2024.2.0
module load mii/1.1.2
module load python/3.11.5
module load blis/1.0
module load hwloc/2.10.0
module load ucx/1.16.0
module load libfabric/1.21.0
module load pmix/5.0.2
module load ucc/1.3.0
module load openmpi/5.0.3
module load libxc/7.0.0-gpu
module load hdf5-mpi/1.14.6-gpu
module load netcdf-mpi/4.9.3-gpu
module load netcdf-fortran-mpi/4.5.4-gpu
Then we used the traditional configure script (and not CMake) with the command
../configure -q --with-config-file=gpu-abinit.ac9 --prefix=/home/pomax/projects/def-antonius/shared/abinit/10.2.7-gpu
And this is the content of the configure file:
# AC9 FILE ABINIT GPU
# You'll probably need to remove -lnvc from abinit executable and recompile without it
# I haven't found a solution to just remove it so just look at the command in install.sh
# Then you can verify with ldd ./abinit | grep libnvc
CC=$(which mpicc)
CXX=$(which mpicxx)
FC=$(which mpifort)
CXX_LDFLAGS="-lxml2 -lcurl -lzip"
CC_LDFLAGS="-lxml2 -lcurl -lzip"
FCFLAGS="-O2"
FC_LDFLAGS="-lcurl -lzip -lxml2"
with_optim_flavor="standard"
# Parallel compilation
# ========================================
with_mpi="yes"
enable_mpi_io="yes"
enable_mpi_inplace="yes"
enable_openmp="yes"
enable_openmp_offload="yes"
# CUDA GPU
# ========================================
with_gpu="/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Core/cudacore/12.6.2"
NVCC="$with_gpu/bin/nvcc"
NVCC_CFLAGS="-O2 -Xptxas=-v --debug --use_fast_math --compiler-options -O2,-fPIC,-DHAVE_CONFIG_H"
GPU_CFLAGS=" -I$with_gpu/include"
GPU_CPPFLAGS="-I$with_gpu/include"
GPU_CXXFLAGS="-I$with_gpu/include"
GPU_FCFLAGS=" -I$with_gpu/include -mp=gpu"
# Here, I cannot use clang as it requires hwloc/2.9.1 while openmpi/5.0.3 requires hwloc/2.10.0
# However, clang library are needed (libomptarget.so and libomptarget.rtl.cuda)
# So I'm just sending them directly hoping it will work
GPU_LIBS="-lcufft -lcusolver -lcublas -lcudart"
GPU_ARCH=80
# NetCDF library
# ========================================
with_hdf5="$H5DIR"
with_netcdf="$NCDIR"
with_netcdf_fortran="$NCFDIR"
# Additional plugins
# ========================================
with_libxc="$EBROOTLIBXC"
# ========================================
## NOTE: we need to set the linalg flags manually, so we do NOT activate with_linalg.
## Otherwise the flags are ignored.
#with_linalg="${MKLROOT}"
LINALG_CPPFLAGS="-I${MKLROOT}/include"
LINALG_CFLAGS="-I${MKLROOT}/include"
LINALG_CXXFLAGS="-I${MKLROOT}/include"
LINALG_FCFLAGS="-I${MKLROOT}/include"
LINALG_LDFLAGS="-L${MKLROOT}/lib -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lm -ldl -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64"
LINALG_LIBS="-L${MKLROOT}/lib -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lm -ldl"
Hello Antonius! Thank you very much for your help. Coincidentally, I was able to build my version (abinit-10.4.3)
just yesterday finally after of course, many syntax/option versions. I had build LibXC-7.0.0 and Wannier90, both with high derivatives, Magma with cpu/gpu, etc. Linear algebra was found as “mkl+magma” but I relied on flags for its detection.
