gromacs build/runtime issues on LC systems

[rfhaque]

1. The external blas/lapack libraries available on lassen (cusolver, cublas, essl etc) do not append an underscore in the symbol names. This results in a linker error in gromacs. It would be good to have a configure option in gromacs to account for this behavior

  >> ../../lib/libgromacs_mpi.so.9.0.0: undefined reference to `sscal_'
  >> ../../lib/libgromacs_mpi.so.9.0.0: undefined reference to `dlanst_'
  >> ../../lib/libgromacs_mpi.so.9.0.0: undefined reference to `sswap_'

Moreover, openblas does not build on lassen (clang assembler error)

Error: invalid switch -mpower10
Error: unrecognized option -mpower10
clang: error: assembler command failed with exit code 1 (use -v to see invocation)

As a result, we are forced to use gromacs' internal blas/lapack implementation on lassen (ppc64le)
This is enforced in the spack package.py for gromacs

2. Using rocblas/rocsolver for blas/lapack on tioga results in a segfault for the rocm experiment (intel mkl library works)

3. Enabling GPU-aware MPI on tioga (rocm mode) requires setting the CXX_EXE_LINKER_FLAGS in gromacs in package.py. However, this breaks the CUDA build on lassen. It would be desirable to have a config option in gromacs to pass in extra compiler/linker flags

4. Bonded interactions do not work on the GPU (its possible the input file used for testing does not have any GPU interactions). Is there a different input file/problem available to test bonded interactions?

5. The mapping of cores, gpus to ranks is not well-understood. Additional performance optimizations need to be investigate

6. On lassen, v2024 does not build for the CUDA variant (v2023.3 works fine)

[pearce8]

@rfhaque please split this issue into 2 separate issues:

• Spack- and benchpark-specific for @pearce8
• Gromacs-specific for @pszi1ard

[pszi1ard]

@rfhaque please split this issue into 2 separate issues:

* [ ]  Spack- and benchpark-specific for [@pearce8](https://github.com/pearce8)[ ]  Gromacs-specific for [@pszi1ard](https://github.com/pszi1ard)

I agree, I can take the GROMACS-related issue, feel free to assign to me.

Concretely related to some of the above:

• points 1/2: BLAS/LAPACK are not performance critical (only used in some post-simulation analysis tools, not for the MD engine), I'd not focus on those
• point 3: not sure how to address this, on our 235a machine I indeed pass -DMPI_CXX_LINK_FLAGS="${PE_MPICH_GTL_DIR_amd_gfx90a} -lmpi_gtl_hsa", but the LUMI EB scripts do not seem to need it: https://github.com/Lumi-supercomputer/LUMI-EasyBuild-contrib/blob/main/easybuild/easyconfigs/g/GROMACS/GROMACS-2024.3-cpeAMD-24.03-rocm.eb
• point 4. indeed, for the "water" case GROMACS won't accept "-bonded gpu" only "-bonded auto"; I will contribute new cases that include bonded workload and are also more modern than those in ramble.
• point 5. I can clarify / help implement that, we might want a separate issue for that?
• point 6: please share details, is it a GROMACS build system issue?