Add apptainer files #10

Edgar-21 · 2025-01-15T18:57:09Z

No description provided.

gonuke

Thanks @Edgar-21 - this seems useful. I have a few suggestions and clarifying questions.

gonuke · 2025-01-16T03:55:29Z

apptainer_files/dagmc_openmc/dagmc_openmc.def

+From: /software/chtc/containers/ubuntu/22.04/openmpi-4.1.6_gcc-11.3.0.sif
+
+%arguments
+    HDF5_VERSION=hdf5_1_14_3


Seems like this should be defined in a way to be useful below? It's currently unused and the version is hard coded.

added args that allow controlling the hdf5 version by providing a url to wget from.

gonuke · 2025-01-16T03:57:24Z

apptainer_files/dagmc_openmc/dagmc_openmc.def

+		libpng-dev \
+		libnetcdf-dev \


indentation

gonuke · 2025-01-16T03:57:43Z

apptainer_files/dagmc_openmc/dagmc_openmc.def

+    wget https://support.hdfgroup.org/releases/hdf5/v1_14/v1_14_3/downloads/hdf5-1.14.3.tar.gz
+    tar -xvf hdf5-1.14.3.tar.gz


see above comment on variables for this version

gonuke · 2025-01-16T03:58:24Z

apptainer_files/dagmc_openmc/dagmc_openmc.def

+    MOAB_VERSION=5.3.0
+    DOUBLE_DOWN_VERSION=v1.0.0
+    EMBREE_VERSION=v3.12.2
+    DAGMC_VERSION=v3.2.1


why not newer versions? v3.2.3 has been around a while and v3.2.4 just dropped

I'll try with the latest supported versions of moab, dagmc and double down and see, but this build process is fairly fragile.

working with dagmc 3.2.4, moab 5.5.1, doubledown 1.1.0

and embree 4.3.3

gonuke · 2025-01-16T03:59:58Z

apptainer_files/dagmc_openmc/build_image.sh

+
+apptainer build \
+        --bind $TMPDIR:/tmp \
+        "${@:2}" \


What does this do?

nothing, it has been removed

gonuke · 2025-01-16T04:00:43Z

apptainer_files/dagmc_openmc/build_image.sh

+#SBATCH --error=job.%J.err
+#SBATCH --output=job.%J.out
+
+export JOB_TMP_PATH=/local/$USER/${SLURM_JOB_ID}


This seems like the only line that is CHTC specific, because of the path, right?

yes, I think so? I've not used other clusters not sure what might change

Maybe we can add a comment that notes that this particular path is correct for CHTC/HPC so that folks no where to change if they want to try this elsewhere

gonuke · 2025-01-16T04:01:23Z

apptainer_files/dagmc_openmc/dagmc_openmc.def

@@ -0,0 +1,148 @@
+Bootstrap: localimage
+From: /software/chtc/containers/ubuntu/22.04/openmpi-4.1.6_gcc-11.3.0.sif


Does CHTC publish the def file for this image? This is the only line that makes this CHTC-specific, right?

the def file is available in that same directory. I think this image is designed to have the openmpi built in a way that is compatible, and somehow also interfaces with spack, to what end I am not sure.

Just trying to think about how to make this portable to other systems, or at least note where others would need to be careful if trying to port to other systems. I wonder if it's possible to summarize what this image has in a comment.

added a few comments noting where things are uw hpc specific, and a brief explanation of the base image, as I understand it.

lewisgross1296

Few comments and suggestions. Open to discuss if you don't like all my suggestions

lewisgross1296 · 2025-01-28T21:16:32Z

apptainer_files/example_submit_script.sh

+#!/bin/bash
+#SBATCH --partition=pre       # default "shared", if not specified
+#SBATCH --time=0-24:00:00       # run time in days-hh:mm:ss
+#SBATCH --nodes=10               # require 1 nodes
+#SBATCH --ntasks-per-node=32   # cpus per node (by default, "ntasks"="cpus")
+#SBATCH --mem=30000


Might be good to mention the upper limits for things like time, nodes, ntasks-per-node, and mem. I know these are probably on the CHTC website, but it would be nice to have in the script so you don't have to stop what you're doing to look it up.

I think I'd prefer to avoid duplicating this information, like you mentioned its on the CHTC website. I'll remove the existing comments for consistency

Fair enough. Especially if these values change in the future, we'd want to make sure the info is accurate. Probably best just to direct people to the CHTC page. Can you add the link to the README.md in this repo?

lewisgross1296 · 2025-01-28T21:19:22Z

apptainer_files/example_submit_script.sh

+
+module load openmpi
+# image_path is the path to the Apptainer .sif file you wish to use


Suggested change

module load openmpi

# image_path is the path to the Apptainer .sif file you wish to use

# load required modules here

module load openmpi

# image_path is the path to the Apptainer .def or .sif file you wish to use

This is an example to submit a job, I don't think a .def is applicable here

You can submit jobs to build containers :)

apptainer_files/example_submit_script.sh

lewisgross1296 · 2025-01-28T21:33:27Z

apptainer_files/dagmc_openmc_uw_hpc/build_image.sh

+#SBATCH --partition=pre
+#SBATCH --time=0-04:00:00
+#SBATCH --nodes=1
+#SBATCH --ntasks-per-node=8
+#SBATCH --mem-per-cpu=4000


Should we maybe request a bit more power to build this quicker?

Co-authored-by: Lewis Gross <[email protected]>

gonuke · 2025-02-04T19:48:16Z

I think @lewisgross1296 should merge when he decides this is ready

Edgar-21 · 2025-02-04T20:32:02Z

Confirmed to be working on UW HPC.

Edgar-21 added 2 commits January 15, 2025 12:53

organize files, add apptainer def

03591d3

tweak readme

efb99cb

Edgar-21 mentioned this pull request Jan 15, 2025

Compile DAGMC on HPC cnerg/cnerg.github.com#288

Open

Edgar-21 requested a review from gonuke January 15, 2025 19:33

gonuke requested changes Jan 16, 2025

View reviewed changes

gonuke requested review from lewisgross1296, connoramoreno and FusionSandwich January 16, 2025 21:16

Edgar-21 added 3 commits January 16, 2025 15:18

use newer dagmc, moab and doubledown, make hdf5 vars useful

25ea126

indentation

8adde98

use latest embree version

2f7142d

Edgar-21 requested a review from gonuke January 16, 2025 22:33

Edgar-21 added 2 commits January 16, 2025 16:51

remove extraneous line

b7ea267

add clarifying comments about uw hpc

b367888

lewisgross1296 requested changes Jan 28, 2025

View reviewed changes

Edgar-21 and others added 3 commits February 4, 2025 12:42

Update apptainer_files/example_submit_script.sh

a7c4932

Co-authored-by: Lewis Gross <[email protected]>

Update apptainer_files/example_submit_script.sh

5a59ef4

Co-authored-by: Lewis Gross <[email protected]>

use more cores for building image

8ac9256

Edgar-21 requested a review from lewisgross1296 February 4, 2025 18:56

add link to chtc hpc overview to readme

0889bc0

lewisgross1296 approved these changes Feb 4, 2025

View reviewed changes

lewisgross1296 merged commit 0a1672c into cnerg:main Feb 4, 2025

Edgar-21 deleted the add_apptainer_files branch February 4, 2025 20:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add apptainer files #10

Add apptainer files #10

Edgar-21 commented Jan 15, 2025

gonuke left a comment

gonuke Jan 16, 2025

Edgar-21 Jan 28, 2025

gonuke Jan 16, 2025

gonuke Jan 16, 2025

gonuke Jan 16, 2025

Edgar-21 Jan 16, 2025

Edgar-21 Jan 16, 2025

Edgar-21 Jan 16, 2025

gonuke Jan 16, 2025

Edgar-21 Jan 16, 2025

gonuke Jan 16, 2025

Edgar-21 Jan 16, 2025

gonuke Jan 17, 2025

gonuke Jan 16, 2025

Edgar-21 Jan 16, 2025

gonuke Jan 17, 2025

Edgar-21 Jan 28, 2025

lewisgross1296 left a comment

lewisgross1296 Jan 28, 2025

Edgar-21 Feb 4, 2025

lewisgross1296 Feb 4, 2025

lewisgross1296 Jan 28, 2025

Edgar-21 Feb 4, 2025

lewisgross1296 Feb 4, 2025

lewisgross1296 Jan 28, 2025

Edgar-21 Feb 4, 2025

gonuke commented Feb 4, 2025

Edgar-21 commented Feb 4, 2025

		wget https://support.hdfgroup.org/releases/hdf5/v1_14/v1_14_3/downloads/hdf5-1.14.3.tar.gz
		tar -xvf hdf5-1.14.3.tar.gz

		@@ -0,0 +1,148 @@
		Bootstrap: localimage
		From: /software/chtc/containers/ubuntu/22.04/openmpi-4.1.6_gcc-11.3.0.sif


		module load openmpi
		# image_path is the path to the Apptainer .sif file you wish to use

Add apptainer files #10

Add apptainer files #10

Conversation

Edgar-21 commented Jan 15, 2025

gonuke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lewisgross1296 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gonuke commented Feb 4, 2025

Edgar-21 commented Feb 4, 2025