Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: admin talk #177

Merged
merged 33 commits into from
Jan 25, 2025
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
e497c7a
feat: a bit more emphasis on workflow managers vs. batch system alone
cmeesters Jan 6, 2025
6f8f110
feat: new image underlining the need for workflow managers
cmeesters Jan 6, 2025
5384b0f
feat: new config for admin talk
cmeesters Jan 8, 2025
2543a30
feat: new title for admin talk - strictly speaking, not for admins alone
cmeesters Jan 8, 2025
7d50148
fix: better title to introduce constituents
cmeesters Jan 8, 2025
3f05133
feat: new dag slide
cmeesters Jan 10, 2025
c7f15b4
fix: pure SLURM solution slide now with text line explaining the image
cmeesters Jan 10, 2025
3bfad7c
feat: added note on EESSI
cmeesters Jan 12, 2025
e7e3f91
fix: cleanup of condarc
cmeesters Jan 12, 2025
4d2aa93
feat: half done - sample slides in bash
cmeesters Jan 13, 2025
9cb381e
fix: removed part which gives rise to misunderstandings
cmeesters Jan 13, 2025
152e235
Merge branch 'feat/admin_talk' of github.com:snakemake/snakemake-hpc-…
cmeesters Jan 13, 2025
43a606c
fix: corrected indentation of sample code
cmeesters Jan 13, 2025
81852d7
fix: renaming of proteotranstomic DAG
cmeesters Jan 17, 2025
1689d5b
feat: course announcement - needs generalization
cmeesters Jan 23, 2025
04761bc
fix: layout
cmeesters Jan 23, 2025
42e2c7c
feat: bash slides - needs to be made optional+
cmeesters Jan 23, 2025
8f0cb7b
feat: software stack better explained
cmeesters Jan 23, 2025
c3c5654
feat: admin part w/o user content for bashrc
cmeesters Jan 23, 2025
507aa46
feat: added syntax part for yaml
cmeesters Jan 23, 2025
51cc3d5
Merge branch 'feat/admin_talk' of github.com:cmeesters/snakemake-hpc-…
cmeesters Jan 23, 2025
77b22af
Merge branch 'main' into feat/admin_talk
cmeesters Jan 23, 2025
f975272
fix: outcommented long bash section
cmeesters Jan 25, 2025
090829e
feat: comment on outcommented section
cmeesters Jan 25, 2025
6fe779a
fix: added missing image
cmeesters Jan 25, 2025
9cba3b0
Merge branch 'feat/admin_talk' of github.com:cmeesters/snakemake-hpc-…
cmeesters Jan 25, 2025
c4b1115
Update slides/Snakemake_HPC_Admins.tex
cmeesters Jan 25, 2025
781d488
fix: typo
cmeesters Jan 25, 2025
4d87421
Merge branch 'feat/admin_talk' of github.com:snakemake/snakemake-hpc-…
cmeesters Jan 25, 2025
f516fee
fix: removing build condition to build slides also upon merger
cmeesters Jan 25, 2025
3e81a79
fix: attempt to fix artifact upload in release merger
cmeesters Jan 25, 2025
f2bb7b0
Update .github/workflows/release-please.yml
cmeesters Jan 25, 2025
c64495b
Update .github/workflows/release-please.yml
cmeesters Jan 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions config/config_Erlangen_Admin.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
instructor:
name: "Dr. Christian Meesters"
affiliation: "Johannes Gutenberg Universität Mainz"
affiliation2: "NHR SüdWest" # may be empty

layout:
beamertheme: "CambridgeUS" # you can choose any, the slides are tested
# with 'CambridgeUS'
colortheme: "beaver" # you can choose any, the slides are tested with
# 'beaver'
# choose any preconfigured color or those from the preamble
beamercolor_structure: "fg=UniRot"
beamercolor_title: "fg=UniRot"
beamercolor_title_head: "fg=UniRot"

beamercolor_block_title: "bg=UniRot!20,fg=darkred"
beamercolor_block_body: "fg=black, bg=plightgrey2"

beamercolor_block_title_alerted: "fg=white,bg=UniRot"
beamercolor_block_title_example: "fg=white,bg=PineGreen!80"

course:
# sets the slides aspectratio (if outcommented, the default is 43=4:3)
aspectratio: 1609 # - 169 = 16:9
date: "14. Januar 2025" # maybe: "\today"
#title: "" # specifiy your title here, else a default is used.
#subtitle: "" # specifiy your subtitle here, else a default is used.
question: "yes" # if set (to any value) the question slide will be shown
# # at chapters with content. If you do not want it, this
# # should be outcommented.
# Editors recommendations are a matter of taste and technological
# setups (e.g. on-demand setups). Hence, specify an editor-slide,
# here:
editorfile: "common/mod_cs_mainz"
# The next line indicates the slides explaining
# a condarc file on a cluster, copy and adjust
# for your cluster.
condarcfile: "common/condarc_mogon.tex"
# the same for admins:
condarc_admins_file: "admin/condarc_mogon.tex"
# We provide a common software stack to avoid long
# installation sessions.
softwarepath: "/lustre/project/nhr-workflow/software_stack"
# The following path will indicate a sample workflow for the user part
# if it is defined, the slide set will instruct participants to copy
# it and adapt its configuration and profile.
pathtoworkflow: "/lustre/project/nhr-workflow/workflow"
# Alternatively a URL may be indicated. In this case the participants
# are instructed to deploy the workflow using snakedeploy.
#deploy_url: ""
# these are the path names to contain sample data
# see the README for explanations.
pathtosetup: "/lustre/project/nhr-workflow/tasks"
pathtosolutions: "/lustre/project/nhr-workflow/solutions"
# after copying the setup path is:
tutorialpath: "~/tasks/tutorial"
# This is the selected slide to explain the 'Hello World' script
# to introduce SLURM. It requires different account and partition
# settings on every cluster.
hello_world_script: "common/Hello_World_HPC_Mogon_NHR.tex"

cluster:
# the cluster name
name: "Mogon-NHR"
# the cluster distro
distro: "Alma-Linux"
# the cluster account and default partition
account: "nhr-workflow"
partition: "smallcpu"
# the local and remote storage prefixes on this cluster
remote-job-local-storage-prefix: "/localscratch/$SLURM_JOB_ID"
local-storage-prefix: "/dev/shm/$USER/snakemake"
# display program for SVG files
display_program: "display"
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified images/workflows/SLURM_workflow_solution.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/workflows/proteotranstomic.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions images/workflows/pure_SLURM.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,6 @@
ax.spines['right'].set_visible(False)
ax.spines['bottom'].set_visible(False)
ax.spines['left'].set_visible(False)
ax.annotate("lines of code", xy=(10,10), xycoords="figure pixels")
#plt.show()
plt.savefig('SLURM_workflow_solution.png', bbox_inches='tight')
31 changes: 30 additions & 1 deletion slides/Snakemake_HPC_Admins.tex
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

\input{common/preamble}

\title[<++course.shorttitle++>]{<+++ if course.title is defined +++><++course.tittle++><+++else+++>Importance of \Snakemake Workflows for Admins<+++endif+++>}
\title[<++course.shorttitle++>]{<+++ if course.title is defined +++><++course.tittle++><+++else+++>Importance of \Snakemake Workflows for Admins and HPC-Users<+++endif+++>}

\subtitle{<+++ if course.subtitle is defined +++><++course.subtitle++><+++else+++>Providing DataAnalytics Service<+++endif+++> - <++course.edition++>}

Expand All @@ -43,8 +43,37 @@
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\include{admin/software_environment}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\include{admin/actual_work}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\include{admin/Workflow_Parameterization_for_HPC}

\begin{frame}
\frametitle{Upcoming Courses}
\begin{columns}[T]
\begin{column}{0.5\textwidth}
\centering
\includegraphics[width=0.7\textwidth]{humor/announcemnt.png}
cmeesters marked this conversation as resolved.
Show resolved Hide resolved
\caption{Me, announcing courses}
\end{column}

\begin{column}{0.5\textwidth}
Do you want to learn:
\begin{itemize}
\item How to create \& publish \Snakemake workflows suitable for HPC clusters?
\item Learn how to deploy and adapt $3^{\mathsf{rd}}$-party workflows?
\item and more \ldots
\end{itemize}
Check out these courses in
\begin{itemize}
\item \lhref{https://indico.zdv.uni-mainz.de/event/23/}{Mainz, 29. \& 30. Jan.}
\item \lhref{https://tu-dresden.de/zih/hochleistungsrechnen/nhr-training/bd-hpda/snakemake?set_language=en}{Dresden, 26. \& 27. Feb.}
\end{itemize}
\end{column}

\end{columns}
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\include{admin/Data_Management}
Expand Down
8 changes: 4 additions & 4 deletions slides/admin/About_Snakemake.tex
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ \subsection{\Snakemake and the Workflow Catalogue}
\begin{figure}
\centering
\caption*{\textbf{>1e6} downloads since 2015\newline
\textbf{>1300} citations\newline
\textbf{>3000} citations\newline
\textbf{>7} citations per week since 2021}
\includegraphics[width=0.6\textwidth]{Snakemake/paper_wall.png}
\end{figure}
Expand Down Expand Up @@ -79,8 +79,8 @@ \subsection{\Snakemake and the Workflow Catalogue}
\begin{columns}
\begin{column}{0.5\textwidth}
\begin{itemize}[<+->]
\item Extremely feature rich: \lhref{https://snakemake.github.io/snakemake-workflow-catalog/}{over 1800 workflows (mostly bioinformatics)}
\item Almost a hundred standardized workflows ready to use (meaning: well documented and with automatic deployment)
\item Extremely feature rich: \lhref{https://snakemake.github.io/snakemake-workflow-catalog/}{over 3000 workflows (mostly bioinformatics)}
\item Almost three hundred standardized workflows ready to use (meaning: well documented and with automatic deployment)
\item Cluster batch systems are supported (and support for various cloud systems, too)
\item There is an option to include Nextflow wrappers, too.
\end{itemize}
Expand All @@ -89,7 +89,7 @@ \subsection{\Snakemake and the Workflow Catalogue}
\begin{figure}
\includegraphics[width=\textwidth]{Snakemake/Snakemake_Workflow_Catalog.png}
\vspace{0.5em}
\caption*{Screenshot of the Workflow Catalogue}
\caption*{Screenshot of the \lhref{https://snakemake.github.io/snakemake-workflow-catalog/}{Workflow Catalogue}}
\end{figure}
\end{column}
\end{columns}
Expand Down
Loading
Loading