Skip to content

Commit

Permalink
Merge pull request #249 from cecilehannay/main
Browse files Browse the repository at this point in the history 
tutorial special queue and one time instructions
  • Loading branch information
@cecilehannay
cecilehannay authored Aug 2, 2024
2 parents 20082c8 + 0def871 commit 9560ac1
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 313 deletions.
65 changes: 0 additions & 65 deletions notebooks/resources/tutorial_one_time_setup.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,71 +8,6 @@
"# Tutorial one time setup"
]
},
{
"cell_type": "markdown",
"id": "4a6f76fe-9658-443c-89a7-9ede364803e5",
"metadata": {},
"source": [
"### Tutorial Project Account\n",
"\n",
"<div class=\"alert alert-warning\">\n",
"You should have access to project account <strong>UESM0013</strong> and use this for your simulations.\n",
"</div>\n",
"\n",
"You can change the account in your case directory using the following command:\n",
"```\n",
"./xmlchange PROJECT=UESM0013\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "9db9d54e-ef78-44ef-a003-4ddd8a416317",
"metadata": {},
"source": [
"### Special Queues\n",
"\n",
"<div class=\"alert alert-warning\">\n",
"\n",
"<strong>Schedule for Tutorial Reservation Queues</strong>\n",
" \n",
"- We have a special queue <strong>tutorial</strong> during the lab sessions to ensure you get through the derecho queues quickly and get your jobs run. This queue are only active during our lab sessions. \n",
"\n",
"- Outside the lab sessions, you should use the queue <strong>cpu</strong>\n",
" \n",
"</div>\n",
"\n",
"If you need to change the value for a run, you can change the reservation queue in your case directory using the following command:\n",
"\n",
"During the lab sessions:\n",
"```\n",
"./xmlchange JOB_QUEUE=tutorial --force\n",
"```\n",
"\n",
"Outside the lab sessions:\n",
"```\n",
"./xmlchange JOB_QUEUE=cpu --force\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "e127deda-73f0-42fc-9c96-e49bb1bb0f8b",
"metadata": {},
"source": [
"### Using these tutorial-specific codes"
]
},
{
"cell_type": "markdown",
"id": "ec3ec832-823f-48ce-b142-05c69dcc72b1",
"metadata": {},
"source": [
"In theory, you _should_ be able to automatically use this account and the reservation queues if you correctly set up your NCAR HPC environment using your `.profile` or `.tcshrc` files, as described below. If you need to change the values for a run, you can change the account and reservation queue in your case directory using the following command:\n",
"\n",
"If you do use these commands, make sure for the `JOB_QUEUE` you are using the **correct** reservation code for the relevant day _and_ time of that lab session. You should not have to set the `PROJECT` more than once during the tutorial if your profile is set up, as described below, but we include it for your reference in case you are having trouble."
]
},
{
"cell_type": "markdown",
"id": "0c07b51d-f999-4737-9739-3ea989f20802",
Expand Down
266 changes: 18 additions & 248 deletions notebooks/resources/tutorial_queue_account.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
" \n",
"- We have a special queue <strong>tutorial</strong> during the lab sessions to ensure you get through the derecho queues quickly and get your jobs run. This queue are only active during our lab sessions. \n",
"\n",
"- Outside the lab sessions, you should use the queue <strong>cpu</strong>\n",
"- Outside the lab sessions, you should use the queue <strong>cpu</strong> to run cesm.\n",
" \n",
"</div>\n",
"\n",
Expand All @@ -56,282 +56,52 @@
]
},
{
"cell_type": "markdown",
"id": "e127deda-73f0-42fc-9c96-e49bb1bb0f8b",
"metadata": {},
"source": [
"### Using these tutorial-specific codes"
]
},
{
"cell_type": "markdown",
"id": "ec3ec832-823f-48ce-b142-05c69dcc72b1",
"metadata": {},
"source": [
"In theory, you _should_ be able to automatically use this account and the reservation queues if you correctly set up your NCAR HPC environment using your `.profile` or `.tcshrc` files, as described below. If you need to change the values for a run, you can change the account and reservation queue in your case directory using the following command:\n",
"\n",
"If you do use these commands, make sure for the `JOB_QUEUE` you are using the **correct** reservation code for the relevant day _and_ time of that lab session. You should not have to set the `PROJECT` more than once during the tutorial if your profile is set up, as described below, but we include it for your reference in case you are having trouble."
]
},
{
"cell_type": "markdown",
"id": "0c07b51d-f999-4737-9739-3ea989f20802",
"metadata": {},
"source": [
"## Setting up your NCAR HPC environment"
]
},
{
"cell_type": "markdown",
"id": "9d6954eb-3505-4a58-b6fa-bea7047c63cf",
"metadata": {},
"source": [
"Every time you log onto the NCAR HPC you want to ensure you have the correct modules loaded."
]
},
{
"cell_type": "markdown",
"id": "7440f1b8-06af-435a-b1db-d444b99bf846",
"metadata": {},
"source": [
"<div class=\"alert alert-warning\">\n",
"<strong>NOTE: The next section describes how to use a file that loads the modules necessary for those attending the in-person tutorial. Once you set up your environment you should not have to do it again for the remainder of the tutorial. Every time you login to NCAR HPC these files will automatically be sourced. However, if you intend to use NCAR HPC after the tutorial you may need to modify your `.profile` or `.tcshrc` file to load the correct modules, project numbers, or queues. </strong>\n",
"\n",
"</div>"
]
},
{
"cell_type": "markdown",
"id": "8e6da3c0-4cbf-42e3-b599-9ef4d1b4a72c",
"metadata": {},
"source": [
"___\n",
"\n",
"### SHELL environment "
]
},
{
"cell_type": "markdown",
"id": "c04124a8-5e76-4104-8d62-7bd0a329ce58",
"metadata": {},
"source": [
"To determine what shell environment you are using on the NCAR HPC, type the following command:"
]
},
{
"cell_type": "markdown",
"id": "9f8488f1-72e7-4594-be66-f875c51cc4e1",
"metadata": {},
"source": [
"<div class=\"alert alert-info\">\n",
"\n",
"```\n",
"echo $SHELL\n",
"```\n",
"\n",
" \n",
"If the command returns, you are a bash user\n",
"```/bin/bash```\n",
"\n",
" \n",
"If the command returns, you are a cshell user\n",
"```/bin/tcsh```\n",
"\n",
"\n",
"</div>"
]
},
{
"cell_type": "markdown",
"id": "91a8addf-1e89-43b9-a364-1e1aa151454d",
"metadata": {},
"source": [
"**NOTE:** All new NCAR HPC accounts default to bash. "
]
},
{
"cell_type": "markdown",
"id": "8c4f6e24-577b-4736-a251-69b51553d704",
"metadata": {},
"source": [
"___\n",
"\n",
"### Bash users"
]
},
{
"cell_type": "markdown",
"id": "a82c1d5f-eddc-4504-bd66-febd2f3179f8",
"metadata": {},
"source": [
"**NOTE:** You may already have a `.profile` file in your home directory. \n",
"- If you have an existing `.profile` file and do not wish to overwrite its contents, please save your original `.profile` file to a new filename before copying the tutorial .profile file into your home directory. \n",
"- Alternatively, you could copy the contents of the tutorial profile file into your existing `.profile` file using a text editor. (see special instructions below)"
]
},
{
"cell_type": "markdown",
"id": "870dc6c7-98f5-4e3a-b226-2f2437d5321a",
"cell_type": "code",
"execution_count": 1,
"id": "6200f0a4-25b7-4013-9062-18a103627c29",
"metadata": {},
"outputs": [],
"source": [
"#### One-time setup\n",
"To set up your environment for the tutorial, follow the following three steps:"
"### Compiling in the Special Queue"
]
},
{
"cell_type": "markdown",
"id": "9a2baee0-804e-4586-b8fc-a3636c528484",
"id": "6ddf7177-0dd6-4ce3-be1e-ee70e5be9419",
"metadata": {},
"source": [
"<div class=\"alert alert-info\">\n",
"By default, we compile cesm on a shared node called `cpudev`. However, as we want to avoid long wait time during the labs, we will compile in the specialed reservation queue `tutorial`. \n",
"\n",
"Copy over the `.profile` file into your home directory: \n",
" \n",
"```\n",
"cp /glade/campaign/cesm/tutorial/tutorial_2024_env/profile ~/.profile\n",
"```\n",
"<br>\n",
"\n",
"Source the file:\n",
"During the lab sessions, use\n",
"\n",
"```\n",
"source ~/.profile\n",
"qcmd -q tutorial -- ./case.build \n",
"```\n",
" \n",
"</div>\n"
]
},
{
"cell_type": "markdown",
"id": "bf95578e-40e5-4c40-8884-0b80e29eb858",
"metadata": {},
"source": []
},
{
"cell_type": "markdown",
"id": "4daab2a7-58c1-41f0-b51b-c9e8a573574b",
"metadata": {},
"source": [
"___\n",
"\n",
"### Tcsh users"
]
},
{
"cell_type": "markdown",
"id": "42432d90-eaef-4205-b4fe-39cf3c7b5881",
"metadata": {},
"source": [
"**NOTE:** You should already have a `.tcshrc` file in your home directory. If you do not wish to overwrite the existing `.tcshrc` file, please save this file to a new name OR copy the contents of the `/glade/campaign/cesm/tutorial/tutorial_2024_env/tcshrc` file to your `.tcshrc` file using a text editor."
]
},
{
"cell_type": "markdown",
"id": "0722df3e-2497-492c-8301-7d54e7485c05",
"metadata": {},
"source": [
"#### One-time setup\n",
"\n",
"To set up your environment for the tutorial, follow the following three steps:"
]
},
{
"cell_type": "markdown",
"id": "a77636dd-1651-431a-a72e-f862947b37a6",
"metadata": {},
"source": [
"<div class=\"alert alert-info\">\n",
"\n",
"<br>\n",
"Outside the lab sessions, use\n",
"\n",
"Copy over the `.tcshrc` file to your home directory:\n",
"```\n",
"cp /glade/campaign/cesm/tutorial/tutorial_2024_env/tcshrc ~/.tcshrc\n",
"```\n",
" \n",
"<br>\n",
"\n",
"Source the file:\n",
"\n",
"```\n",
"source ~/.tcshrc\n",
"```\n",
" \n",
"</div>\n"
"qcmd -- ./case.build \n",
"```"
]
},
{
"cell_type": "markdown",
"id": "637d5219-104b-4e53-ab41-b3969b275527",
"metadata": {},
"source": []
},
{
"cell_type": "markdown",
"id": "5edccc8b-0080-4837-aa70-d50360cb2c6c",
"id": "e127deda-73f0-42fc-9c96-e49bb1bb0f8b",
"metadata": {},
"source": [
"___\n",
"\n",
"### If you have any trouble\n",
"\n",
"**Please ask for help if you are having trouble with this step!**\n",
"\n",
"The modules necessary for this tutorial and loaded through these files are:\n",
"- ncarenv/1.3\n",
"- intel\n",
"- ncl\n",
"- nco\n",
"- ncview\n",
"- netcdf\n"
"### Using these tutorial-specific codes"
]
},
{
"cell_type": "markdown",
"id": "71cae660-0d1a-4193-8333-909310c49dac",
"id": "ec3ec832-823f-48ce-b142-05c69dcc72b1",
"metadata": {},
"source": [
"____\n",
"\n",
"### Special instructions (if you already had an account on derecho)\n",
"\n",
"If you already have a `.profile` or `.tcsh` and you do not wish to overwrite its contents, you need to add the following into your existing `.profile` or `.tcsh` using a text editor.\n",
"\n",
"#### Bash:\n",
"\n",
"<div class=\"alert alert-info\">\n",
"\n",
"Add to your .profile: \n",
"\n",
"```\n",
"export PROJECT=UESM0013 #<- This should never change over the course of the tutorial\n",
"export TUTORIAL_QUEUE=tutorial #<- Special queue for the tutorial\n",
"```\n",
"\n",
"Source the file:<br> \n",
"\n",
"```\n",
"source ~/.profile\n",
"```\n",
"</div>\n",
"\n",
"#### C-shell:\n",
"\n",
"<div class=\"alert alert-info\">\n",
"\n",
"Add to your .tcshrc: \n",
"```\n",
"setenv PROJECT UESM0013 #<-This should never change over the course of the tutorial\n",
"setenv TUTORIAL_QUEUE tutorial #<- Special queue for the tutorial\n",
"```\n",
" \n",
"Source the file:<br>\n",
"\n",
"```\n",
"source ~/.tcshrc\n",
"``` \n",
"In theory, you _should_ be able to automatically use this account and the reservation queues if you correctly set up your NCAR HPC environment using your `.profile` or `.tcshrc` files, as described below. If you need to change the values for a run, you can change the account and reservation queue in your case directory using the following command:\n",
"\n",
"</div>"
"If you do use these commands, make sure for the `JOB_QUEUE` you are using the **correct** reservation code for the relevant day _and_ time of that lab session. You should not have to set the `PROJECT` more than once during the tutorial if your profile is set up, as described below, but we include it for your reference in case you are having trouble."
]
},
{
Expand Down

0 comments on commit 9560ac1

Please sign in to comment.