-
Notifications
You must be signed in to change notification settings - Fork 7
ELM on Windows Subsystems for Linux (WSL) Ubuntu
A NOTE on 03-15-2022: New Windows OS offers WSL installing by default of Ubuntu via PowerShell command: 'wsl --install'. My test shows that it's working, but NOT works if restart your system and login with non-admin account. So my suggestion is still to install it by following the mannual steps in WSL installing docs.
Now, it's time to download one of WSL supported LINUX Kernel Packages. SEEWSL linux kernel
Here it's an example of Ubuntu 20.4 for WSL. Ubuntu for WSL
(NOTE: the version is always in changing. So just go to Windows app store and download it )
Right click the downloaded package to run it to install (as an Admin).
NOW click Ubuntu in your Windows applications menus (You may pin it into taskbar so that next time open it by clicking its logo from Taskbar).
THEN you are entering Ubuntu linux terminals.
IF this is the first time to enter it, you will be asked to setup a user account as power user having SUDO authority with password setting now.
Also it's suggested to upgrade/update Ubuntu, with sudo, when first time entering it.
First, you may want to see a list of utilities/tools upgradable:
sudo apt list --upgradable
Or,
sudo apt update
Then, type command to upgrade/update them:
sudo apt upgrade
NOTE (1): this command also will update all later-installed packages if running again.
NOTE (2): You may have gotten errors, like 'http://archive.ubuntu.com ... temporarily unresolvable ... failure ....'. THIS actually implies that you have with WSL2, which automatically generate '/etc/resolv.conf' file, which very likely incorrectly setup. SO do the following:
FIRST, back to WSL1, by open a PowerShell terminal using admin login, and,
>wsl --set-version Ubuntu 1
It will shutdown any Ubuntu linux terminal and backward WSL2 to 1. Then start Ubuntu again. And make a copy of '/etc/resolv.conf', which actually is a link to '/run/resolvconf/resolv.conf' or '/mnt/wsl/resolvconf/resolv.conf', and edit it as following, assuming 'resolv.conf' linked to '/run/resolvconf/resolv.conf':
$sudo cp /run/resolvconf/resolv.conf /etc/wsl.conf
$ sudo vi /etc/wsl.conf
(comment out 2 lines by removing #, while leave the rest nameserver(s) as is(are):
[network]
generateResolvConf = false
)
SECOND, in the PowerShell terminal, restart WSL to 2, as following,
> wsl --set-version Ubuntu 2
It will shutdown Ubuntu terminal again. When WSL2 ready, open Ubuntu terminal, and copy '/etc/wsl.conf' to '/etc/resolv.conf',
$sudo rm /etc/resolv.conf
(This will remove the fake link)
$sudo cp /etc/wsl.conf /etc/resolv.conf
FINALLY, restart Ubuntu and the name servers issue will be gone.
Ubuntu default packages usually don't include compilers. Here is how to get GNU Compilers, usually of the most recent version, from Ubuntu update servers.
sudo apt-get install gcc
This will install GNU C/C++ compilers (gcc, g++), default version, into /usr/bin/. NOTE: sudo apt install gcc is working as well.
sudo apt-get install gfortran
This will install GNU Fortran compiler (gfortran) into /usr/bin.
IF for some reasons, having troubles to install GNU compilers, the following installation will do all (including gnu make) together:
sudo apt update
sudo apt install build-essential
ELM can be built/run in non-mpi modes, so MPI wrappers for gcc compilers are OPTIONAL. BUT usually local laptop/desktop system for ELM is a basical system for ELM development for further work in HPC, it's strongly to have MPI wrappers and running environments.
MPI wrapper, such as MPICH, can be installed as following:
sudo apt-get install mpich
This will install MPICH, of recent version, into /usr/bin, usually with gcc/g++/gfortran just installed above.
FOR Openmpi, it's installed as:
sudo apt install openmpi-bin
These include: BLAS/LAPACK, and, Netcdf-c/-fortran (which depending upon HDF5 if installing parallel netcdf, i.e. netcdf4.)
There are many options to get/install BLAS/LAPACK libraries. The following is from GSL (GNU Scientific Library).
sudo apt-get install libblas-dev liblapack-dev
These libraries will be installed into /usr/lib/x86_64-linux-gnu/. NOTE: Most libraries with applications installed above could be found in this directory, in case that code building/linking cannot find needed libraries.
Ubuntu provides hdf5/netcdf4 full packages. BUT what ELM needed is parallel version, i.e. built by MPI wrappers just installed above. THIS is critical to ELM building and running on any system.
- ZLIB is required for HDF5. However, the system zlib is sort of missing 'zlib.h' headers. So update it as following:
sudo apt-get install zlib1g-dev
NOTE that package name is '...1g-...' (before letter g is number 1 NOT letter l)
- HDF5 downloading and building:
wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.7/src/hdf5-1.10.7.tar.gz --no-check-certificate
NOTE: IF wget not installed, try to install it: sudo apt-get install wget
tar -xf hdf5-1.10.7.tar.gz
cd hdf5-1.10.7
CC=mpicc FC=mpif90 CPPFLAGS='-DgFortran -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64' ./configure --prefix=/usr/local/hdf5-1.10-parallel --enable-parallel --enable-fortran
NOTE: For configure command OR following make all && make checkto work, likely need to install GNU m4, like: sudo apt-get install m4
If configuring above shows no error, then build/check/install as following:
make all
make check
This 'check' may not pass for some parallel testing. It shall be OK. If checking stops due to this, run make check -i, ignoring those failures.
sudo make install
This final step will install it into /usr/local/hdf5-1.10-parallel. For easy use of hdf5 commands and libraries, edit ~/.bashrc to append HDF5 path and library, like following:
export HDF5_PATH=/usr/local/hdf5-1.10-parallel
export PATH=$HDF5_PATH/bin:$PATH
export LD_LIBRARY_PATH=$HDF5_PATH/lib:$LD_LIBRARY_PATH
TO allow this modification effective immediately, run command: source ~/.bashrc
- Netcdf4-C library (and utilities) downloading and building
wget https://github.com/Unidata/netcdf-c/archive/refs/tags/v4.8.0.tar.gz
tar -xf v4.8.0.tar.gz
cd netcdf-c-4.8.0
NOTE: this directory name is upon what the untar command above produced.
CC=mpicc CPPFLAGS='-DgFortran -I/usr/local/hdf5-1.10-parallel/include' LDFLAGS='-L/usr/local/hdf5-1.10-parallel/lib' ./configure --prefix=/usr/local/netcdf-4.x-hdf5-parallel --enable-netcdf4 --enable-shared --enable-static --disable-doxygen --disable-dap
IF configuring above is successful, then run command:
make all
make check
sudo make install
The installation directory is: /usr/local/netcdf-4.x-hdf5-parallel. ONCE again, modifying .bashrc to have netcdf path/lib.
export NETCDF_PATH=/usr/local/netcdf-4.x-hdf5-parallel
export PATH=$NETCDF_PATH/bin:$PATH
export LD_LIBRARY_PATH=$NETCDF_PATH/lib:$LD_LIBRARY_PATH
- Netcdf-4 Fortran downloading and library building
NOTE: Either netcdf-4 fortran or netcdf-cxx is based on netcdf-4 C library just built/installed above.
Downloading:
wget https://github.com/Unidata/netcdf-fortran/archive/refs/tags/v4.5.3.tar.gz
Building:
tar -xf v4.5.3.tar.gz
cd netcdf-fortran-4.5.3
CC=mpicc FC=mpif90 CPPFLAGS='-DgFortran -I/usr/local/netcdf-4.x-hdf5-parallel/include -I/usr/local/hdf5-1.10-parallel/include' LDFLAGS='-L/usr/local/netcdf-4.x-hdf5-parallel/lib -L/usr/local/hdf5-1.10-parallel/lib' ./configure --prefix=/usr/local/netcdf-4.x-hdf5-parallel --enable-shared --enable-static
NOTE: This configuring includes netcdf-c headers and library directories. IF successful, then build/check/install:
make all
make check
sudo make install
The netcdf-fortran library is installed into same directory as netcdf-c library. So NO need to modify .bashrc.
TIPS: At this point, it would show configurations of both netcdf-c and netcdf-fortran, by run command:
nc-config --all
And also it will show how to obtain headers/libraries, including full path of directories, for either netcdf-c or netcdf-fortran, using nc-config or nf-config
ELM Configuring/Building/Running are using shell scripts, python, and specifically perl(5) scripts. One special module is perl5 LibXML, which depending on libxml2 and probably xmllint (included within libxml2-utils).
- libxml2 installing:
sudo apt-get install libxml2-utils
sudo apt-get install libxml2 libxml2-dev
- perl5 LibXML installing:
sudo cpan install XML::LibXML
NOTE: IF this is first-time using cpan, ubuntu will install it (automatically), if not yet within it, prior to pulling in XML::LibXML
- Python
Ubuntu default has already installed Python3. For easy installing python modules, it's better to have 'pip' installed.
sudo apt update
sudo apt-get install python3-pip
OPTIONAL, but numpy, scipy, and matplotlib packages are required if using OLMT for ELM configuring/building/running.
sudo pip install numpy
sudo pip install scipy
sudo pip install matplotlib
With OLMT, when generating point data from global datasets, it needs nco package:
sudo apt install nco
CAUTIOUS, this nco package with Ubuntu will install its own netcdf package in system lib directories. So be careful not to mess up with the parallel version installed above.
- An example $HOME/.bashrc for WSL/Ubuntu: see bashrc for ELM on WSL
NOTE: The following examples are using E3SM model and point-level input datasets:
-
E3SM repository: E3SM with porting to WSL
-
Offline ELM point-level inputs repository: pt-e3sm-inputdata examples
NOTE: after cloning this dataset into local, please tar -xf 3 compressed data under: atm/datm/NASA_LIS/, lnd/clm2/firedata/