The IT staff will install software on our Linux servers on request, but compiling software can be a difficult task. EasyBuild aims to make it easier, and what's more: you can do it yourself!
EasyBuild works by having a large online database of build instructions for various software, with pointers for download. Compiling and installing a piece of software with all dependencies ("toolchain") can be done with a single command.
EasyBuild can be run on mimi.uio.no (our main RHEL8 server), and also on any other RHEL8 machine at the department (for instance your personal desktop machine).
The IT staff can use it to install software to a central software location, so that you only need to load the software as a module. Or you can install it yourself in either your home directory, or perhaps on a secondary storage area.
This guide uses the convention where user commands are prefixed by $, while commands to be executed by root (or with sudo) are prefixed by #. Don't type the prefix character. Lines without a prefix are output from the command.
(Note: This is brand-new September 2022, some teething problems might arise)
Install EasyBuild
Start by checking that you have a RHEL8 machine:
$ lsb_release -d Description: Red Hat Enterprise Linux release 8.6 (Ootpa)
You should prepare a filesystem on /opt/software/ with plenty of space, if possible. Get in touch, if you're unsure how to do this or don't have root access.
Then, run the install script, as root:
# /net/otto/opt/software/utils/eb-sync
The script will install EasyBuild and the module software, and some prerequisites.
NOTE: If you run the script again later, you risk overwriting and deleting everything you have installed with EasyBuild. Be careful!
Log in to the machine again, and you should be able to see the EasyBuild module:
$ module avail ------ /opt/software/easybuild/modules/all ------ EasyBuild/4.6.0
Load the module:
$ module load EasyBuild/4.6.0
You are now ready to run EasyBuild!
Running EasyBuild
Once EasyBuild is installed and the module is loaded, you can use it to install software.
Running EasyBuild should always be done as a non-privileged user (not root). If you want to install software in a non-default location, you (or your IT staff) can consider giving your ordinary user permission to write to this location.
Take a look at the list of available software, and see if you find something you like! As an example, let's install Julia!
Use EasyBuild to search:
$ eb --search-filename Julia (...) * Julia-1.7.2-linux-x86_64.eb * Julia-1.7.3-linux-x86_64.eb
We just pick the latest version (as of this writing):
$ eb Julia-1.7.3-linux-x86_64.eb --robot
This will install the requested software in your home directory. The --robot option instructs EasyBuild to build all dependencies as well, you'll usually want that.
Building software can be time-consuming, but Julia usually just takes a minute or so. Pay attention to the output from the build process.
To use your new software (assuming all went well), enable and load the module:
$ module use $HOME/.local/easybuild/modules/all
$ module avail
------ /uio/kant/geo-adm-u1/hpverne/.local/easybuild/modules/all ------
Julia/1.7.3-linux-x86_64
------ /opt/software/easybuild/modules/all ------
EasyBuild/4.6.0 (L)
$ module load Julia/1.7.3-linux-x86_64
$ julia --version
julia version 1.7.3
Install where?
It may be inconvenient to install the software in your home directory. Some EasyBuild installations can be huge, and perhaps you don't have the space for it.
We recommend you make notes on what commands you use to install software. That could come in handy should you later choose to install it in a different location.
Here are some alternative install locations:
System-wide install
If you have the right permissions, you can install in /opt/software/, along with the EasyBuild software itself. You do so by specifying the system-wide config:
$ eb --configfile=/opt/software/easybuild/config.cfg Julia-1.7.3-linux-x86_64.eb --robot
Before you do so, you might want to inspect the contents of that config, and do a dry-run (pretend) install:
$ eb --show-config --configfile=/opt/software/easybuild/config.cfg $ eb --configfile=/opt/software/easybuild/config.cfg Julia-1.7.3-linux-x86_64.eb --robot --dry-run
Then, update the module cache:
$ /opt/software/sbin/update_lmod_cache
It is a good practice to test your EasyBuild software installations in your home directory (or /scratch) before you install it system-wide.
Install in /scratch
On mimi, you can install in /scratch. Note that the contents in /scratch will be deleted now and then, so this should only be an ad-hoc install, for software you only need here and now, or for testing.
Create a subdirectory for this purpose, include your username. Set the variable EASYBUILD_PREFIX to point to this directory, and you're ready to roll!
$ mkdir /scratch/hpverne/eb $ export EASYBUILD_PREFIX=/scratch/hpverne/eb/ $ eb --show-config $ eb Julia-1.7.3-linux-x86_64.eb --robot
Enable the new module tree, similar to the previous example:
$ module use /scratch/hpverne/eb/modules/all/
If you have some other location you think can be used for EasyBuild installs, the procedure would be similar.
Troubleshooting
When building software, Easybuild will sometimes tell you it needs some system packages to be installed. For example:
$ eb WRF-4.3-foss-2021a-dmpar.eb --robot
(...)
ERROR: (...) One or more OS dependencies were not found: [('libibverbs-dev', 'libibverbs-devel', 'rdma-core-devel')]
If this happens, you'll need to install (some of) the packages mentioned. In this case, it's these two:
# yum install libibverbs-devel rdma-core-devel
Ask IT for assistance if needed.
Removing software
To remove something you have installed with easybuild, start by unloading the module:
$ module purge
Then, simply remove the installation directory and module file. For a homedir installation:
$ rm -r ~/.local/easybuild/software/Julia/1.7.3-linux-x86_64/ $ rm ~/.local/easybuild/modules/all/Julia/1.7.3-linux-x86_64.lua
If you only install in your home directory for test purposes, you can simply zap it all:
$ rm -r ~/.local/easybuild
If you're unsure where the installation directory is, module avail should give a hint.
References, further reading (possibly restricted):