Modules

In Metacentrum infrastructure all applications are packed into modules, which make sure that a given application will run within a correct environment (libraries, system variables etc.). Therefore to run a certain software, you must first load a module for this software.

To work with modules, you will need to use command module (with options).

A complete guide to module command can be found on https://modules.readthedocs.io/en/latest/modulefile.html. You can also browse the manpage (man module).

On this page we will show only the most common usecases of the module command.

Basic usecases

Search for modules

List all modules

$ module avail  # list all available modules (but not their versions)
                # the list is very long; move down the list by pressing Space

List all modules starting by a certain string

$ module avail intel    # list all available modules starting with "intel"
intel-parallel-studio/  intel/  intelcdk/  intelmpi/

List versions of a specific module

$ module avail intelcdk/   # list the modules within the intelcdk/ directory (the "/" is important!)

will get list of versions:

intelcdk/12  intelcdk/13  intelcdk/14  intelcdk/15  intelcdk/16  intelcdk/17  intelcdk/17.1  intelcdk/17.1-cvmfs  intelcdk/19u3

You can modify the search behaviour by e.g. options --indepth or -a. For all options, see man module.

$ module avail intelcdk --indepth # list all the modules within the intelcdk directory
$ module avail intelcdk -a        # similar to above, plus list aliases to modules

Load modules

Load a default version of a module

$ module load intelcdk           # load default version of intelcdk
$ module load intelcdk --default # dtto
$ module add intelcdk            # dtto

There is always a default version of a module (usually the newest, but can be also the most stable). If you have no specific reason to do otherwise, use the default version.

Load a specific version of a module

    $ module load intelcdk/17.1

Unload modules

Unload a module

$ module unload matlab/9.9   # unload matlab-9.9
$ module del matlab/9.9      # dtto
$ module rm  matlab/9.9      # dtto

Clear all currently loaded modules

$ module purge   # get rid of everything!
$ module purge && module add metabase/1 # see below

Warning

The module purge command will remove also an automatically loaded module metabase/1. This module makes accessible utils in /software/meta-utils/public, like qextend, clean_scratch, walltime_prolong_user_info, check-local-quota and other commands. For this reason, we recommend to add module metabase/1 after using module purge command.

Display info

List currently loaded modules

$ module list
Currently Loaded Modulefiles:
 1) metabase/1   2) intelcdk/19u3

Display information about the module

$ module display intelcdk/17.1

Some important variables are

PATH - path to the executable
LD_LIBRARY_PATH - path to libraries for linker
LIBRARY_PATH - path to libraries

Specific usecases

Limit module span

Applications available through the module system automatically load all necessary dependencies, i.e. other modules. Loading many modules together may thus raise a conflict of loaded dependencies and the subsequent crash of some of the programs. When the user needs to use more modules in one job, it is advisable to limit the use of individual modules to only the necessary part of the script.

Within a single session you can use modules separately by enclosing them to parentheses. Parentheses, in general, create a subshell in bash, i.e. all modules loaded within a certain subshell will be unloaded automatically after the subshell exits.

In general

(module add module_1
command_1 ... <input> <output>
)
    
# Here, after the right parenthesis, module module_1 is no longer active (including its dependencies)
    
(module add module_2
command_2 ... <input> <output>
)

Example: conflicting python versions

Normally the following two modules will conflict if loaded within the same session:

(BUSTER)user123@skirit:~$ module add python/3.8.0-gcc
This is the user wrapper module using spack module python/3.8.0-gcc-rab6t.
(BUSTER)user123@skirit:~$ module add python/3.8.0-gcc-rab6t
python/3.8.0-gcc-rab6t(49):ERROR:150: Module 'python/3.8.0-gcc-rab6t' conflicts with the currently loaded module(s) 'python/3.8.0-gcc'
python/3.8.0-gcc-rab6t(49):ERROR:102: Tcl command execution failed: conflict python

However you can load them separately in subshells:

(BUSTER)user123@skirit:~$ (module add python/3.8.0-gcc; python)
This is the user wrapper module using spack module python/3.8.0-gcc-rab6t.
...
>>> quit()
(BUSTER)user123@skirit:~$ (module add python/3.8.0-gcc-rab6t; python) 
...
>>> quit()

Modulefiles

A modulefile is a text file that collects paths to the binary, paths to external libraries and possible dependencies on other modules.

The modulefiles reside in some of the directories included in variable MODULEPATH (echo $MODULEPATH).

For many applications, there are several modules available. This means there are several copies of the software available, differing in version, compiler and other features. For example,

$ module avail tensorflow/
tensorflow/1.3.0-cpu          tensorflow/1.3.0-gpu-python3  tensorflow/1.5.0-gpu          tensorflow/1.13.1-gpu-python3  
tensorflow/1.3.0-cpu-python3  tensorflow/1.4.1-gpu          tensorflow/1.7.1-cpu-python3  tensorflow/2.0.0-gpu-python3   
tensorflow/1.3.0-gpu          tensorflow/1.5.0-cpu-python3  tensorflow/1.7.1-gpu-python3

says there are 11 versions of TensorFlow software differing by version, Python version (2 or 3) and whether it can be used for GPU computing (gpu) or not (cpu).

Modulefile example

#%Module1.0
#!
#! Title: gsl-1.16
#! Platforms: amd64_linux32
#! Version: 1.16
#! Description: GNU Scientific Library tools collection
#!
#! Author: Petr Hanousek, #35289 
#!
#!
proc ModulesHelp {} {
global ModulesCurrentModulefile
puts stdout "modulehelp $ModulesCurrentModulefile"
}
    
set basedir /software/gsl/1.16/gcc
    
prepend-path PATH ${basedir}/bin 
prepend-path PKG_CONFIG_PATH ${basedir}/lib/pkgconfig
prepend-path LD_LIBRARY_PATH ${basedir}/lib
prepend-path MANPATH ${basedir}/share/man

Notes:

the number after the author’s name you can see in some modulefiles is the number of RT ticket which requested the installation
the only necessary path is PATH, where the executable is
a modulefile can add other modules (module add …) if there is a dependency
basedir is path to the install directory - it can be pretty anywhere; in case of non-standard installation, e.g. to home directory, don’t forget to set variable MODULEPATH accordingly.