Learning how to use HPC infrastructure (part III: Submitting jobs, using containers, checkpointing, workflows)

HPC program portability

19 Oct 2023, 12:15

30m

Maxwell/Shannon (first floor) (Louvain-La-Neuve)

Speaker

Dr Olivier Mattelaer (UCLouvain/CISM)

Description

Singularity is a container solution for HPC. Containers help with reproducibility as they nicely package software and data dependencies, along with libraries that are needed. It allows users to install and run software that required root access to be installed on clusters where they only have regular user permissions. The rationale is to perform all the software installation in a container image (a kind of lightweight virtual machine, that can use a different Linux distribution than the one on the compute nodes!) on a machine where you have root access and then transfer and run that image on the machine on which you do not have root access. Images can be built from recipes shared by others, and from recipes made for Docker, the leader container solution outside the HPC world.
Contents: Container concepts and benefits Starting a Singularity container on the cluster Accessing the cluster filesystems Building a container image from a recipe Building a container image from scratch Singularity hub	Prerequisite: Being able to use SSH with private keys  Being familiar with a text editor  Mastering the Linux command line and the GNU utilities (mkdir, cp, scp, etc.) Basic knowledge of Linux system administration
Type: Hands-on Target audience: Advanced user Must: This session is a must-have for anyone dealing with software that only installs on Ubuntu...

Presentation materials

23_10_25_Singularity_CISM.pdf