Going Parallel (part I: linux tools, advanced Slurm, MPI programming)

Tuesday 21 Nov 2023, 09:30 → 17:30 Europe/Brussels

Shanon (building Maxwell, first floor) (Louvain-La-Neuve)

Going parallel is crucial to speed up your gain and to use the full power of HPC center. In this session, we will start by an introduction to teach you various strategy/trick of parallelism and then deep dive into MPI, the method to share computation between different computer with communication between the various computer to organise the computation/share results,....
Contents: Introduction to parallel computing Parallel programming with MPI advanced SLURM Must: This session is a must-have for anyone writing or even simply using MPI programs Type: Hands-on	Prerequisite: Being able to use SSH with private keys  Being familiar with a text editor  Mastering the Linux command line and the GNU utilities (mkdir, cp, scp, etc.) Passive knowledge of C or Fortran Target audience: Rookie programmer

1 Introduction to parallel computing

Before diving into the concrete programming examples with MPI and OpenMP, this session introduces some theoretical concepts and presents the several paradigms and tools offered by Linux for parallel computing when a program itself is not able to run in parallel.
Contents: Theoretical concepts: parallelism, speedup, scaling, overhead, etc. Common parallel computing paradigms (SPMD, Map/Reduce, etc.) GNU tools for parallel computing (xargs and GNU parallel) Parallel computing with pipelines (UNIX pipes and FIFO files)	Prerequisite: Being able to use SSH with private keys  Being familiar with a text editor  Mastering the Linux command line and the GNU utilities (mkdir, cp, scp, etc.)
Type: Hands-on Target audience: Everyone Must: This session is a nice-to-have.

Speaker: Damien François (UCLouvain/CISM)

parcomp2023.pdf

2 Advanced SLURM job submission

This session follows the first Slurm session and teaches participants how to enhance submission scripts using concepts learned in the Workflows, Parallel computing, Bash scripting, and MPI sessions.

 

Contents: Scripting submissions  Arrays and Job packing  Heterogeneous jobs  Process placement	Prerequisite: Being able to use SSH with private keys  Being familiar with a text editor  Mastering the Linux command line and Bash scripting Working knowledge of Slurm
Type: Hands-on Target audience: Everyone Must: This session is useful.

Speaker: Damien François (UCLouvain/CISM)

slurm2023-2debug.pdf

slurm2023-2.pdf

3 Parallel programming with MPI (1)

MPI is a standard for passing messages between processes running on distinct computers. It offers high-level primitives for efficient communication.
Contents: Parallel programming paradigms Shared-memory vs message passing Compiling an MPI program Collective communication Reduction operations Communication modes	Prerequisite: Being able to use SSH with private keys  Being familiar with a text editor  Mastering the Linux command line and the GNU utilities (mkdir, cp, scp, etc.) Passive knowledge of C or Fortran
Type: Hands-on Target audience: Rookie programmer Must: This session is a must-have for anyone writing or even simply using MPI programs

Speaker: Orian Louant (Université de Liège)

CECI-MPI-1-Nov23.pdf

mpi-examples.zip