Login

HPC Workflow @ENCCB

15–16 Feb 2022
Europe/Brussels timezone

Contribution List

9 out of 9 displayed
  1. 1. Welcome & Introduction
    Leplae Raphaël (ULB)
    15/02/2022, 08:00

    To start the workshop, this talk will first present the ecosystem and environment related to HPC in Belgium and the contexts of PRACE and EuroCC. It will then move on to a short introduction on Workflows in HPC and a description of the different seminars to come in this workshop.

    Go to contribution page
  2. 2. Workflows with basic GNU tools (and Maestro)
    François Damien (UCLouvain)
    15/02/2022, 08:30

    This presentation will present the two basic building blocks of workflows that are the job arrays and job dependencies. Job arrays allow creating parametrised jobs that all look identical except for one parameter that varies through the workflow, while job dependencies enable a fixed ordering of jobs and make sure the steps of the workflows are carried on only when their requirements (input...

    Go to contribution page
  3. 3. Checkpoint/Restart
    Mattelaer Olivier (UCLouvain)
    15/02/2022, 10:30

    This session will discuss one specific type of workflows that is checkpoint/restart and how Linux signals can be leveraged to build self-resubmitting jobs that can run longer than the maximum wall time of the cluster.

    Go to contribution page
  4. 4. atools
    Bex Geert Jan (Uhasselt – KULeuven)
    15/02/2022, 13:00

    This presentation will present a collection of tools named atools that help building and managing large job arrays for parametrised studies. Such workflows can be referred to as "wide" workflows: many similar jobs siblings one to another, with no dependency among them.

    Go to contribution page
  5. 5. Makeflow
    François Damien (UCLouvain)
    15/02/2022, 14:30

    This session will discuss Makeflow, a tool that can be used to model workflows with many dependencies among jobs. Such workflows can be referred to as "deep" workflows by contrast with the "wide workflows" described earlier.

    Go to contribution page
  6. 7. CI/CD implementation with gitHub
    Mattelaer Olivier (UCLouvain)
    16/02/2022, 08:00

    This session will be about GitHub and its continuous integration/continuous deployment (CI/CD) features and how it can be used on clusters with a regular user to automatically compile software and even submit benchmark jobs whenever new features or improvements are added to the software you are writing.

    Go to contribution page
  7. 6. Singularity
    Mattelaer Olivier (UCLouvain)
    16/02/2022, 09:30

    This presentation will be about Singularity and how to build containers and deploy them on clusters so as to install software in a uniform way, not being stopped by the Linux flavour or available software modules.

    Go to contribution page
  8. 8. Snakemake
    Louant Orian (ULiège)
    16/02/2022, 13:00

    We are back to scientific workflows and the seventh presentation will be a tutorial on SnakeMake, a tool that is a bit more complex to use than the other two but that can handle both wide and deep workflows, and can do more things like templating, containers, etc.

    Go to contribution page
  9. 9. User testimonials
    16/02/2022, 14:30

    15h30-15h50: NextFlow for bioinformatics by Luc Cornet
    15h50-16h10: Fireworks for material science by Guillaume Brunin
    16h10-16h30: Coral, a home made workflow system tool to manage numerical climate simulations by François Klein
    16h30-16h50: Examples of CI/CD in research codebases using git-based websites and HPC by Denis-Gabriel Caprace

    Go to contribution page