4th EasyBuild User Meeting at Louvain-la-Neuve

Europe/Brussels
Shannon room (Maxwell building)

Shannon room

Maxwell building

Place du Levant 3 1348 Louvain-la-Neuve Belgium
Kenneth Hoste (Genth University)
Description

The EasyBuild  (https://easybuilders.github.io/easybuild/) User Meeting is an open and highly interactive event that provides a great opportunity to meet fellow EasyBuild enthusiasts, discuss related topics and learn about new aspects of the tool.

It is intended for people that are already familiar with EasyBuild, ranging from occasional users to Easybuild core developers and experts, and topics will be less introductory in nature than during EasyBuild hackathons/workshops that have been organised in the past.

The program includes presentations by both EasyBuild users and developers, next to hands-on sessions.

The presentations will cover a wide variety of EasyBuild related topics, including how EasyBuild has been integrated into existing infrastructures, how it can be combined with other tools, (new) EasyBuild features that are considered particularly useful, or other tools that are somehow (loosely) related to EasyBuild and could be of interest to attendees.

The hands-on sessions are intended for becoming more familiar with the practical aspects of EasyBuild, resolving problems with the help of others, and/or working together to further enhance EasyBuild. Participants are expected to decide for themselves what they work on during the hands-on sessions.

Note that registration is free BUT mandatory.

    • 09:30 10:00
      coffee/Welcome 30m Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 10:00 12:00
      plenary talk Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
      • 10:00
        Welcome 15m

        President of the Institute of Condensed Matter and Nanosciences

        Speaker: Prof. Bernard Nysten (IMCN)
      • 10:15
        Easybuild State of the union 1h 45m

        Report on the survey
        Status of easybuild
        Future of easybuild

        Speaker: Kenneth Hoste (Genth University)
    • 13:00 15:00
      Site report Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium

      Site report

      • 13:00
        Deploying Easybuild on a multi-site HPC infrastructure 20m
        Speaker: Frédéric Wautelet (University of Namur)
      • 13:25
        CSCS site update 20m

        Overview of scientific software management at CSCS, including EasyBuild setup, recipe review, automated deployment and Continuous Integration with Jenkins pipelines + Github

        Speaker: Luca Marsella (ETH Zurich / CSCS)
      • 13:50
        EasyBuild on IT4Innovations 20m

        We would like to present our site, the simple way we use the EasyBuild and our own extensions. It can be an inspiration for starting EasyBuild users.

        Speaker: Josef Hrabal (IT4Innovations, VŠB-TU Ostrava)
      • 14:30
        site report Fred Hutch + easy_update future 30m

        Hutch

        Fred Hutch Cancer Research Center has about 230 scientific faculties and
        just over 3,000 staff. Located in Seattle, Washington on a 15 Acre
        Campus (60,702 M^2). We have an HPC group of 6 people and support 6
        thousand core cluster on campus.

        R

        The Fred Hutch has hundreds of R users. Robert Gentleman who is one of
        the co-developers of R worked at the Hutch while still actively
        developing the language. Gentleman now works at 23andMe. Bioconductor
        originated at the Fred Hutchinson Cancer Research Center in the Fall of
        2001. Martin Morgan led Bioconductor development since 2008 joined the
        Fred Hutch in 2005. The BioConductor group has since moved to Roswell
        Park Cancer Center. R is the predominant language used by BioInformatics
        staff at the Hutch. Our local R build has over 700 modules. Our HPC
        group (me) has to devote considerable effort to maintain R for the center.

        R Strategy

        Our users want the latest version of R built as quickly as possible.
        Using the foss-2018b toolchain, we will deploy the standard EB release
        of R with a day of its Release. The Fred Hutch release of R will be
        built as Bundle using the community release as a base. The Fred Hutch R
        build has over 800 modules. Our local version of R has a version suffix
        of "-fhX."

        Easy_Update and Easy_Annotate

        I am the author of easy_update. Easy_update is used to update package
        versions for R and Python easyconfig recipes. Version information is
        obtained from the respected package authority CRAN, PyPi, and
        BioConductor. Easy_update resolves package dependencies.
        Easy_Annotate documents our R and Python builds at the package level.
        Fred Hutch uses GitHub pages to document our Easybuild scientific
        software: https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.github.io_easybuild-2Dlife-2Dsciences_&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc&m=aKY87YplHcgIjHlZTGEyDTp23LoyK5aKhezrP87D0Y0&s=HDoEsgVW8-k4qfXW0pL4xEB3uL0wj73fJ6kVKN8rOho&e=
        I plan to add the easy_update features to easybuild. Easy_update has
        been under much change due to the move to PyPi and needing to support
        Bioconductor packages for R., And I keep finding horrible bugs which I
        would not want to in EasyBuild.

        Future EB Plans

        • We would like to benefit from foss-2018b library consistency
        • We want to contribute packages to the community
        • Have implemented a Docker build environment to ensure more consistent
          and repeatable packages. GitHub: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.github.com_FredHutch_ls2&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc&m=aKY87YplHcgIjHlZTGEyDTp23LoyK5aKhezrP87D0Y0&s=skrWOrz8DuoJ6VARpFoTtO1WH6KERpyuPQ0cHe_3Kvc&e=
        • Committed to foss-2018b toolchain for the next few years. Our site is
          planning to upgrade our cluster to Ubuntu 18.04.
        • We meet people who are using EasyBuild but are not contributing back
          to the community. We want to encourage more sites globally to
          participate in making Scientific Software more accessible.
        Speaker: Mr John Dey (Fred Hutchinson Cancer Research Center)
    • 15:00 15:30
      coffee 30m Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 15:30 17:30
      EasyBuild hands on Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 09:00 10:30
      plenary talk Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
      • 09:00
        Collective Knowledge framework to automate, crowdsource, reproduce and share HPC experiments 30m

        Validating experimental results from articles has finally become a norm at many HPC and systems conferences. Nowadays, more than half of accepted papers pass artifact evaluation and share related code and data. Unfortunately, lack of a common experimental framework, common research methodology and common formats places an increasing burden on evaluators to validate a growing number of ad-hoc artifacts. Furthermore, having too many ad-hoc artifacts and Docker snapshots is almost as bad as not having any (!), since they cannot be easily reused, customized and built upon.

        While overviewing more than 100 papers during artifact evaluation at PPoPP, CGO, PACT and other conferences, I noticed that many of them use similar experimental setups, benchmarks, models, data sets, environments and platforms. This motivated me to develop Collective Knowledge (CK), an open workflow framework with a unified Python API to automate common researchers’ tasks such as detecting software and hardware dependencies, installing missing packages, downloading data sets and models, compiling and running programs, performing autotuning and co-design, crowdsourcing time-consuming experiments across computing resources provided by volunteers similar to SETI@home, applying statistical analysis and machine learning, validating results and plotting them on a common scoreboard for open and fair comparison, automatically generating interactive articles, and so on: http://cKnowledge.org.

        In this talk I will introduce CK concepts and present several real world use cases from General Motors, Amazon and Arm on collaborative benchmarking, autotuning and co-design of efficient software/hardware stacks for deep learning. I will also present results and reusable CK components from the 1st ACM ReQuEST optimization tournament: http://cKnowledge.org/request. Finally, I will introduce our latest initiative to create an open repository of reusable research components and workflows, and conclude with an open discussion on how to possibly connect it with the EasyBuild package manager.

        Speaker: Dr Grigori Fursin (dividiti; cTuning foundation)
      • 09:30
        OpenHPC Introduction 30m

        High performance computing (HPC) - the aggregation of computers into clusters to increase computing speed and power- relies heavily on the software that connects and manages the various nodes in the cluster. Linux is the dominant HPC operating system, and many HPC sites expand upon the operating system's capabilities with different scientific applications, libraries, and other tools.

        To avoid duplication of the necessary steps to run an HPC site the OpenHPC project was created in response to these issues. OpenHPC is a collaborative, community-based effort under the auspices of the Linux Foundation to solve common tasks in HPC environments by providing documentation and building blocks that can be combined by HPC sites according to their needs.

        This talk gives an introduction in OpenHPC and how it tries to help to set up HPC systems.

        Speaker: Dr Adrian Reber (Red Hat)
      • 10:00
        ReFrame: A Regression Testing and Continuous Integration Framework for HPC systems 30m

        Regression testing of HPC systems is of crucial importance when it comes to ensure the quality of service offered to the end users. At the same time, it poses a great challenge to the systems and application engineers to continuously maintain regression tests that cover as many aspects as possible of the user experience. In this presentation, we present ReFrame, a new framework for writing regression tests for HPC systems. ReFrame is designed to abstract away the complexity of the interactions with the system and separate the logic of a regression test from the low-level details, which pertain to the system configuration and setup. Regression tests in ReFrame are simple Python classes that specify the basic parameters of the test plus any additional logic. The framework will load the test and send it down a wel-defined pipeline which will take care of its execution. All the system interaction details, such as programming environment switching, compilation, job submission, job status query, sanity checking and performance assessment, are performed by the different pipeline stages. Thanks to its high-level abstractions and modular design, ReFrame can also serve as a tool for continuous integration (CI) of scientific software, complementary to other well-known CI solutions. Finally, we present the use cases of two large HPC centers that have adopted or are now adopting ReFrame for regression testing of their computing facilities.

        Speaker: Dr Victor Holanda Rusu (ETHZ/CSCS)
    • 10:30 11:00
      coffee 30m Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 11:00 12:30
      Site report Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium

      Site report

      • 11:00
        Group picture 20m
      • 11:20
        Deploying Electronic-Structure Codes with EasyBuild 20m

        EasyBuild is very helpful to researchers using electronic-structure codes by progressively replacing a lot of quick-and-dirty installation scripts and hastily set-up directory tree structures by a well-organised and systematic deployment frameowrk for HPC clusters.

        One of the major challenges this field is currently facing is the so-called "dependency hell" arising from an increasing and necessary modularization of the numerous software components developed within larger and larger collaborative efforts. The geographical and cultural dispersion of the various developer teams is another one.

        Through an overview of EasyBuild-related efforts undertaken for ABINIT and SIESTA since 2017, as well as the provision of various EasyConfigs in 2019 for the Electronic Structure Library, we will describe what the different stakeholders are experiencing within these endeavours and what could be done to further support them.

        Speaker: Dr Yann Pouillon (Universidad de Cantabria)
    • 13:00 13:45
      plenary talk Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 13:45 14:30
      EasyBuild hands on Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 14:30 15:00
      coffee 30m Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 15:00 17:15
      plenary talk Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
      • 15:00
        Cloud and local software deployment using CVMFS, EasyBuild and Lmod 40m
        Speakers: Dr Davide Vanzo (ACCRE - Vanderbilt University), Dr Eric Appelt (ACCRE - Vanderbilt University)
      • 16:00
        Lmod/XALT update (remote) 45m
        Speaker: Robert McLay (TACC)
      • 16:45
        Python at Compute Canada 20m

        I will explain our Python setup: we compile it using the "dummy " toolchain with a very limited set of packages. We ask users to use virtual environments for anything beyond the Scientific Python stack. We also developed and are contributing a method for a single module to work with multiple versions of Python instead of being bound to a specific version. See also https://docs.computecanada.ca/wiki/Python

        Speaker: Dr Bart E. Oldeman (McGill HPC Centre)
    • 17:15 18:30
      EasyBuild hands on Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
      • 17:15
        visit data center 30m
    • 18:30 21:30
      conference dinner 3h Brasse-temps

      Brasse-temps

      4 Place des Brabançons Louvain-la-Neuve, Nivelles, 1348 Brabant Wallon
    • 09:00 10:30
      plenary talk Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
      • 09:00
        Modules: automatic and consistent handling of modulefile dependencies 30m

        Capabilities to consistently handle dependencies between modulefiles have been improved recently on Modules. This talk will introduce the new automated module handling mechanisms and will also shed lights on further ongoing work on that field.

        These features offer new perspectives for build and installation framework like EasyBuild when it comes to provide a way to consistently access the numerous software installed.

        Some ideas will be shared with the audience on how to design modulefiles to leverage the automated module handling mechanisms in a multi build-toolchain context.

        Speaker: Mr Xavier DELARUELLE (CEA)
      • 09:30
        Ansible module for easybuild 30m
        Speaker: Dr Olivier Mattelaer (UCLouvain/CISM)
      • 10:00
        Finding a home for EasyBuild in EuroHPC 30m

        The European High-Perfomance Computing Joint Undertaking (EuroHPC JU) will pool European resources to develop exascale supercomputers for European researchers and industry. The amount of funding involved is very large (in the billions) and there may well be some opportunities to fund development of EasyBuild through some of the funding calls that are expected to emerge in 2019. I would like to look at some of the scope indicated by the EU to date and also discuss this in the context of HPC Centres of Excellence.

        Speaker: Alan O'Cais
    • 10:30 11:00
      coffee 30m Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 11:00 12:00
      EasyBuild hands on Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
      • 11:00
        EasyBuild hands on 1h
    • 13:00 13:45
      plenary talk Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 13:45 15:00
      EasyBuild hands on Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
      • 13:45
        EasyBuild hands on 1h 15m
    • 15:00 15:30
      coffee 30m Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
    • 15:30 17:30
      EasyBuild hands on Shannon room

      Shannon room

      Maxwell building

      Place du Levant 3 1348 Louvain-la-Neuve Belgium
Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×