Benchmarking Parallel Performance of Numerical MPI Packages
Description of the project: This project aims to set up parallel/HPC performance testing for our MPI numerical packages. The project has two goals. Firstly, to provide benchmarking of debian MPI packages such as fenicsx (finite element computation) to demonstrate scalability of the software in cloud computing environments. Secondly, to provide a type of CI testing ensuring that MPI scalability is maintained as MPI- or computation-related packages such as OpenMPI, PETSc, OpenBLAS are upgraded. We aim to set up benchmarking tests to run regularly and report over varying test conditions, for instance comparing performance with different BLAS implementations. It would be useful to be able to quantify how well our HPC packages actually scale (in cloud computing environments) and monitor if there is any drop in performance (e.g. with version updates). It will also be useful to report their performance with the various BLAS alternatives.
Some packages have benchmarks already at hand. The FEniCS project for instance offers fenicsx-performance-tests (both prebuilt and source). The outcome of the project will be the preparation of a protocol for setting up and launching MPI CI testing on an appropriate cloud computing service, identifying which parameters (test size etc) must be used to get meaningful numbers. A suggested tool for managing test parameters and results might be https://reframe-hpc.readthedocs.io/en/stable/ The report format may be similar to https://fenics.github.io/performance-test-results/ , with a web-based UI (i.e. buttons) to select different reports.
Confirmed Mentor: Drew Parsons
How to contact the mentor: dparsons@debian.org
Confirmed co-mentor: Francesco Ballarin <francesco.ballarin@unicatt.it>
Difficulty level: Medium
Project size: Medium sizes (175 hours) with the possibility to split into two small (90 hour) subprojects.
Deliverables of the project:
- Setting up benchmark conditions
- Protocol for launching a cloud computing instance and running MPI benchmarks. b. Determining benchmark parameters (job size) suitable for generating meaningful performance comparisons. c. Configuration of a test management system (e.g. reframe).
- Implementing and Reporting Benchmarks
- Implementation of the 1.a. protocol to commence regular benchmarking on a suitable cloud computing system. b. Setting up data management for test results. c. Setting up web-based reporting of test results
- Setting up benchmark conditions
Desirable skills: Skills that the student has or is willing to develop.
- unix shell commands and debian package installation
- launching parallel MPI calculations
What the intern will learn: The intern will learn how to characterise performance of parallelised computations (speedup, strong and weak scalability) and gain experience configuring and maintain cloud computations.
Application tasks:
- How is a parallel job launched using MPI?
- Summarise the difference between strong and weak scaling.
- Which MPI implementations does Debian provide? Which one is default?
- Which BLAS implementations does Debian provide? How is the preferred BLAS implementation configured on a given installation?
- Prepare a bash shell script that installs the packages required for running autopkgtest on fenicsx-performance-tests, then loops through each available BLAS implementation, installing each blas package one by one and running autopkgtest on fenicsx-performance-tests for each BLAS implementation.
Related projects:
Benchmarking for the FEniCS Project
Grid5000 academic High Performance Computing network in France