This post is written by Christian Kniep, Senior Developer Advocate for HPC

Containers are rapidly maturing within the high performance computing (HPC) community and we’re excited to be part of the movement: listening to what customers have to say and feeding this back to both the community and our own product and service teams. Containerization has the potential to unblock HPC environments, so AWS ParallelCluster and container-native schedulers like AWS Batch are moving quickly to reflect the best practices developed by the community and our customers.

This year is the seventh consecutive year we are hosting the ‘High Performance Container Workshop’ at ISC High Performance 2021 conference (ISC’21). The workshop will be taking place on July 2nd at 2PM CEST (7AM CST).

About the workshop series

The series focuses on how containerization improves the state-of-the-art in HPC. Previous workshops aimed to dissect the challenges with containers in HPC into digestible slices to make sure that the discussion was focused. For example, last year’s workshop at ISC’20 was a deep dive into each aspect of running containers for HPC (the runtime, how to build containers, how to orchestrate or schedule, etc.) and the recordings are still online. You’ll quickly get a feel for the variety of topics after watching these sessions.

New this year

We will be dedicating this year’s installment to five specific use-cases. We will keep the segmentation by aspect (runtime, build/distribute, scheduling, HPC-specific), but with only a short intro/recap. After the presentation the workshop speakers will talk about best practices and pitfalls and spark a discussion for each use-case.

HPC containers use-cases

We assembled a diverse set of use-cases, which are represented by brave individuals. Our goal is to keep the focus of each use-case small to reduce the cognitive load for everyone involved. Below are details about each use-case and questions presenters plan to answer in their session.

1. Application Power user (Carsten Kutzner, Max Planck Institute, Göttingen)

Carsten is an HPC application power user. How can we provide a sane environment to researchers within a group? How to choose the right image? How to build images? How to be flexible when it comes to the underlying system (workstations, department clusters and supercomputers)?

2. Education Use Case (Michael Kuhn, Otto von Guericke University Magdeburg)

Michael represents a university setup for teaching students (undergrads through post-docs) how to approach HPC applications. Development of I/O benchmarks that don’t need a massive amount of setup and are able to leverage laptops/workstations, local clusters of the department (with control over the system configuration) in addition to compute clusters without control of what is installed.

3. HPC site SysOps (David Brayford, Leibniz Supercomputing Center – LRZ)

David represents system administrators from large HPC sites (in his case LRZ). Their focus is system configuration, and one of their goals is to allow users to run their code containerized. This also includes novices, who may only have limited background knowledge. How should we build containers for users? How should we build with them? How should we onboard containers from other sites?

4. Bootstrap HPC (Umesh Upadhyaya, HPC Nepal)

This use-case portrays a new and upcoming HPC chapter in Nepal to support local HPC usage. Umesh’s focus is on ease of use without the baggage of ‘traditional’ HPC. Applications must run on laptops, and move to clusters when they’re accessible.

5. Scientist with a complex software stack (Abdulrahman Azab, University of Oslo)

This use-case speaks to how we can use containers with complex software stacks and workflows. Workflows in which a user might have a script using multiple different stacks. Abdulrahman will touch on how ‘module load’ is used in a world without containers and how we cope with containerizing workflows.

We will also discuss security guidance for end users so that attendees are aware of security best practices for each use-case.

Conclusion

If you are interested in containers in HPC, you should register for the ISC’21 conference and attend our workshop on July 2^nd. You can view the full ISC’21 program and register at the ISC’21 website. The full program for the workshop is available on the High Performance Container Workshop page.

To prepare, we encourage you to check out last years’ workshop to familiarize yourself with the aspects we are going to discuss. The agenda and recordings are all online.

Finally, to gain some hands-on experience with containers, you can follow the Container Runtimes/Engines on AWS ParallelCluster workshop. In this workshop you will learn about and implement workloads on a selection of HPC container runtimes in the context of a traditional HPC environment with Slurm as the HPC scheduler.

You can find more AWS HPC sessions at ISC’21 here. We look forward to seeing you at ISC’21!