HPSF Conference 2025

GlideinWMS is a pilot and pressure-based workload manager for distributed scientific computing. Many experiments like CMS and Fermilab’s Neutrino experiments use it to provision elastic clusters for their analysis and simulations, split in close to a million concurrent jobs. The majority of the user jobs require containers and the pilots use Apptainer to setup the desired platform. For the pilots, that run as regular batch jobs, Apptainer is safer, lighter and easier to use than other containerization solutions. Many images used by the pilots are expanded SIF images distributed via the CernVMFS: this combination is very efficient.

Managing software on supercomputers is challenging due to diverse architectures, dependency complexities, and security constraints. Traditional tools like Spack help with software compilation, but they lack portability and require system-specific configurations. Apptainer (formerly Singularity) solves these issues by enabling secure, portable, and reproducible containerized environments tailored for HPC. Unlike Docker, Apptainer runs without root access, ensuring security and compliance in multi-user clusters. It supports MPI, GPUs, and Slurm, allowing high-performance workloads to run seamlessly across systems. By integrating Apptainer into NSM supercomputers, researchers benefit from faster deployments, reproducible results, and reduced administrative overhead. This session explores Apptainer’s role in transforming HPC workflows, its advantages over traditional software management, and best practices for adopting containers in large-scale computing environments.

This session delves into the advanced utilization of Apptainer (formerly Singularity) for distributed computing in high-performance computing (HPC) environments. It will cover containerized execution methodologies, security enhancements, and scalability optimizations critical for scientific computing. The discussion will emphasize MPI integration, multi-node orchestration, and performance considerations for HPC workloads within Apptainer containers. Researchers and scientists will gain a comprehensive understanding of deploying, managing, and optimizing containerized scientific applications across distributed HPC infrastructures.

The lead developer of Apptainer will discuss recent changes to Apptainer.