HPSF Conference 2025

GlideinWMS is a pilot and pressure-based workload manager for distributed scientific computing. Many experiments like CMS and Fermilab’s Neutrino experiments use it to provision elastic clusters for their analysis and simulations, split in close to a million concurrent jobs. The majority of the user jobs require containers and the pilots use Apptainer to setup the desired platform. For the pilots, that run as regular batch jobs, Apptainer is safer, lighter and easier to use than other containerization solutions. Many images used by the pilots are expanded SIF images distributed via the CernVMFS: this combination is very efficient.

Managing software on supercomputers is challenging due to diverse architectures, dependency complexities, and security constraints. Traditional tools like Spack help with software compilation, but they lack portability and require system-specific configurations. Apptainer (formerly Singularity) solves these issues by enabling secure, portable, and reproducible containerized environments tailored for HPC. Unlike Docker, Apptainer runs without root access, ensuring security and compliance in multi-user clusters. It supports MPI, GPUs, and Slurm, allowing high-performance workloads to run seamlessly across systems. By integrating Apptainer into NSM supercomputers, researchers benefit from faster deployments, reproducible results, and reduced administrative overhead. This session explores Apptainer’s role in transforming HPC workflows, its advantages over traditional software management, and best practices for adopting containers in large-scale computing environments.

This session delves into the advanced utilization of Apptainer (formerly Singularity) for distributed computing in high-performance computing (HPC) environments. It will cover containerized execution methodologies, security enhancements, and scalability optimizations critical for scientific computing. The discussion will emphasize MPI integration, multi-node orchestration, and performance considerations for HPC workloads within Apptainer containers. Researchers and scientists will gain a comprehensive understanding of deploying, managing, and optimizing containerized scientific applications across distributed HPC infrastructures.

The lead developer of Apptainer will discuss recent changes to Apptainer.

In this presentation, we will offer a concise introduction and overview of the Trilinos project. Established in 2001, Trilinos is a collaborative initiative focused on developing algorithms and technologies to address complex multi-physics engineering and scientific challenges on high-performance computing (HPC) architectures. This includes a range of capabilities such as linear and nonlinear solvers, optimization techniques, and sensitivity analysis. A key mission of Trilinos is to cultivate a vibrant community of developers and users who contribute to a modular software framework comprised of diverse computational packages. This collaborative environment facilitates resource sharing among library developers, enhancing software compatibility and overall efficiency amortization of costs.

The Trilinos core area includes packages that form the basic building blocks for Trilinos capabilities. This talk
will briefly introduce each package and demonstrate how these can be pulled together into an application. We will
cover the following packages. The Kokkos performance portability library provides data structures and parallel
algorithms that are performant on both CPU and GPU architectures. The Kokkos-Kernels library provides
performance portable BLAS and LAPACK routines. The Tpetra library provides MPI-based parallel distributed linear
algebra data structures built on Kokkos. The Teuchos library provides utilities for a common look and feel
across Trilinos packages, including a reference counted smart pointer, print utilities, timers and a parameter list
for user input. The Zoltan library provides parallel distributed load balancing tools. The PyTrilinos package
provides python wrappers for Trilinos capabilities. The Thyra package provides abstraction layers for writing
linear algebra, linear solvers and nonlinear solvers. The talk will conclude with examples of Trilinos capabilities
integrated into the Empire electromagnetic particle-in-cell code.

Trilinos provides a variety of high-performance sparse iterative and direct linear solvers that are portable across CPU and GPU architectures. In this talk, I will provide motivation for why scientists and engineers developing a high-performance application should consider using Trilinos solvers. I'll give an overview of current solver capabilities across a spectrum of science applications, and discuss ongoing research as well as future directions.

In this presentation, we introduce the discretization and analysis tools available in the Trilinos software suite. Using real-world scientific applications as exemplars, we briefly explain how to efficiently implement portable high-order finite element discretizations on unstructured grids for a wide variety of problems, including leveraging time integration schemes for transient problems. Additionally, we describe how to utilize the automatic differentiation and adjoint capabilities for solving nonlinear problems and computing sensitivities. Finally, we present the simulation-constrained optimization capabilities available through the ROL package, highlighting how different Trilinos tools work together to efficiently solve large-scale inference problems.

The Trilinos project is one of many in the scientific software community that employs the concept of Continuous Integration and thusly has a testing process that ensures code functionality and quality as part of its contribution process. This talk is aimed at software developers and researchers interested in enhancing their understanding of Trilinos’ CI practices. We will examine the current processes that contributors must navigate, alongside emerging paradigms shaping the evolution of these practices. Key topics will include the implementation of containerized software testing environments, requirements for running tests on Trilinos compute resources, and objective standards that contributions must meet to guarantee software quality. Attendees will also gain insights into the challenges of providing a highly configurable set of software packages while maintaining user-friendly build processes. By the end of the session, participants will have a clearer understanding of how to effectively contribute to Trilinos, and the direction that DevSecOps efforts in Trilinos are heading in the future.

The 4C (Comprehensive Computational Community Code) multiphysics simulation framework has been developed to address complex physical phenomena across various scientific and engineering domains. From its inception, 4C has relied on the Trilinos project, an open-source software library for scalable numerical computations, as its backbone for sparse linear algebra and MPI-parallel computing. This integration enhances 4C's computational capabilities and, more importantly, allows the 4C developers to focus on their core research interest: the numerical modeling of multiphysics systems. The synergy between 4C's physics models and Trilinos' numerical solvers facilitates the simulation of coupled multiphysics systems with improved accuracy and performance. Over the years, this synergy has — in parts — evolved to a co-development of both software frameworks. This presentation will delve into the methodologies employed to incorporate Trilinos into the 4C framework, discuss software and development challenges, and showcase application case studies that demonstrate the practical benefits of this integration in simulating complex multiphysics systems such as fluid-solid interactions, contact mechanics or beam-solid interaction.

The solution of multiphysics problems depends on the construction of robust preconditioners which significantly influence the convergence rate and computational performance of linear solvers. We employ physics-based block preconditioning strategies from the Trilinos/Teko package developed by Cyr, Shadid, and Tuminaro [SIAM Journal on Scientific Computing, 38(5):S307–S331, 2016]. Teko preconditioner parameter selection, however, remains a difficult task for users. This talk presents an automated approach for the selection of multiphysics preconditioner parameters in the Trilinos/Teko package. We discuss design principles for preconditioners, emphasizing the importance of physics blocking, subblock ordering, and subblock solver choices based on equation types and coupling strengths. Finally, we present an extension to the Trilinos/Teko package in the form of a user-friendly software tool for automatic multiphysics preconditioner design with minimal user input.

The Trilinos scientific software library is a key enabling technology for application codes in a variety of physics and engineering areas across the US Department of Energy and beyond. This presentation will "pull back the curtain" and describe the process by which the Trilinos Tpetra/Performance team (a) performs nightly testing across five DOE laboratories, (b) how we identify and remedy performance regressions, (c) how we make the validated build scripts available to Trilinos developers, and (d) how we interface with early users and vendors to ensure that vendor library updates work as advertised.

PyTrilinos2 is an auto-generated Python interface for several Trilinos packages. I will explain how additional C++ functionality can be exposed to Python and demonstrate existing solver capabilities.