HPSF Conference 2025

As the demand for efficient DevOps infrastructure for software applications continues to grow, key components such as build processes, build time tests, regression testing, and deployment mechanisms have become critical to successful project delivery. Spack is increasingly being adopted as an orchestration architecture for DevOps, offering a framework that can adapt to various project needs. However, while the overarching patterns among projects may be similar, the specific implementation details can differ significantly, leading to challenges in achieving seamless integration. In this talk, we will present the gaps identified by researchers at Sandia National Laboratories in Spack's current services and our ongoing efforts to address these challenges.Key topics will include advancements in binary caches, binary provenance, reporting test results, enhancing build performance, and improving the overall developer user experience. Attendees will gain valuable insights into successful initiatives that have effectively closed certain gaps, as well as ongoing issues that remain open for the community to tackle.

In this presentation, we will describe how we manage to develop multi-physics applications with a software stack deployed with Spack.

We will describe how we have designed a workflow, first using Spack features for development, such as "setup," "dev-build," and "develop." And how we ended up creating a Spack plugin to generate custom CMake presets.

CMake presets are a portable way to set up CMake configurations. They can describe several configurations. And they can be exploited by different tools, from the command line to IDE.
Generating these presets from Spack concretizations allows users to exploit their classical environment, benefiting both from a correct installation of their dependencies as well as advanced features of the IDE that do not need to integrate explicitly with Spack.

We will present our journey between the different spack solutions, the development of a CMake "preload cache"-based answer, and we will illustrate use cases of why we ultimately switched to CMake presets.

In this presentation I will present the feedback of packaging and deploying the dependencies of the open source library `gyselalibxx` using Spack on a local cluster and on the national supercomputer Adastra. I will explain the challenges faced to deploy the library and why we chose Spack.

The Linac Coherent Light Source (LCLS) at the SLAC National Accelerator Laboratory takes X-ray snapshots of atoms and molecules in action, revealing fundamental processes in materials, technology and living things. Advanced data and computing systems are playing an increasingly vital role in LCLS’s operation: data acquisition software developed at LCLS allows the collection of very high throughput data, and makes sure that all the collected data is stored to disk. Additionally, LCLS provides scientists with sophisticated data analysis frameworks and helps them run their own bleeding-edge scientific analysis tools. With new detectors and analysis techniques being introduced all the time, new software is constantly being developed and maintaining compatibility with old hardware while supporting new frameworks and programs is an everyday challenge. Spack seems to offer a solid foundation for the software development process and is starting to be used at LCLS. This talk will discuss how Spack is making the development process easier and what challenges the LCLS Data Analysis group is still facing in making Spack a useful tool for everyday work.

So, we got bored with installing stuff. Our project aims to accelerate time to results, improve reproducibility, and reduce reliance on proprietary programming environments and manual installs. Built on Spack, our tooling enables the rapid deployment of versioned programming environments to globally distributed HPC clusters, ensuring consistency across clusters and regions. This talk will present our solution and the problems it solves for us, discuss its benefits for HPC productivity, and invite community feedback on its broader applicability.

Managing software stacks on high-performance computing (HPC) systems is a complex challenge, particularly when working within vendor-provided programming environments. To address this, the National Center for Computational Sciences (NCCS) at Oak Ridge National Laboratory has developed NCCS Software Provisioning (NSP), a unified framework for deploying and managing software stacks while monitoring their usage.

NSP leverages Spack and Ansible to automate the deployment of software environments. Using templated configurations, it streamlines Spack-based installations while also managing non-Spack software through custom Ansible roles. Additionally, NSP enhances LMOD-based module environments by incorporating hooks and Spack module projections, enabling a dynamic and responsive software layout that adapts seamlessly to changes in the programming environment.

This presentation will discuss the motivation behind NSP, its strategies for managing software complexity in vendor-provided environments, and its implementation on Frontier, the world’s fastest supercomputer for open science.

The rapid growth in features, capabilities, and adoption of Spack has transformed it into a vital tool for managing software dependencies in scientific computing. However, one of the significant challenges faced by large organizations is the effective communication and information sharing across diverse teams. This talk will present a case study from Sandia National Laboratories, detailing the evolution and impact of strategies implemented to address these communication hurdles. We will explore the formation of a self-organized working group dedicated to fostering collaboration and knowledge exchange around Spack, as well as a recently funded initiative aimed at enhancing Spack collaboration and development efforts. By examining the successes and lessons learned from these organizational approaches, we aim to provide insights into best practices for engaging teams in the adoption of Spack, ultimately promoting a more cohesive and efficient development environment. Attendees will leave with a deeper understanding of how structured engagement strategies can facilitate the integration of Spack within large organizations and enhance collaborative software development efforts.

Spack team members manage a complex continuous integration (CI) infrastructure to compile and test software distributed via a centralized build cache. The workflows are executed in virtualized cloud environments, an approach that offers considerable flexibility in resource provisioning, resulting in quick feedback on pull requests submitted by the Spack community. However, fixed resource allocations for build jobs lead to job failures, slowdowns, and suboptimal resource usage.

spack-gantry aims to address these challenges by providing resource allocation predictions to build pipelines. We use historical build data to forecast future utilization and set resource limits. We will present strategies used to find optimal predictions, as well as experiments to create an online prediction service modeled after a genetic algorithm. The results of this work include improvements in usage efficiency and cost-per-job metrics.

Because spack-gantry involves modifications to the current CI system, we will discuss ramifications for users and developers and solicit feedback about how to make the infrastructure more user-friendly, reliable, and functional for the Spack community.