HPSF Conference 2025

Spack 1.0 will feature faster builds through multi-package parallelism. In this talk, I’ll describe how I accelerated Spack’s package installation process by parallelizing its main installer loop. By enabling Spack to spawn multiple package builds concurrently, this feature increases available parallelism and significantly reduces overall build times for multi-package installations. I'll talk about the design decisions, tradeoffs, and performance gains achieved, providing valuable lessons for optimizing package builds on platforms from laptops to large-scale HPC environments. I’ll also talk about how, even as a a relatively new Spack user, I was able to come up to speed quickly and contribute meaningful improvements to the project.

Binary package managers allow for the fast installation of binary artifacts, but limit configurability to ensure compatibility between binaries due to rigid ABI requirements. Source package managers allow for more flexibility in building software, since binaries are compiled on demand, but compilation can take a considerable amount of time. Spack has existing a mechanism for mixing source and precompiled packages; however, because Spack does not model ABI compatibility between packages, all transitive dependencies of a binary package must have been built at the same time as that package in order to maintain ABI compatiblity. We present an extension to Spack, which we call splicing, that models ABI compatibility in the package ecosystem and allows seamless mixing of source and binary distribution of packages. This extension augments both the packaging language and dependency resolution engine of Spack in order to maximize reused binaries while maintaining the flexibility of source based management. Through empirical evaluation, we show that our extension incurs minimal performance overhead to dependency resolution while greatly extending the modeling capability of Spack.

While primarily run, tested, and developed on Unix-like operating systems, in particular HPC systems, scientific software is often deployed on Windows. Spack's recent expansion to Windows marks a significant milestone, enabling its powerful package management capabilities on a new platform.This talk will provide technical insight into the process of adapting Spack to support a platform orthogonal in design to anything Spack has supported in the past. We cover the path to initial support, the current state of Spack on Windows, and look at the roadmap for future Windows development. We’ll explore unique challenges supporting Windows and their solutions. This talk will address design, new features introduced to support Windows development, and how Spack brings needed robust package management to the Windows ecosystem. We’ll cover best practices for porting new or existing packages to Windows, deploying and managing Spack environments in real-world scenarios, and standardizing your Windows development workflows with a focus on common pitfalls for Windows developers. This talk will provide a pathway for attendees interested in supporting Windows to go forth and Spack!

Spack's Continuous Integration (CI) system is essential for building and distributing reliable HPC software packages. It has dramatically scaled from building hundreds to hundreds of thousands of packages weekly. Leveraging GitLab, Spack CI has evolved from a Linux-only build system to orchestrating runners across diverse providers, architectures, and platforms, supporting multiple domain stacks like E4S, HEP, and Radiuss. Through enhanced monitoring and data-driven methodologies, including machine learning, Spack CI has gained insights to optimizing resource allocation and analyzing failure modes. Improvements to binary caching reduce storage and prevent race conditions. The core goal is to maintain a resilient and efficient CI ecosystem, ensuring the reliability of Spack's HPC software for its expanding community.