Python Exchange

Clear documentation is crucial for the success of open-source libraries, but it’s often hard to get right. In this talk, I’ll share our experience applying the Diataxis documentation framework to improve two HoloViz ecosystem libraries, hvPlot and Panel. Attendees will come away with practical insights on applying Diataxis and strengthening documentation for their own projects.

Good documentation turns users into contributors — but achieving it requires more than good intentions. This talk shares the journey of applying the Diataxis framework to improve two open-source Python libraries from the HoloViz ecosystem: Panel and hvPlot.. We’ll start with a short introduction to Diataxis (its four documentation types: tutorials, how-to guides, explanations, and references), then briefly present the libraries we worked on and their documentation challenges.

The heart of the talk focuses on practical lessons learned: how we mapped existing content into the Diataxis structure, handled content gaps and redundancies, engaged with the user community, and evolved our approach over time. We’ll also discuss what we would do differently if we started again.

The goal is to give attendees a realistic, hands-on perspective on adopting Diataxis — including both its benefits and its challenges.

Watch on Youtube

With the rise of machine-learned interatomic potentials in the atomistic simulation community in Materials Science, the complexity of simulation workflows changed from directed acyclic graph (DAG) based simulation workflows with typically just a single simulation code to simulation workflows coupling simulation codes at different length and time scales and thousands of individual simulations. To orchestrate these workflows a number of simulation frameworks were developed. Still as part of the Exascale Computing Project we realized these were limited in their flexibility and scalability.

So, we developed executorlib[1] based on the concurrent futures Executor interface in the Python standard library and with the goal to distribute Python functions over hundreds of compute nodes. Internally, executorlib leverages the Simple Linux Utility for Resource Management (SLURM) and the flux framework from Lawrence Livermore National Laboratory[2] to up-scale simulation workflows from a workstation or traditional high-performance computers (HPC) to the latest generation of Exascale machines. In contrast to previous solutions, it does not require any daemon process or database but instead directly interfaces with the job manager to maximize computational efficiency. At the same time, it is designed with a focus on debugging capabilities to minimize the overhead of migrating workflows to the Exascale machines.

In this presentation I introduced executorlib, highlighting the lessons learned from development of the pyiron atomistic simulation suite which led to the development of executorlib as minimalistic workflow manager. Finally, I highlight the general applicability of executorlib to distribute python functions of any scientific domain on HPC clusters of all sizes.

[1]: Janssen et al., JOSS, 10(108), 7782, (2025).
[2]: Dong H. Ahn et al., Fut. Gen. Comp. Sys., 110, (2020).

Watch on Youtube