Benchmark investigation

Optimal condition to run n2p2 WF.

Some investigations have been carried out to determine whether n2p2 performs simple parallelization using MPI or also supports OpenMP (multi-threaded setup). It appears that the best setup for n2p2 is with NODES=2 and THREADS=1. This means that n2p2 does not seem to perform multi-threaded parallelization efficiently, but rather only multi-node parallelization.

Moreover, even when the number of threads is fixed to 1, the total number of training epochs (i.e., the amount of data trained) as a function of the number of nodes does not increase linearly. The linear approximation still holds up to two nodes.

Another benchmarking investigation concerns the variation of the input dataset size. The entire dataset consists of a collection of 51,702 structures of boron atoms. Subsets of the dataset were investigated in increments of 5,000 structures using NODES=1, CORES=32, THREADS=1, and the command mpirun -n 32. In this specific dataset, 5,000 structures correspond to 69.4 MB, while 51,702 structures correspond to 984.1 MB.

The throughput as a function of the number of structures is plotted in Figure 1, and it is defined as the ratio between the total training time and the number of structures.

Figure 1: Efficiency R as function of number of processed asked. x-axes in log2 scale

The fact that n2p2 is not implemented with multi-threading suggests conducting a more detailed investigation by fixing our test calculations with NODES=1, CORES=32, and THREADS=1. In order to evaluate the efficiency trend as a function of the number of processes, the number of epochs was fixed to 100, and the training calculation was executed using the following command:

mpirun -n num_proccess --bind-to core

where num_process is the number of processes requested, and num_process ∈ [1, 2, 4, 8, 16, 32, 64]. In the specific case of num_process = 64, the number of nodes is set to 2. The -bind-to core option is important to ensure correct parallelization of the processes without overloading a single socket. More information can be found in the official mpirun documentation mpirun documentation.

The plot in Figure 2 shows the efficiency (R) as a function of the number of processes, and it can be observed that the maximum efficiency occurs for mpirun -n 1, then decreases as the number of processes increases. Moreover, a comparison between the native installation and the Apptainer installation within a Guix environment is presented, showing that the performance loss when using the Apptainer version is minimal.

Figure 2: Efficiency R as a function of number of processed asked. x-axes in log2 scale

From the present investigation, we can conclude that, for n2p2 calculations, the optimal configuration is NODES=1, CORES=8, THREADS=1, and mpirun processes equal to 32. In fact, as shown in Figure 2, after 8 processes we lose the linear scaling trend; consequently, the total execution time increases exponentially. Further analysis will be performed for 64 processes to complete the investigation.

Concerning the comparison between the native installation and the Apptainer version, we observe that the Apptainer execution is slower. In particular, it requires approximately 20–25% more total execution time. Nevertheless, it allows calculations to be reproduced much more easily, which represents a good compromise. In the specific case of NODES=1, CORES=8, THREADS=1, 100 epochs run in 7h and 22 minutes and 6h and 8 minutes respectively for Apptainer vs normal installation.