How to use n2p2 Apptainer image

For more information on Apptainer containers and their use, we provide a description of Apptainer, a crash course on how to use Apptainer, and of course there’s also the official Apptainer’s documentation.

Input files

To illustrate the various commands, a set of n2p2 input files is available in the form of an archive via this link.

The archive contains the following files for training a neural network potential (NNP) for Cu2S and predicting energies and forces:

• train/input.nn: Neural network architecture and training parameters.
• train/input.data: Training dataset containing atomic configurations, forces, and energies for Cu2S.
• train/plot_histo.py: Python script to visualize symmetry function histograms from nnp-scaling.
• train/plot_learning_curve.py: Python script to plot the learning curve from nnp-train.
• predict/input.data: Input configurations for energy and force predictions.

In this tutorial, we will assume that the input files contained in this archive are in the current directory. To extract them:

tar -xzf n2p2-tutorial-inputs.tar.gz

For more details on the input files and their format, refer to the n2p2 training procedure documentation.

Quickstart

For impatient folks, here is how to train a neural network potential for Cu2S and predict energies and forces using the container image in the case where the current directory contains the n2p2.sif container image and all necessary input files:

cd train
apptainer exec ../n2p2.sif mpirun -np 4 nnp-scaling 500
apptainer exec ../n2p2.sif mpirun -np 8 nnp-train
cp weights.016.000010.out ../predict/weights.016.data
cp weights.029.000010.out ../predict/weights.029.data
cp input.nn ../predict/
cp scaling.data ../predict/
cd ../predict
apptainer exec ../n2p2.sif nnp-predict 1

Detailed usage for the n2p2 container

This section presents a step-by-step guide to using the n2p2 image for training a neural network potential (NNP) for Cu2S and predicting energies and forces. For more details about Apptainer commands, please look at this tutorial.

Introduction

n2p2 is an MPI-parallelized open-source software suite for generating and using neural network-based energy potentials for atomistic and molecular modeling. It enables the training of models on data derived from ab initio calculations, such as those from Density Functional Theory (DFT), to accurately predict the energetic and structural properties of atomic systems.

The main executables in the image are:

• nnp-scaling: Scales symmetry functions for the training dataset.
• nnp-train: Trains the neural network potential.
• nnp-predict: Predicts energies and forces for new configurations.

For more information on the n2p2 tools, refer to the official documentation.

Workflow Overview

The workflow for training and using a neural network potential consists of three main phases:

• Phase 1: Data Scaling (nnp-scaling)
• Phase 2: Neural Network Training (nnp-train)
• Phase 3: Prediction (nnp-predict)

The following sections describe each phase in detail.

Phase 1: Data Scaling

The first step in training a neural network potential is to scale the symmetry functions of the training dataset. This normalization ensures that all symmetry functions have comparable ranges, which is crucial for stable and efficient training. For more details on the scaling process, refer to the nnp-scaling documentation.

This phase requires two input files:

• input.nn: Defines the neural network architecture and symmetry functions. For Cu2S, this file specifies:

  • Elements: S (Sulfur) and Cu (Copper).
  • Symmetry functions,
  • Network architecture: 2 hidden layers with 15 nodes each,
  • Training parameters.

• input.data: Contains the training dataset with atomic configurations, forces, and energies for Cu2S. Each configuration is defined within a begin/end block, including:

  • Lattice vectors.
  • Atom positions, types, and forces.
  • Total energy and charge for each configuration.

The following command runs the scaling phase on 4 CPUs in parallel. The 500 command line parameter determines the number of bins for symmetry function histograms. The screen output of this command contains a useful section about memory requirements during training.

cd train
apptainer exec ../n2p2.sif mpirun -np 4 nnp-scaling 500

The nnp-scaling command generates several output files, among which:

• scaling.data: Contains the scaling parameters (minimum, maximum, mean, and standard deviation) for each symmetry function. These parameters are used to normalize the symmetry functions during training. For example, the first few lines of scaling.data show the scaling parameters for the symmetry functions of element S:

  #  e_index   sf_index   sf_min   sf_max   sf_mean   sf_sigma
  1          1   9.06E-06   1.05E-04   3.68E-05   1.78E-05
  1          2   9.74E-01   1.37E+00   1.14E+00   7.82E-02

• sf.*.histo: Histogram files for each symmetry function, showing the distribution of symmetry function values across the dataset. For example, sf.016.0001.histo contains the histogram for the first symmetry function of element S.

• nnp-scaling.log.*: Log files for each MPI process, containing detailed information about the scaling process.

To visualize the symmetry function histograms, use the provided plot_histo.py script:

python plot_histo.py

This script reads a histogram file and generates a bar plot showing the distribution of symmetry function values (e.g., sf.016.0064.histo contains the histogram for symmetry function 64 for S atoms). The name of the file specified in the plot_histo.py file can be modified to visualize a different histo output file.

For more details on interpreting the histograms and scaling parameters, refer to the nnp-scaling documentation.

Phase 2: Neural Network Training

After scaling the symmetry functions, the next step is to train the neural network potential using the scaled dataset. The nnp-train tool uses the architecture defined in input.nn and the scaled data to optimize the weights of the neural network. For more details on the training process, refer to the nnp-train documentation.

This phase requires the following input files:

• input.nn: Neural network architecture and training parameters (same as in Phase 1),

• input.data: Training dataset (same as in Phase 1).

• scaling.data: Scaling parameters generated by nnp-scaling in Phase 1. This file is automatically read by nnp-train to normalize the symmetry functions.

To train the neural network potential using 8 processors in parallel, use the following command:

apptainer exec ../n2p2.sif mpirun -np 8 nnp-train

This command typically takes a few minutes to complete, depending on the size of the dataset and the number of MPI processes used.

The nnp-train command generates several output files, among which:

• learning-curve.out: Contains the learning curve data, including the root mean square error (RMSE) for energies and forces for both training and test sets across all epochs. For example, the first few lines of learning-curve.out show the following columns:

  # epoch RMSEpa_Etrain_pu  RMSEpa_Etest_pu  RMSE_Ftrain_pu  RMSE_Ftest_pu
  0   8.06E-02   7.56E-02   5.21E-01   4.57E-01
  1   4.88E-03   1.63E-03   2.20E-01   2.44E-01
  2   1.82E-03   6.41E-04   1.35E-01   1.46E-01

  Here, RMSEpa_Etrain_pu is the RMSE of training energies per atom, and RMSE_Ftrain_pu is the RMSE of training forces. The learning curve shows how the errors decrease with each epoch, indicating the improvement of the neural network potential.

• weights.*.data: Neural network weight files for each epoch. For example, weights.016.000010.out contains the weights after the 10th epoch. These files are used to restore the neural network potential for predictions or further training.

• timing.out: Timing information for each epoch.

To visualize the learning curve, use the provided plot_learning_curve.py script:

python plot_learning_curve.py learning-curve.out

This script reads the learning-curve.out file and generates a plot showing the RMSE for energies and forces for both training and test sets across all epochs. The learning curve helps monitor the training progress and identify issues such as overfitting or slow convergence.

For example, the learning curve for this tutorial shows a steady decrease in RMSE for both energies and forces, indicating successful training:

• Epoch 0: RMSE energy (train) = 8.06E-02, RMSE force (train) = 5.21E-01
• Epoch 10: RMSE energy (train) = 2.72E-04, RMSE force (train) = 4.99E-02

This demonstrates that the neural network potential is learning to accurately predict both energies and forces for the Cu2S system.

For more details on interpreting the learning curve and training output, refer to the nnp-train documentation.

Phase 3: Prediction

After training the neural network potential, the final step is to use it to predict energies and forces for new configurations. The nnp-predict tool applies the trained potential to input configurations and outputs the predicted values. For more details on the prediction process, refer to the nnp-predict documentation.

This phase requires the following input files:

• input.nn: Neural network architecture (same as in Phases 1 and 2).

• weights.*.data: Neural network weight files for each element. For Cu2S, this includes:

  • weights.016.data: Weights for element S (Sulfur).
  • weights.029.data: Weights for element Cu (Copper). These files are generated by nnp-train and must be copied from the training directory. For this tutorial, we use the weights from the last epoch (epoch 10).

• scaling.data: Scaling parameters generated by nnp-scaling in Phase 1. This file is required to normalize the symmetry functions during prediction.

• input.data: Input configurations for which energies and forces will be predicted. This file contains atomic configurations in the same format as the training data.

To predict energies and forces for configuration described in input.data, use the following commands:

cd ../predict
cp ../train/weights.016.000010.out weights.016.data
cp ../train/weights.029.000010.out weights.029.data
cp ../train/input.nn .
cp ../train/scaling.data .
apptainer exec ../n2p2.sif nnp-predict 1

The nnp-predict command generates several output files:

• energy.out: Contains the comparison between predicted and reference energies. For example:

  # Ennp    Eref    Ediff    E_offset
  -5.73669358E+02  -5.73700369E+02  -3.10110046E-02   0.00000000E+00

  Here, Ennp is the energy predicted by the neural network potential, Eref is the reference energy, and Ediff is the difference between them. In this example, the predicted energy is -573.669 eV, while the reference energy is -573.700 eV, resulting in a difference of -0.031 eV.

• nnforces.out: Contains the comparison between predicted and reference forces for each atom. For example, the first few lines of nnforces.out show:

  # fx     fy     fz     fxRef   fyRef   fzRef   fxDiff   fyDiff   fzDiff
  -1.26E-01  1.93E-02  5.72E-02  -1.25E-01  2.84E-02  3.12E-02  7.56E-04  9.13E-03  -2.60E-02

  Here, fx, fy, and fz are the predicted force components, while fxRef, fyRef, and fzRef are the reference force components. The differences (fxDiff, fyDiff, fzDiff) show how closely the neural network potential reproduces the reference forces.

• output.data: Contains the input configuration with the predicted energies and forces added.

• structure.out: Contains detailed structure information, including atomic positions and predicted forces.

For more details on interpreting the prediction output, refer to the nnp-predict documentation.

To go further

For further reading, refer to the n2p2 documentation and the specific documentation for each tool:

• nnp-scaling
• nnp-train
• nnp-predict

The commands above use Apptainer “embedded” parallel mode. More information on using Apptainer containers in parallel, including usage on clusters and the difference between embedded and hybrid parallel modes, can be found on this page.