The question of determining the actual chemical structure underlying a sensorially perceivable substance, such as the product of a chemical reaction, is a core challenge in chemistry. Analytical chemistry provides a number of techniques to address this problem. Amongst those nuclear magnetic resonance (NMR) spectroscopy is one of the most informative. Each NMR spectrum is characteristic of a specific molecular structure, as different structures produce different spectra. Skilled spectroscopists can infer a structure from spectra, a process known as structure elucidation. For in-depth details of NMR spectroscopy and structure elucidation methodologies, we refer the reader to Elyashberg (2015) and the bibliography there.

The ambition to automate the process of structure elucidation emerged as soon as computer systems became available in chemistry. Software created for this is known as computer-aided structure elucidation (CASE) software (Kuhn et al. 2025). Major developments in the area included COCON (Lindel et al. 1997), SENECA (Kuhn and Steinbeck 2010), LSD/pyLSD (Nuzillard 2003), Mestrelab MNova (MestraLab 2024), Bruker CMC-se (Kessler and Godejohann 2018), ACD/Structure Elucidator (Elyashberg et al. 2004 and Sherlock (Wenk et al. 2023). NMRFilter (Kuhn et al. 2019, Kuhn et al. 2020) is an example of compound annotation in mixtures. Quantum calculations-aided structure elucidation also offers high-precision insights by simulating NMR parameters directly from first-principles (Costa et al. 2021), but those methods will not be addressed here. Those programmes typically compare candidate structures to the spectral data measured using spectrum prediction. Given the vastness of chemical space, they typically use heuristics (e.g. genetic algorithms or simulated annealing) to restrict the search space. Further constraints (e.g. known functional groups) are used to retrieve a result as unambiguous as possible. Nevertheless, the outcome is frequently a ranked list of candidate structures rather than a definitive solution. Given the recent significant successes achieved by artificial intelligence systems (for example, the essays produced by large language models, Binz et al. (2025)), it is a natural question to ask if this task could be performed by an AI system without the explicit steps contained in CASE systems (Kuhn et al. 2025). Whilst, to our knowledge, there is currently no such complete system, steps in the direction have been taken. One example is Hu et al. (2024), which predicts substructures from 1D spectra and assembles these into structures. The DeepSAT system (Kim et al. 2023) combines structure generation with database searches. As opposed to the directed generation of compounds to find one that matches the measured spectra best, DeepSAT predicts a fingerprint and a chemical class from the spectra using machine-learning techniques and searches a database for the best fit to those predicted properties. In Kuhn et al. (2022), it was demonstrated that a neural network can tell if a certain fragment is contained in a structure or not from an NMR spectrum. This work uses two-dimensional experiments, namely Heteronuclear Single Quantum Coherence (HSQC) and Heteronuclear Multiple Bond Correlation (HMBC). They show 1-bond and 2/3-bond 13C-1H shift correlations and, therefore, contain a significant amount of structural information. This neural network can be viewed as a step towards data-driven automated structure elucidation.

Whilst ML methods hold significant promise, there are downsides as well. Chief amongst these is their "black-box" nature, as models often produce predictions without providing insight into the underlying reasoning. They produce a result, but there is no explanation given as to why or any visible reasoning. Not only is this unsatisfying for the user, but it also makes users distrust the results. Hence, explainable AI is a critical development. This paper shows some attempts to make the results from Kuhn et al. (2022) explainable. To the best of our knowledge, this represents the first attempt to incorporate explainable AI into the context of NMR-based structure elucidation.

NMR and Machine Learning

Machine learning, specifically supervised machine learning, works by presenting a model with solved examples of a class of problems and adjusting parameters in the model to approach the solutions. If the model is then presented with unsolved problems, it can apply the knowledge learnt and give a (good) solution. One possible class of models are neural networks, which are loosely modelled on the structure of the human brain. A specific class of neural networks are convolutional neural networks (CNNs), which have proven their strength particularly for interpreting image and other array-like data. That means that they may also be suitable for NMR spectra, as demonstrated in Kuhn et al. (2022). Fig. 1 shows the networks employed for the task of classifying spectra by substructures. In Fig. 2, the three substructures for which the network has been trained are shown. The three structures will be used as examples throughout the paper. For details of the machine-learning architecture and tools used, we refer the reader to Kuhn et al. (2022).

Figure 1.  

Architecture of the single spectrum convolutional neural network used in this work.

Figure 2.  

Example for the three categories of structures, with the characteristic substructure highlighted. Left: Elaidic Acid, bmse000643, middle: Hydroxyindoleacetic Acid, bmse000364, right: α-Cholestan-3-one, bmse000489. Substructure searches exclude side chains on the fatty acid, but allow any groups otherwise.

Class Activation Maps (CAM), originally introduced by Zhou et al. (2016), are a possibility to overcome the black-box nature of neural networks in the context of image analysis. Their aim is to display regions in an image that contribute the most to the result. A further development of CAM is Grad-CAM (Gradient-weighted class activation mapping, Selvaraju et al. 2017, Selvaraju et al. 2019), which we will mainly use. Fig. 3 shows an example of an application of Grad-CAM, the identification of a cat and a dog in an image. Not surprisingly, the regions used are the animals themselves. Less obvious is that, seemingly, the identification of the dog mainly uses the face, whereas for the cat, the body is the main contributor.

Figure 3.  

An example of Grad-Cam explaining how a neural network identifies a cat and a dog in an image. Heatmap from Selvaraju et al. (2019), original image "Sabian and Maggy" by Joyell VanGelder is licensed under CC BY 2.0.

Grad-CAM is more universal than CAM since it does not require a specific architecture. Grad-CAM determines the importance of the features by calculating the gradient in respect to the majority class (or any target class of interest) of the feature maps. Then the given gradients are global-average-pooled over the width and height dimensions to obtain the importance of the neurons. The importance scores are then used as the weight of the corresponding feature maps. By combining these weighted maps, a single localised map is created that highlights the areas of the image that had the most significant impact on the decision of the network. After that, the result is passed through ReLu activation to obtain only the areas that had a positive effect on the network's decision-making process, thus creating a heat map. Gradient-weighted class activation mapping plus plus (Grad-CAM++, Chattopadhay et al. (2018)) is a derivative and extension of the Grad-CAM algorithm. It tries to improve on the accuracy of the heatmaps by multiple object instances or small target regions by also using second- and third-order gradients of the feature map for the importance calculation in addition to first-order gradients.

In Fig. 4, we show the ideal result of applying CAM to the NMR fragment learning, using the HSQC spectrum of a fatty acid. Ideally, exactly those peaks from the fatty acid fragment would be highlighted since only they have been used. Of course, in reality, some mismatches are to be expected and this result will not be possible.

Figure 4.  

Left: The unchanged HSQC spectrum of a fatty acid (bmse000643). Right: The ideal result of the tool, applied to this spectrum. The tool would highlight exactly those peaks belonging to the atoms in the fatty acid fragment. In the image, the region where important peaks are located is enlarged.

Data

To evaluate the effectiveness of Class Activation Mapping (CAM) methods and neural networks in chemical structure elucidation, a series of heatmap images were generated. The evaluation focuses on comparing the heatmaps of pure substances with those of artificially created mixtures that include the original pure substances. These artificial mixtures are produced by overlaying spectra from the three target compound classes - steroids, fatty acids and indoles - with spectra from compounds that do not belong to any of these subclasses. Each artificial mixture will include spectral data from one of the target classes along with one or two additional spectra from unrelated compounds. This method enables the assessment of the models’ ability to accurately classify relevant substructures and identify the substructure region in the spectra using Grad-CAM and Grad-CAM++ algorithms.

Grad-CAM

Pure substances

By evaluating pure compounds, the neural network managed to correctly classify all of the substances by both HMBC and HSQC spectra (Fig. 5 and Fig. 6). The generated heat map images showed minor misalignment with the actual location of the spectra. This could be due to the upscaling of the heatmap when it is generated from the last dense layer of the neural network. Since we are trying to specifically identify substructures, the spectra also contain peaks that are not part of the substructure. As, during the learning process, a multitude of spectra of compounds with the same core substructure are presented to the network, the network can identify the relevant parts. This was demonstrated by the ability of the network to identify test structures not used in the training. Looking at the heat maps, it is obvious that they mostly highlight all regions with peaks. That is a very similar to a first evaluation that an experienced spectroscopist would do. An example where this is not the case is the HSQC spectrum of the steroid, where only a portion of the visible spectral cluster is highlighted, with a substantial amount of peaks left unaccounted for. Ideally, we would see peaks precisely identified like this everywhere.

Figure 5.  

Grad-CAM analysis of pure HMBC spectra showing feature importance for different compound classes. Left: A fatty acid spectrum, bmse000643, centre: A steroid spectrum, bmse000489, right: An indole spectrum, bmse000364.

Figure 6.  

Grad-CAM analysis of pure HSQC spectra showing feature importance for different compound classes. Left: A fatty acid spectrum, bmse000643, centre: A steroid spectrum, bmse000489, right: An indole spectrum, bmse000364.

Figure 7.  

Fatty acid HMBC spectrum (bmse000643), artificial mixture with bmse000060 (Adenine), Grad-CAM analysis of the mixture.

Figure 8.  

Indole HMBC spectrum (bmse000364), mixture with bmse000060 (Adenine), Grad-CAM analysis of the mixture.

Mixture with one additional compound

When using artificial mixtures for analysis, it becomes evident that the network partly retains its ability to accurately classify the presence of the three target substructures. This was indicated in \cite{fragments}, where, by using the HMBC spectra, all three examples could be identified correctly and, by using the HSQC spectrum, one of the three examples was identified correctly. An example for this is the HMBC spectrum for a steroid (Fig. 9), where we can observe that the additional spectral information introduced by the mixtures exerts only a minimal influence on the network's decision-making process. The heatmap reveals that regions containing spectral signals from non-target chemical compounds - not the steroid in this case - exhibits slight activation only. On the other hand, for the indole and fatty acid structures (Fig. 7 and Fig. 8), these areas are still used. It should be noted that the result is correct, but this could be an accidental result. Overall, it is not clear from the heatmaps if the network is able to consistently separate the compounds.

Figure 9.  

Steroid HMBC spectrum (bmse000489), artificial mixture with bmse000060 (Adenine), Grad-CAM analysis of the mixture.

Figure 10.  

Fatty acid HSQC spectrum (bmse000643), bmse000060 (Adenine) and bmse000061 (Adenosine), Grad-CAM analysis of the artificial mixture. Misclassified as steroid.

Figure 11.  

Indole HSQC spectrum (bmse000364), artificial mixture with bmse000060 (Adenine) and bmse000061 (Adenosine), Grad-CAM analysis of the artificial mixture.

Mixture with two additional compounds

In order to further investigate the effect of mixtures, we checked artificial mixtures of three components (the target substance plus two other substances). In \cite{fragments}, only three of six cases were identified correctly. The heatmaps show a similar picture. In case of the steroid HMBC spectrum (Fig. 12), the additional peaks are only slightly activated. On the other hand, the HSQC spectra for the indole and fatty acid structures (Fig. 10 and Fig. 11) show activation of wrong peaks as well and one of them was misclassified. Overall, it can be seen that, by mixtures with more than one additional spectra, the effectiveness of the network can vary significantly. In Fig. 10, the neural network along with Grad-CAM analysis also includes non-relevant spectral information by the decision, although it classified the substructure correctly.

Figure 12.  

Steroid HMBC spectrum (bmse000489), artificial mixture with bmse000060 (Adenine) and bmse000061 (Adenosine), Grad-CAM analysis of the artificial mixture.

Grad-CAM++

By applying the Grad-CAM++ algorithm, the heat maps generated for the pure spectra (Fig. 13 and Fig. 14) are very similar to those generated with Grad-CAM. We also show a mixture Grad-CAM++ heatmap (Fig. 15). Again, the heatmap looks very similar to the Grad-CAM heatmap. What can be observed is that the misalignment between peaks and regions of attention is less in Grad-CAM++ than in Grad-CAM, using the same network. On the other hand, judging from the examples, fundamental differences are not observable. We therefore do not repeat the details of all experiments with Grad-CAM++ here.

Figure 13.  

Grad-CAM++ analysis of pure HMBC spectra showing feature importance for different compound classes. Left: A fatty acid spectrum, bmse000643, centre: a steroid spectrum, bmse000489, right: An indole spectrum, bmse000364.

Figure 14.  

Grad-CAM++ analysis of pure HSQC spectra showing feature importance for different compound classes. Left: A fatty acid spectrum, bmse000643, centre: a steroid spectrum, bmse000489, right: An indole spectrum, bmse000364.

Figure 15.  

Fatty acid HMBC spectrum (bmse000643), artificial mixture with bmse000060 (Adenine), Grad-CAM++ analysis of the mixture. Misclassified as steroid.

From our observations, we can draw a number of conclusions about the use of CAMs in structure elucidation.

One observation is that, at least Grad-CAM and Grad-CAM++, do not provide the level of detail which would be needed to identify peaks. This makes the ideal goal of this exercise a target we have not reached.

On the other hand, the heatmaps produced show that, for pure compounds, relevant parts of the spectrum are considered by the networks. On the other hand, when looking at the mixtures, the picture is not clear - in many cases (but not all), parts of the compounds which should not contribute, are used as well. This confirms the results of Kuhn et al. (2022), where fragments were identified reliably in pure compounds, whereas mixtures often failed.

Due to this, heat maps, even if not ideal for determining peaks, can help to evaluate a model beyond the usual metrics. As we have seen, using Grad-CAM, we better understand what is going on in the model than without it. In so far, this can serve as a step towards explainable AI. It should be noted that the basis for the application is the use of spectral images and convolutional neural networks, which are commonly used for image analysis.

We have demonstrated that a class activation map, particularly Grad-CAM, can help better understand the results of analysing spectra using neural networks. Whilst we did not achieve our ideal goal, we have demonstrated that the previous results become better understandable and easier to interpret. It is also clear that better models are needed for the mixtures. In general, finer-grained methods than Grad-CAM would be helpful. Finally, more data will help establish a broader base for analysis.

We are grateful to the BMRB project for providing the spectral data used in this paper. Carlos Corbas from MestreLab Research S.L.U, Santiago, Spain and colleagues have kindly helped with data processing.

Conceptualisation, S.K.; methodology, S.K.; software, E.K.; validation, R.M.B., R.P.J.; investigation, S.K., E.K.; writing---original draft preparation, E.K.; writing---review and editing, S.K., R.M.B., R.P.J.; supervision, S.K. All authors have read and agreed to the published version of the manuscript.