Pat Walters

Pat Walters

Cheminformatics, ML

GNN’s can extrapolate for some properties, but there’s a trick

2 minute read

Published:


This guest post was written by Jeffery Zhou and Alan Cheng, and is a follow-up to “Why Don’t Machine Learning Models Extrapolate?”

One of the great things about Practical Cheminformatics is the discussion it spurs from the community. We’ve long been interested in the practical aspects of model extrapolation for small molecule and biologics properties, and we were intrigued by Pat’s findings.

Some GNN background. Graph neural networks (GNNs) such as Chemprop can often show significantly improved predictions when datasets are sufficiently large. They work by generating learned representations (embeddings) for each atom and bond which are dependent on the neighboring atoms and bonds. Relevant to our discussion here is how embeddings are aggregated to generate a single molecular embedding for the whole molecule (think of it as creating a fingerprint representation). They can be aggregated by simply summing up the embeddings, ‘sum aggregation’, or, alternatively, by averaging the embeddings, ‘mean aggregation’. In practice, instead of doing a simple summing up, every bit is divided by a constant value to make the values smaller and this is called ‘norm aggregation’. Our group has empirically found that ‘norm aggregation’ performs better for predicting small molecule activities and properties and this was also reported in the recent Chemprop paper.

Here’s the trick. We intuitively expected GNNs to extrapolate well for a simple property like MW, but this is not what Pat found and we were able to recreate his plot (left below). Jeffery noticed that Pat used ‘mean aggregation’ instead of ‘norm aggregation’. We tried ‘norm aggregation’ and got the good extrapolative results on the right! In Jeffery’s Jupyter notebook, he does additional experiments to show there is no ‘unfair’ data leakage here relative to RF, LGB, and linear regression.

Extrapolation for more complex models. We’ve been investigating the potential of GNNs like Chemprop to extrapolate for more complex endpoints such as small molecule and peptide ADMET endpoints, and have demonstrated improvements in GNN models over RF/LGB models when applied to the extrapolative situations of larger molecules (higher MW), more polar molecules (greater tPSA), and more lipophilic molecules (higher SlogP). Check out our preprint for more details!

Jeffery Zhou (jzhou59[at]jh.edu; jeffery.zhou[at]merck.com)
Alan Cheng (alan.cheng[at]merck.com)

You May Also Enjoy

Useful RDKit Utils - A Mötley Collection of Helpful Routines

5 minute read

Published:


A few years ago, I assembled an open-source collection of Python functions and classes that I use regularly. My motivations for putting this together were primarily selfish; I wanted to quickly pip install the functions I use all the time. The result was useful_rdkit_utils, a library of Cheminformatics and machine learning (ML) functions. The library is available on GitHub, and the documentation can be found on readthedocs. The GitHub repo also includes a set of Jupyter notebooks that demonstrate some of the library’s capabilities. I recently refactored the code and added new functionality, so I thought it might be worth writing a blog post to reintroduce the library. Here is a brief overview of the useful_rdkit_utils library. Read more

The Trouble With Tautomers

13 minute read

Published:


Introduction
One factor often overlooked when applying machine learning (ML) in small-molecule drug discovery is the influence of tautomers on model predictions. Drug-like molecules, especially those containing heterocycles and conjugated pi systems, can exist in several different tautomeric forms. These forms feature varying bond orders between the atoms. Consequently, the molecular representation used in an ML model varies. This remains true regardless of whether we’re using molecular fingerprints, topological descriptors, or message passing neural networks (MPNN). Read more

Why Don’t Machine Learning Models Extrapolate?

5 minute read

Published:


Introduction
One thing newcomers to machine learning (ML) and many experienced practitioners often don’t realize is that ML doesn’t extrapolate. After training an ML model on compounds with µM potency, people frequently ask why none of the molecules they designed were predicted to have nM potency. If you’re new to drug discovery, 1nM = 0.001µM. A lower potency value is usually better. It’s important to remember that a model can only predict values within the range of the training set. If we’ve trained a model on compounds with IC50s between 5 and 100 µM, the model won’t be able to predict an IC50 of 0.1 µM. I’d like to illustrate this with a simple example. As always, all the code that accompanies this post is available on GitHub. Read more