Practical Cheminformatics

Model Validation Central

Published machine learning methods for molecular property prediction, re-run on the same molecules, the same folds and the same statistics, so their claims can be read against one another instead of against whatever each paper chose to compare itself with.

The reports, newest first

12 reports from 11 studies: expansion-ml-comparison produces two, asking different questions of the same folds. Each entry links to the directory it came from.

1. 2026-10-30Graph-transformer pre-training, with an exposure control

   KERMT v2 and GROVER, fine-tuned on the same folds

   Does NVIDIA's re-pretrained GROVER encoder, fine-tuned by its own code, hold up against the ADMET field — and how much of any gain survives the fact that it was pre-trained on these molecules?

   The contrastive re-pretraining beats both original GROVER checkpoints, and scaling GROVER does not help at all, but none of the three reaches fine-tuned Atom-JEPA or Monroe + TabPFN 3 on Biogen. Frozen, KERMT's latent is far behind Monroe's.

   Tests GROVER, KERMT, NV-KERMT-70M-v2 · ExpansionRx, Biogen ADME · code

2. 2026-10-233D molecular pre-training at scale

   Uni-Mol and Uni-Mol2, out of the box

   Do Uni-Mol and Uni-Mol2, fine-tuned with their own package at its defaults, hold up against the ADMET field?

   Uni-Mol2 84M is a real step up from Uni-Mol and beats fine-tuned CheMeleon, but it trails fine-tuned Atom-JEPA everywhere and Monroe + TabPFN 3 on Biogen. Frozen, neither Uni-Mol vector wins a single call against Monroe's.

   Tests Uni-Mol, Uni-Mol2 · ExpansionRx, Biogen ADME · code

3. 2026-10-20Causal inference on a learned embedding

   Counterfactuals from one analog

   Does a structural causal model on Chemprop's embedding, anchored on one measured member of a series, rank the rest of the series better?

   Only with the released checkpoints. Refit with the authors' own code it loses within-task ρ on 10 of the paper's 11 tasks, and on ExpansionRx and Biogen it ranks worse than the Chemprop it is built on in 172 of 204 series.

   Tests Causal-Chemprop · ExpansionRx, Biogen ADME, Causal-Chemprop kinase tasks · code

4. 2026-10-123D quantum-property pre-training

   Pre-training in 3D on quantum mechanics

   Does pre-training a 3D transformer on quantum-mechanical properties help ADMET prediction?

   Quantum pre-training beats training the same transformer from scratch on 14 of 15 endpoints, as the paper says. Against the field it is middling: roughly level with fine-tuned CheMeleon, well behind fine-tuned Atom-JEPA and Monroe + TabPFN 3.

   Tests QuADMET-Former · ExpansionRx, Biogen ADME · code

5. 2026-10-093D self-supervised pre-training

   Fine-tuning in 3D

   Does a 3D equivariant encoder pre-trained on 19 million conformers beat the best frozen encoder?

   Against its own comparators the paper's claim holds, ahead of fine-tuned CheMeleon on 38 of 45 combinations. Against Monroe + TabPFN 3 the data sets disagree: it leads on ExpansionRx and trails on Biogen, and what it wins it wins through fine-tuning, not the frozen encoder.

   Tests Atom-JEPA · ExpansionRx, Biogen ADME · code

6. 2026-10-04Error correction

   Borrowing errors

   Does correcting each prediction with the known errors of similar training molecules help?

   The correction works as claimed, ahead of its own uncorrected baseline on 41 of 45 combinations, but it is a small lever, a median +0.024 R². Fine-tuning CheMeleon or reading Monroe's embedding is worth far more, and MolPAIR loses to both.

   Tests MolPAIR · ExpansionRx, Biogen ADME · code

7. 2026-09-30Tabular foundation models

   Swapping the head

   Holding Monroe's frozen embedding fixed, which in-context tabular model reads it best?

   Kumo Tabular large is the best head, on top of all 45 combinations and at an eighth of TabFM's GPU time. But the head is the small lever: swapping it moves R² by a median 0.042, against 0.208 for the representation.

   Tests TabICLv2, Kumo Tabular, TabFM · ExpansionRx, Biogen ADME · code

8. 2026-09-18Does k-NN really match modern ML?

   What survives a hard split

   A paper says nearest-neighbour lookup matches modern ML under every split, and that all of them collapse to a naive baseline out of distribution. Does that hold against models it did not have?

   Half the paper replicates: error does climb as the splits harden, but k-NN leads none of the 18 scheme × metric combinations and Monroe's frozen encoder leads all 18. Nothing is more robust to distance; the better methods are uniformly better.

   Tests Split fragility, Nearest neighbours match ML · ChEMBL 30, ten potency targets (Lee et al.) · code

9. 2026-09-16Foundation models

   Nine ways to model ADME

   Which foundation model, and does it replicate?

   Of four molecular foundation models only Monroe wins anything, on top of 21 of 27 ExpansionRx and all 18 Biogen combinations with no downstream training at all. The representation is worth about ten times the tabular head that reads it.

   Tests Monroe, Mol-JEPA, MEGA-CL, CheMeleon · ExpansionRx, Biogen ADME · code

10. 2026-09-01Self-supervised pre-training

    Pre-training on fingerprints

    A graph network taught to predict fingerprints, against the fingerprints

    Pre-training a GIN to predict ECFP wins six endpoints of fifteen and loses six, and on its own Biogen benchmark wins one and loses three. It helps where fingerprints were weak and hurts where they were strong.

    Tests PT-GIN · ExpansionRx, Biogen ADME · code

11. 2026-08-31Fusion strategies

    Does multimodal fusion help?

    Unimodal versus multimodal, on ADME data

    As the paper found, fusing four modalities barely moves accuracy, and none of the gain survives a multiple-comparison correction; what fusion buys is calibration. A single well-initialised graph network beats all 33 configurations.

    Tests Unimodal vs Multimodal Learning · ExpansionRx, Biogen ADME · code

12. 2026-08-29Model selection

    Sixty models per fold

    Is picking a model per endpoint worth it?

    Choosing one of sixty candidate models per endpoint and fold loses to a single well-initialised graph network on both data sets. Nothing in its pool learns a representation, and what it picks does not replicate.

    Tests Trimole-Hybrid · ExpansionRx, Biogen ADME · code

Every method, on the same folds

The studies share their folds, and the four reference arms are copied between them rather than refit, so their per-fold numbers are identical to the last bit. That is checked before anything is pooled, and it means every method below can go into one comparison — which is the whole reason for collecting them.

One Tukey HSD per endpoint, over the 25 folds, on MAE — the same metric the panels below draw, so a row and its panel can never disagree about who won. A method counts as being on top whenever the correction cannot separate it from the leading mean, so an endpoint with several methods on top gives each of them a tie rather than crowning whichever had the better mean. Only an endpoint with exactly one method on top awards a best alone. There are no bold maxima.

That rule is also why the three Monroe rows are almost never alone at the top. They differ only in which tabular model reads the same frozen embeddings, and methods the correction cannot separate take each other out of the best alone column wherever they are both on top. Read them as one method with a footnote, not as three that beat each other.

multimodal-fusion's 33 configurations would swamp a wide comparison and widen the correction for everyone, so it is represented by one cell of its grid chosen in advance rather than on the results: all four modalities, early fusion, LightGBM. Its own report ranks the whole grid. Counts here are also stricter than the ones in the individual reports: Tukey's correction widens with the number of methods, so pooling this many of them turns some outright wins into ties.

Lower is better, and the panels share an axis so a difference is the same length everywhere. Read down a panel rather than across them: MAE carries the units of its endpoint, and the endpoints differ in how hard they are. On ExpansionRx they are also on a log10(x + 1) scale, which is nearly linear below x = 1, so an MAE there is not a clean fold-error — every method shares the target, so the comparison holds, but the number does not convert. The Biogen endpoints arrive log-transformed at source.

What makes these comparable

Every study here is scored the same way, which is what makes the cards comparable rather than merely adjacent. Molecules are clustered, a fixed fraction is held out once, and the 25 replicates come from five repeats of a five-fold GroupKFold over the clusters in what remains. Every method sees identical training molecules in every fold and is scored on the same untouched test set, so a difference in the metrics is a difference in the method.

The statistics follow the protocol of Ash et al. and Even More Thoughts on ML Method Comparisons: distributions over folds, corrected for multiple comparisons, and no bold maxima. Where two methods cannot be separated, both are reported as tied.

Everything is run on two unrelated data sets. A comparison run once is a hypothesis, and the interesting question about any result below is which half of it replicates.

Each entry links to that study's full report and to the code that produced it. Every study directory carries its own README, its own requirements, and the per-fold metrics behind its figures, so any of them can be rebuilt on its own. Source at PatWalters/model-validation-central. Built 2026-10-30.