Published machine learning methods for molecular property prediction,
re-run on the same molecules, the same folds and the same statistics, so their claims
can be read against one another instead of against whatever each paper chose to
compare itself with.
The studies share their folds, and the four reference arms are copied between them rather than refit, so their per-fold numbers are identical to the last bit. That is checked before anything is pooled, and it means every method below can go into one comparison — which is the whole reason for collecting them.
One Tukey HSD per endpoint, over the 25 folds, on MAE — the same metric the panels below draw, so a row and its panel can never disagree about who won. A method counts as being on top whenever the correction cannot separate it from the leading mean, so an endpoint with several methods on top gives each of them a tie rather than crowning whichever had the better mean. Only an endpoint with exactly one method on top awards a best alone. There are no bold maxima.
That rule is also why the three Monroe rows are almost never alone at the top. They differ only in which tabular model reads the same frozen embeddings, and methods the correction cannot separate take each other out of the best alone column wherever they are both on top. Read them as one method with a footnote, not as three that beat each other.
Every study here is scored the same way, which is what makes the cards comparable rather than merely adjacent. Molecules are clustered, a fixed fraction is held out once, and the 25 replicates come from five repeats of a five-fold GroupKFold over the clusters in what remains. Every method sees identical training molecules in every fold and is scored on the same untouched test set, so a difference in the metrics is a difference in the method.
Everything is run on two unrelated data sets. A comparison run once is a hypothesis, and the interesting question about any result below is which half of it replicates.
Does k-NN really match modern ML? · ChEMBL 30, ten potency targets (Lee et al.)
A paper says nearest-neighbour lookup matches modern ML under every split, and that all of them collapse to a naive baseline out of distribution. Does that hold against models it did not have?
The papers put to the test
The Fragility of Bioactivity Prediction: Rigorous Dataset Splits Expose the Illusion of ML Accuracy
Lee, K.; Moldagulov, G.; Grzybowski, B. A.
Chem. Eur. J. 2026, e71208 · doi:10.1002/chem.71208
Simple nearest-neighbour analysis meets the accuracy of compound potency predictions using complex machine learning models
Janela, T.; Bajorath, J.
Nat. Mach. Intell. 2022, 4, 1246-1255 · doi:10.1038/s42256-022-00581-6
Eleven arms on the ten ChEMBL potency sets and six splitting schemes of Lee, Moldagulov and Grzybowski, run on that paper's own folds: four methods fitted here, the paper's naive baseline, and the paper's own six arms imported at their published per-fold numbers. 1,500 fold models, and a re-implementation of the paper's k-NN that reproduces 250 of 250 of its released folds to 4e-9, which is what licenses putting the two sources on one axis.
Half the paper replicates. Error does climb as the splits harden, but so does the naive baseline it is measured against, and reading the gap instead of the raw MAE splits the two claims apart. k-NN is at the top of none of the 18 scheme x metric combinations and Monroe's frozen encoder is at the top of all 18, separating from k-NN on all six schemes and on five of six under a conservative target-level test. On the hardest schemes k-NN takes 0.005 and 0.010 pIC50 off the naive predictor and the paper's own GNN goes negative, while Monroe holds 0.121 and 0.141. The decay rates are nearly identical across arms, 0.81 to 0.98; what differs is the offset, so nothing here is more robust to distance, it is just uniformly better.
The paper's six arms are its own published per-fold metrics, not re-run, and each had 50 Optuna trials per fold where the four arms fitted here use library defaults. The upstream release ships two disagreeing copies of the splits; the .pkl copy is used, because it reproduces the paper's numbers and the .json copy does not.
11arms
6splitting schemes
1,500fold models fitted
250/250published folds reproduced
0/18combinations k-NN leads
Read the report →|split-fragility|2026-09-18
Foundation models · ExpansionRx, Biogen ADME
Which foundation model, and does it replicate?
The papers put to the test
Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference
Banaszewski, B.; Fitzgibbon, A. W.
Preprint, 2026 · arXiv:2608.18982
Mol-JEPA: A Multimodal Joint Embedding Predictive Architecture for Molecules
Rottach, F.; Schieferdecker, S.; Rudman, W.; et al.
Preprint, 2026 · arXiv:2608.22642
MEGA-CL: A Molecular Foundation Model for Generalizable ADMET Prediction through Graph External Attention and Contrastive Learning
Jin, T.; Jin, K.; Li, Y.; et al.
Preprint, 2026 · arXiv:2607.24314
Deep Learning Foundation Models for Low-Data Regimes from Classical Molecular Descriptors
Burns, J. W.; Zalte, A. S.; Abreu, C. R. A.; et al.
J. Chem. Inf. Model. 2026, articles ASAP · doi:10.1021/acs.jcim.6c01546
Nine modelling approaches over fifteen ADME and physicochemical endpoints on two unrelated data sets, 25 replicate models each, every one scored on a held-out test set it never saw. Four of the nine are pre-trained molecular foundation models, and only one of those four wins anything at all. The one that wins does no downstream training whatsoever — it freezes a pre-trained encoder and predicts each endpoint in context — and it appears three times, under three different tabular models, which is what lets the page price the representation against the predictor bolted onto it.
Monroe's frozen encoder is at the top of 21 of 27 ExpansionRx combinations and all 18 Biogen ones; MEGA-CL and Mol-JEPA are best on nothing, anywhere. Crossing two frozen representations with two in-context heads settles which half does the work: changing the head moves R² by a median of 0.016 and not even in a consistent direction, while changing the representation moves it by 0.168 on fifteen endpoints out of fifteen. The representation is worth about ten times the head.
9methods
15endpoints
2,925fold models
3.19Mpredictions kept
Read the report →|expansion-ml-comparison|2026-09-16
Model selection · ExpansionRx, Biogen ADME
Is picking a model per endpoint worth it?
The paper put to the test
A Multimodal Representation Learning Platform for Accurate Molecular ADMET Prediction
Luo, Z.; Huang, D.; Shao, Y.; Yu, Q.; Li, Y.
Bioinformatics 2026, in review · doi:10.1101/2026.08.24.746660
Trimole-Hybrid is not a model but a selection procedure. For every endpoint and every fold it builds sixty candidate predictors — five molecular views crossed with three blocks of chemistry priors crossed with four backends — fits all of them, scores them on the held-out validation fifth, and keeps exactly one to predict the test set. It is put against four fixed architectures on the same fifteen endpoints.
It loses to a single well-initialised graph network on both data sets, for two reasons only visible in what it selected: nothing in its pool learns a representation, and on ExpansionRx the split it selects on does not resemble the split it is scored on. What it chose does not replicate either.
A reimplementation. The authors' release describes itself as not runnable, so this reproduces the method, not the paper's numbers.
5methods
60candidates per fold
22kcandidate fits
1,425fold models
Read the report →|expansion-ml-comparison|2026-08-29
Self-supervised pre-training · ExpansionRx, Biogen ADME
A graph network taught to predict fingerprints, against the fingerprints
The paper put to the test
On Improving Graph Neural Networks for QSAR by Pre-training on Extended-Connectivity Fingerprints
Money-Kyrle, S.; Dablander, M.; Hanser, T.; Werner, S.; Deane, C. M.; Morris, G. M.
Preprint, 2026 · arXiv:2605.10722
PT-GIN is a Graph Isomorphism Network pre-trained on 462,189 QMugs molecules to predict each molecule's own 2,048-bit hashed ECFP4, then frozen and handed to LightGBM. Its authors report it beating hashed ECFP on five of six Biogen ADME tasks. Here it is run against those same fingerprints through that same predictor, so the only thing that differs between the two arms is the representation.
It wins six endpoints of fifteen and loses six; on Biogen, its own benchmark, it wins one and loses three. Which six it wins is not about how much data an endpoint has — it is about how well the fingerprints were already doing. PT-GIN helps where the baseline was weak and hurts where it was strong, Spearman ρ = −0.65 with the baseline R².
5methods
15endpoints
1,425fold models
10released checkpoints
Read the report →|ecfp-pretrain-comparison|2026-09-01
Fusion strategies · ExpansionRx, Biogen ADME
Unimodal versus multimodal, on ADME data
The paper put to the test
Unimodal vs Multimodal Learning: A Systematic Evaluation of Fusion Strategies and Model Design for Molecular Property Prediction and Uncertainty Quantification
Wasswa, J.; Kajjumba, G. W.; Ramsundar, B.
J. Chem. Inf. Model. 2026 · doi:10.1021/acs.jcim.6c01878
Thirty-three ways of combining four molecular representations — RDKit descriptors, Mol2Vec, a supervised GNN's graph embedding and a character BiGRU's SMILES embedding — crossed with early and late fusion and three final learners, over the same fifteen endpoints. 12,375 fitted configurations and 6,375 more for the two controls.
The paper's central finding replicates: fusing modalities moves accuracy very little, and none of it survives a correction for multiple comparisons. What fusion buys is calibration. Two things the paper could not see from inside its own design also show up — a single well-initialised graph network beats all thirty-three configurations on both data sets, and when the GNN modality is actually a learned representation, which in the released code it is not, the ranking of which modality matters inverts.
A reimplementation. The authors' MIT-licensed release is a set of Colab dumps that cannot be run, so this is written from that source read as a specification alongside the paper's Supporting Information.
33configurations
15endpoints
18,750fits
4modalities
Read the report →|multimodal-fusion|2026-08-31
