Graph-transformer pre-training · 5×5 cross validation · two data sets

KERMT v2 and the two GROVER checkpoints it descends from, fine-tuned on the same folds

KERMT¹ is NVIDIA's GPU-optimised reimplementation of GROVER³, the dual-view graph transformer that pre-trains on 11 million unlabelled molecules by predicting masked subgraph motifs; KERMT v2² re-pretrains that encoder with a contrastive, probabilistic objective and is the checkpoint NVIDIA released. All three are fine-tuned here by KERMT's own main.py finetune, one multitask model per fold, on the same fifteen ExpansionRx and Biogen endpoints, folds and held-out test sets as the rest of this site, and v2's frozen 512-d latent is also read by TabPFN 3⁶ exactly as Monroe is. The re-pretraining earns its keep inside the family: fine-tuned v2 beats fine-tuned GROVER base 30–13–2 and GROVER large 33–12–0 over 45 endpoint × metric calls, and all three clear LightGBM and fine-tuned CheMeleon⁷. Scaling the original checkpoint does not: GROVER large goes 3–26–16 against GROVER base. But none of the three reaches the front of this field. Fine-tuned v2 trails fine-tuned Atom-JEPA⁴ 7–11–27, wins no Biogen call against Monroe + TabPFN 3⁵ (0–2–16), and is never best alone on either data set. Frozen, v2's latent loses 38 of 45 calls to Monroe's vector. And the comparison carries a caveat the GROVER arms exist to expose: v2 was pre-trained on a corpus that contains every molecule in both test sets.

The scoreboard

Tukey HSD per endpoint and metric, counted the same way across every study on this site: a method is on top if Tukey's simultaneous correction cannot separate it from the best mean at α = 0.05, and it is only best alone if no other arm shares the top group. MAE is in the log units of each endpoint, followed by R².

ExpansionRx¹¹

Biogen ADME¹²

On ExpansionRx fine-tuned Atom-JEPA is on top of 19 of 27 combinations (13 of them alone) and Monroe + TabPFN 3 of 9 (5 alone). The three arms fine-tuned here land in a band below that and close together: KERMT v2 on top of 7 combinations, GROVER base 7, GROVER large 6, and none of the three ever alone. v2's are three endpoints: LogD, LOG_MLM and LOG_MGMB. On Biogen ADME the picture is Monroe's — on top of 17 of 18 combinations, 13 of them alone — and the fine-tuned graph transformers mostly drop out: KERMT v2 reaches the top tier on 2 of them, all on LOG_RLM, and neither GROVER arm nor the frozen latent reaches it at all.

Head-to-head across all 45 endpoint × metric calls

Paired by fold, a two-sided t-test over the 25 shared folds at p < 0.05 for each endpoint and metric. Inside the family the ordering is clear and in the direction the v2 paper argues: fine-tuned v2 over fine-tuned GROVER base 30–13–2 (mean +0.023 R²) and over GROVER large 33–12–0 (+0.035 R²), with the two GROVER arms close enough that most of their calls go unmade — GROVER large against base is 3–26–16, so the bigger original checkpoint is if anything the weaker one. Outside the family it gets harder. All three beat LightGBM almost everywhere and CheMeleon more often than not. Against Monroe, v2 is ahead on ExpansionRx (15–1–11) and loses everything that is called on Biogen (0–2–16); the same split, deeper, holds for both GROVER arms. Against fine-tuned Atom-JEPA every one of the three loses clearly: v2 7–11–27, GROVER base 4–4–37, GROVER large 5–4–36.

Frozen or fine-tuned

Reading v2 frozen through Monroe's TabPFN 3 call splits the distance from Monroe to fine-tuned v2 into two steps: first swap Monroe's vector for v2's latent, then fine-tune v2 instead of reading it.

As a frozen vector v2's latent is behind Monroe's on 13 of the 15 endpoints — ahead only on LOG_MBPB and LOG_MPPB — by a median −0.119 R² on ExpansionRx and −0.209 on Biogen, and it wins only 3 of 45 calls against Monroe. It also loses to CheMeleon 2–2–41, though it does beat LightGBM 26–7–12, so the latent is not uninformative — just not a drop-in for an encoder built to be read frozen. Fine-tuning then lifts R² on 13 of 15 endpoints, by a median +0.131 R² on ExpansionRx and +0.164 on Biogen, and wins 41 of 45 calls against the frozen arm while losing 0. The net against Monroe is a median +0.009 R² on ExpansionRx and −0.079 on Biogen — the one data set it gains on and the one it does not. This is the same shape the Uni-Mol study found: an encoder pre-trained to reconstruct or contrast, rather than to summarise a molecule in one vector, needs its weights moved before it competes.

What it costs

The 150 fine-tuned fold models took 149 GPU hours across two machines (apollo, an RTX 5070 Ti; spark, a DGX Spark GB10). The one arm both machines ran, GROVER base on ExpansionRx, took a median 4.2× as long on spark, though that figure flatters apollo: spark ran two fine-tuning jobs at a time for most of the sweep, which raises its time per fold and its throughput together. KERMT runs its full epoch budget with no early stopping, so a fold costs the same whatever epoch it ends up keeping. The kept epoch is usually well inside the 100 available — a median of 61 for v2, 40 for GROVER base and 35 for GROVER large — so the budget is not the binding constraint, and an early-stopping patience would cut most of this bill. On ExpansionRx, the one arm pairing both ran on apollo, a GROVER large fold costs 1.4× a GROVER base one for its 1200-wide encoder, and gets nothing for it. The frozen arm needs one forward pass per molecule and then TabPFN 3, minutes rather than hours, and is not tabulated.

Reported the way “Even More Thoughts on ML Method Comparisons” and the protocol paper it points at¹³ argue comparisons should be: distributions over folds, simultaneous confidence intervals, paired tests on the folds, and no bolded maxima. Where two methods cannot be separated, both are reported as tied.

Code, per-fold metrics and every figure: studies/kermt.

References

1. Adrian, C.; Mahajan, S. P.; Sirelkhatim, A.; et al. KERMT: Accelerating Molecular Property Prediction with GPU-Optimised Graph Transformers. Preprint, 2025. NVIDIA's accelerated reimplementation of GROVER, and the code that fine-tunes every arm fitted here. github:NVIDIA-BioNeMo/KERMT
2. Xue, D.; Mahajan, S. P.; Adrian, C.; et al. Contrastive Pre-training for Molecular Property Prediction at Scale. Preprint, 2026. The re-pretrained encoder released as NV-KERMT-70M-v2, fine-tuned here as kermt_ft and read frozen as kermt_tabpfn. hf:nvidia/NV-KERMT-70M-v2
3. Rong, Y.; Bian, Y.; Xu, T.; Xie, W.; Wei, Y.; Huang, W.; Huang, J. Self-Supervised Graph Transformer on Large-Scale Molecular Data. NeurIPS 2020. The dual-view graph transformer KERMT reimplements; its released base and large checkpoints are the two unexposed arms here. NeurIPS 2020
4. Petersen, K. H.; Tirsgaard, R. H.; Cornet, F. R. J.; Jordahn, M.; Schmidt, M. N. Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems. Preprint, 2026. The 39M-parameter 3D self-supervised comparator, fine-tuned. arXiv:2610.08400
5. Banaszewski, B.; Fitzgibbon, A. W. Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference. Preprint, 2026. The frozen encoder in the strongest in-context arm. arXiv:2608.18982
6. Grinsztajn, L.; Flöge, K.; Key, O.; et al. TabPFN-3: Technical Report. Preprint, 2026. The head of the Monroe arm and of the frozen KERMT arm. arXiv:2605.13986
7. Burns, J. W.; Zalte, A. S.; Abreu, C. R. A.; et al. Deep Learning Foundation Models for Low-Data Regimes from Classical Molecular Descriptors. J. Chem. Inf. Model. 2026, articles ASAP. The pre-trained 2D D-MPNN comparator. 10.1021/acs.jcim.6c01546
8. Graff, D. E.; Morgan, N. K.; Burns, J. W.; et al. Chemprop v2: An Efficient, Modular Machine Learning Package for Chemical Property Prediction. J. Chem. Inf. Model. 2026, 66 (1), 28–33. 10.1021/acs.jcim.5c02332
9. Kelley, B. Descriptastorus. Software, 2018–. The fixed-CDF RDKit 2D descriptors KERMT concatenates to its readout. github:bp-kelley/descriptastorus
10. Rogers, D.; Hahn, M. Extended-Connectivity Fingerprints. J. Chem. Inf. Model. 2010, 50 (5), 742–754. The LightGBM baseline's representation. 10.1021/ci100050t
11. OpenADMET; Expansion Therapeutics. OpenADMET–ExpansionRx Blind Challenge Data. Hugging Face, 2026, CC BY 4.0. 10.57967/hf/9687
12. Fang, C.; Wang, Y.; Grater, R.; et al. Prospective Validation of Machine Learning Algorithms for Absorption, Distribution, Metabolism, and Excretion Prediction: An Industrial Perspective. J. Chem. Inf. Model. 2023, 63 (11), 3263–3274. 10.1021/acs.jcim.3c00160
13. Ash, J. R.; Wognum, C.; Rodríguez-Pérez, R.; et al. Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery. J. Chem. Inf. Model. 2025, 65 (18), 9398–9411. 10.1021/acs.jcim.5c01609