3D molecular pre-training at scale · 5×5 cross validation · two data sets

Uni-Mol and Uni-Mol2, fine-tuned the way the package does it, against the ADMET field

Uni-Mol¹ pre-trains a 3D SE(3)-invariant transformer on 209 million RDKit conformers by masked-atom prediction and coordinate denoising; Uni-Mol2² scales the idea to a two-track atom/pair transformer and reports downstream error falling as it grows from 84 million to 1.1 billion parameters. Both come with a fine-tuning package, unimol_tools, and that is how they are run here: the package's own recipe at its defaults, on the same fifteen ExpansionRx and Biogen endpoints, folds and held-out test sets as the rest of this site, and each encoder also frozen and read by TabPFN 3⁵ exactly as Monroe is. Fine-tuned Uni-Mol2 84M is the stronger of the two and a real step up from Uni-Mol (30 wins, 12 no call, 3 losses over 45 endpoint × metric calls): it beats fine-tuned CheMeleon⁶ (31–8–6), never loses to LightGBM (42–3–0), and on ExpansionRx reaches the top Tukey tier on 14 of 27 combinations, more than Monroe + TabPFN 3⁴ (9). But it is never best alone, it trails fine-tuned Atom-JEPA³ (3–6–36), and on Biogen it loses all 18 calls to Monroe and is on top of none. Frozen, neither Uni-Mol vector wins a single call against Monroe's; fine-tuning is what makes these models competitive.

The scoreboard

Tukey HSD per endpoint and metric, counted the same way across every study on this site: a method is on top if Tukey's simultaneous correction cannot separate it from the best mean at α = 0.05, and it is only best alone if no other arm shares the top group. MAE is in the log units of each endpoint, followed by R².

ExpansionRx⁹

Biogen ADME¹⁰

On ExpansionRx fine-tuned Atom-JEPA is on top of 19 of 27 combinations (7 of them alone), fine-tuned Uni-Mol2 of 14 — on six endpoints: LogD, LogS, LOG_HLM, LOG_MLM, LOG_MPPB and LOG_MGMB — always sharing the tier, and Monroe + TabPFN 3 of 9. Fine-tuned Uni-Mol reaches it one time. On Biogen ADME the picture is Monroe's: on top of 18 of 18 combinations (13 alone), Atom-JEPA of 5, and no Uni-Mol arm of any. Neither frozen Uni-Mol arm is on top anywhere.

Head-to-head across all 45 endpoint × metric calls

Paired by fold, a two-sided t-test over the 25 shared folds at p < 0.05 for each endpoint and metric. Fine-tuned Uni-Mol2 against fine-tuned Uni-Mol goes 30–12–3, so at the same recipe the larger, newer model is better, by a mean +0.060 R². It beats fine-tuned CheMeleon 31–8–6 where Uni-Mol does not (13–8–24), and both beat LightGBM (42–3–0 and 41–0–4). Against Monroe + TabPFN 3 Uni-Mol2 splits by data set: 14–5–8 on ExpansionRx and 0–0–18 on Biogen. Against fine-tuned Atom-JEPA, the other 3D model and one that was pre-trained on Uni-Mol's own conformers, both lose clearly: Uni-Mol2 3–6–36, Uni-Mol 3–5–37.

Frozen or fine-tuned

Reading each encoder frozen through Monroe's TabPFN 3 call splits the distance from Monroe to fine-tuned Uni-Mol2 into two steps: first swap Monroe's vector for Uni-Mol2's, then fine-tune Uni-Mol2 instead of reading it.

As a frozen vector Uni-Mol2's is behind Monroe's on all 15 endpoints, by a median −0.203 R² on ExpansionRx and −0.150 on Biogen, and neither frozen arm wins one of the 45 calls against Monroe (Uni-Mol2 0–0–45, Uni-Mol 0–0–45). Frozen Uni-Mol is behind even LightGBM (5–12–28); frozen Uni-Mol2 edges it (23–9–13) but loses to CheMeleon 3–5–37. Fine-tuning then lifts R² on 12 of 15 endpoints for Uni-Mol2 and 13 of 15 for Uni-Mol, by a median +0.213 and +0.075 R² for Uni-Mol2 on the two data sets (+0.207 and +0.158 for Uni-Mol), winning 40 of 45 calls against its own frozen arm (41 for Uni-Mol). The net against Monroe is a median +0.010 R² on ExpansionRx and −0.120 on Biogen. Unlike Monroe, whose encoder never sees these labels, these vectors were never meant to be used frozen: Uni-Mol is pre-trained to denoise coordinates and recover atoms, not to summarise a molecule in one token.

Fold to fold

The fine-tuned Uni-Mol arms are the least stable fine-tuned arms here. Across the fifteen endpoints the median ratio of their fold-to-fold R² standard deviation to fine-tuned Atom-JEPA's is 2.0 for Uni-Mol2 and 1.8 for Uni-Mol. It is worst on the two Caco-2 endpoints of ExpansionRx, LOG_Caco_AB and LOG_Caco_Efflux, where Uni-Mol2's mean R² is below zero and its standard deviation reaches 0.44: some folds are fine and some are badly miscalibrated. A likely contributor is that the package early-stops each fold model on one validation score pooled over every endpoint, so the epoch it keeps is the one that suits the data set as a whole, not the hardest endpoint in it.

What it costs

The 100 fine-tuned fold models took 43 GPU hours, 3 for Uni-Mol and 40 for Uni-Mol2, across two machines (apollo, an RTX 5070 Ti; spark, a DGX Spark GB10, which took a median 3.3× as long per Uni-Mol2 fold). On the same machine a Uni-Mol2 fold took a median 8.8× as long as a Uni-Mol one, though the two keep similar epochs (a median best epoch of 33 against 30), so the cost is the larger model, not longer training. Time per fold is one MolTrain call's wall clock, five folds and the test predictions, shared evenly. The frozen arms need one forward pass per molecule and then TabPFN 3, minutes rather than hours, and are not tabulated.

Reported the way “Even More Thoughts on ML Method Comparisons” and the protocol paper it points at¹¹ argue comparisons should be: distributions over folds, simultaneous confidence intervals, paired tests on the folds, and no bolded maxima. Where two methods cannot be separated, both are reported as tied.

Code, per-fold metrics and every figure: studies/unimol.

References

1. Zhou, G.; Gao, Z.; Ding, Q.; Zheng, H.; Xu, H.; Wei, Z.; Zhang, L.; Ke, G. Uni-Mol: A Universal 3D Molecular Representation Learning Framework. ICLR 2023. The 3D transformer fine-tuned here as unimol, from its released mol_pre_all_h_220816 weights. openreview:6K2RM6wVqKu
2. Ji, X.; Wang, Z.; Gao, Z.; et al. Uni-Mol2: Exploring Molecular Pretraining Model at Scale. NeurIPS 2024. The two-track transformer scaled from 84M to 1.1B parameters; its 84M model is fine-tuned here as unimol2. arXiv:2406.14969
3. Petersen, K. H.; Tirsgaard, R. H.; Cornet, F. R. J.; Jordahn, M.; Schmidt, M. N. Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems. Preprint, 2026. The 39M-parameter 3D self-supervised comparator, fine-tuned. arXiv:2610.08400
4. Banaszewski, B.; Fitzgibbon, A. W. Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference. Preprint, 2026. The frozen encoder in the strongest in-context arm. arXiv:2608.18982
5. Grinsztajn, L.; Flöge, K.; Key, O.; et al. TabPFN-3: Technical Report. Preprint, 2026. The head of the Monroe arm and of both frozen Uni-Mol arms. arXiv:2605.13986
6. Burns, J. W.; Zalte, A. S.; Abreu, C. R. A.; et al. Deep Learning Foundation Models for Low-Data Regimes from Classical Molecular Descriptors. J. Chem. Inf. Model. 2026, articles ASAP. The pre-trained 2D D-MPNN comparator. 10.1021/acs.jcim.6c01546
7. Graff, D. E.; Morgan, N. K.; Burns, J. W.; et al. Chemprop v2: An Efficient, Modular Machine Learning Package for Chemical Property Prediction. J. Chem. Inf. Model. 2026, 66 (1), 28–33. 10.1021/acs.jcim.5c02332
8. Rogers, D.; Hahn, M. Extended-Connectivity Fingerprints. J. Chem. Inf. Model. 2010, 50 (5), 742–754. The LightGBM baseline's representation. 10.1021/ci100050t
9. OpenADMET; Expansion Therapeutics. OpenADMET–ExpansionRx Blind Challenge Data. Hugging Face, 2026, CC BY 4.0. 10.57967/hf/9687
10. Fang, C.; Wang, Y.; Grater, R.; et al. Prospective Validation of Machine Learning Algorithms for Absorption, Distribution, Metabolism, and Excretion Prediction: An Industrial Perspective. J. Chem. Inf. Model. 2023, 63 (11), 3263–3274. 10.1021/acs.jcim.3c00160
11. Ash, J. R.; Wognum, C.; Rodríguez-Pérez, R.; et al. Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery. J. Chem. Inf. Model. 2025, 65 (18), 9398–9411. 10.1021/acs.jcim.5c01609