Unimodal versus multimodal, on ADME data

Thirty-three ways of combining four molecular representations, over fifteen endpoints across two unrelated data sets, 25 replicate models each: 12,375 fitted configurations and 6,375 more for the controls. The design is Wasswa, Kajjumba and Ramsundar's¹; the endpoints, folds and splits are those of an earlier comparison¹⁰, so a fingerprint baseline and three D-MPNNs already scored on exactly these molecules sit on the same axis.

Their central finding replicates. Fusing modalities moves accuracy very little, and none of it survives a correction for multiple comparisons. What fusion does buy is calibration, and on the larger data set that buys enough to be significant.

Two things the paper could not see from inside its own design also show up. A single well-initialised graph network beats all thirty-three configurations on both data sets. And when the GNN modality is actually a learned representation -- in the released code it is not -- the ranking of which modality matters inverts.

Three results

1. Adding modalities does not reliably add accuracy

Best multimodal model against best unimodal model, per endpoint, is a coin flip: fusion is ahead on 7 of 9 ExpansionRx endpoints by a mean of +0.010 R squared, and on 4 of 6 Biogen endpoints by -0.019. Stepping up the modality ladder -- GNN+RDKit to three modalities to four, holding the strategy and the learner fixed -- separates in 0 of 24 comparisons on ExpansionRx and 0 of 24 on Biogen, after Holm correction. So does the choice of learner: 0 of 24 and 0 of 24.

On Biogen the single best configuration in the entire grid is a unimodal one, RF - RDKit at 0.348, ahead of every fusion model built on top of it.

2. Calibration is what fusion actually buys

This is the paper's other claim, and it holds up better than the accuracy one. Against the best unimodal model, the best multimodal model improves all three uncertainty measures on both data sets, and on ExpansionRx all three survive Holm correction: error-uncertainty correlation +0.044 (p = 0.027), expected calibration error -0.016 (p = 0.012), miscalibration area -0.021 (p = 0.012). On Biogen the same three move by similar amounts in the same direction at p = 0.094.

The reference methods make the size of that plain. The best fusion configuration reaches an error-uncertainty correlation of 0.229 on ExpansionRx and 0.303 on Biogen, where the best of the four reference methods manages 0.182 and 0.189. Fusion models know when they are wrong better than any single model here does, including the one that is most accurate.

3. A single well-initialised graph network beats the whole grid

ChemProp initialised from CheMeleon⁶ averages 0.407 R squared on ExpansionRx and 0.376 on Biogen. The best of the thirty-three configurations manages 0.324 and 0.348. It is not close, and it does not depend on which data set you look at.

That is the same shape of answer the earlier comparison¹⁰ got from a different direction: what makes a graph network win is a good initialisation, not the number of things bolted to it. Multimodal fusion is a way of combining fixed representations, and the ceiling on that is the quality of the representations being combined.

What was compared

Nine unimodal baselines and twenty-four fusion models: 33 configurations per endpoint per fold. The paper's fifth modality is MS2 fragmentation spectra; no ADME collection carries them, so that modality is dropped and nothing else about the design changes.

The 25 replicates come from five repeats of a five-fold GroupKFold over the training molecules, grouped by chemical cluster. Every configuration early-stops on the held-out fifth where it has a training loop to stop, and every one is scored on the same untouched test set. Statistics follow the protocol of Ash et al.⁷: distributions over folds, corrected for multiple comparisons, and no bold maxima.

Uncertainty, from the folds

The paper estimates epistemic uncertainty from three independently seeded models. Here the five folds within a repeat play that role: each test molecule is predicted five times, by five models fit on overlapping four fifths of the same training set. Five repeats give five ensembles per configuration rather than one, it costs nothing extra, and it means the reference methods can be scored on calibration without being rerun.

That expected calibration error compares an error magnitude directly against a standard deviation, so a perfectly calibrated Gaussian scores about 0.2 sigma rather than zero, and it carries the units of the endpoint. Read it within a panel, not against another paper.

This is a reimplementation, and two controls measure the difference

The authors' code² is MIT licensed and public, which is more than most releases offer. It is also a set of Colab dumps: hard-coded paths, six empty notebook stubs, scripts that consume globals left behind by other scripts, and no committed generator for the feature matrix everything loads. It cannot be run, so this is written from that source read as a specification together with the paper's Supporting Information.

Their GNN modality is not a learned representation

Their extractor hooks the first nn.Linear it finds by module order. DeepChem's AttentiveFP³ has no .ffn attribute, so the fallback lands on gnn.init_context.project_node[0] -- the projection applied to raw atom features before any message passing. What comes back is a 30-wide mean of unlearned atom features. Their own Table S3 records the modality as 30 features while describing it as a learned graph representation; the readout is 200 wide.

Running the LightGBM half of the grid on both blocks puts a number on it: the learned readout is worth +0.073 R squared on ExpansionRx and +0.040 on Biogen.

It also inverts the paper's modality ranking. They found RDKit descriptors indispensable and the GNN embedding nearly free to drop -- which is what you would expect if the GNN block were a mean of atom features already implied by the descriptors. With a real graph embedding, the GNN modality carries 49.9% of the fused model's SHAP attribution on ExpansionRx against RDKit's 4.6%, and removing it costs -0.170 R squared under early fusion where removing RDKit costs -0.009.

Their late fusion leaks, but the leak is small

Their base learners fit the whole training set and then predict that same training set, with no out-of-fold scheme anywhere in the release. With LightGBM base learners those predictions nearly interpolate, so the meta-learner is fit on optimistically biased inputs. Refitting the identical meta-learner on the fold's held-out fifth instead is worth +0.011 R squared on ExpansionRx and +0.013 on Biogen. Real, in the direction you would expect, and small enough that it is not what is holding late fusion back.

And both encoders early-stop here

Theirs train for a fixed 50 and 20 epochs with no validation monitoring, which they can afford because their protocol has no validation split. This one has one, and every other method in the comparison uses it. Hyperparameters otherwise follow their protocol unchanged: 60 sampled settings, mean squared error over three folds, searched once and reused for every replicate, with the inner folds grouped by cluster as the outer folds already are.

Early against late fusion

The one place the design axes separate at all. On ExpansionRx 4 of 12 early-against-late comparisons survive correction, and every one of them is the graph learner preferring late fusion, by between +0.170 and +0.344 R squared. On Biogen none of the twelve separate.

The reason is visible in the accuracy table: early fusion hands AttentiveFP several hundred standardised descriptor columns alongside its own readout, and the readout drowns. Late fusion hands it four numbers instead. Where the fused block is small, concatenation is harmless; where it is large, it costs more than it adds.

ExpansionRx

37 methods over 9 endpoints, 8,325 fold scores: the 33-configuration grid plus the four reference methods, every one of them fit on the same molecules.

The grid

The eight best configurations, and the reference methods

Does fusion beat the best single view?

Best against best, per endpoint. Fusion is ahead on 7 of 9 endpoints, by a mean of +0.010 R squared. The best multimodal configuration overall is AttentiveFP | Late | GNN + RDKit + Mol2Vec + SMILES at 0.324, against RF - RDKit at 0.276 for the best single view.

What an extra modality buys

Over endpoints, with Holm correction inside each family: 0 of 24 steps up the modality ladder separate, 4 of 12 early-against-late comparisons separate, and 0 of 24 learner-against-learner comparisons separate.

Which modalities the model needs, and which it uses

Uncertainty and calibration

Against a fingerprint baseline and three D-MPNNs

ChemProp initialised from CheMeleon averages 0.407 R squared here, against 0.324 for the best of the thirty-three configurations, 0.319 for a multitask D-MPNN and 0.209 for LightGBM on Morgan fingerprints.

Two controls

Late fusion, with the meta-learner moved off the molecules its base learners were fit on: +0.011 R squared on average.

The GNN modality as a 200-wide learned readout rather than the 30-wide mean of raw atom features the released extractor returns: +0.073 R squared.

What it costs

Biogen ADME

37 methods over 6 endpoints, 5,550 fold scores: the 33-configuration grid plus the four reference methods, every one of them fit on the same molecules.

The grid

The eight best configurations, and the reference methods

Does fusion beat the best single view?

Best against best, per endpoint. Fusion is ahead on 4 of 6 endpoints, by a mean of -0.019 R squared. The best multimodal configuration overall is AttentiveFP | Early | GNN + RDKit at 0.341, against RF - RDKit at 0.348 for the best single view.

What an extra modality buys

Over endpoints, with Holm correction inside each family: 0 of 24 steps up the modality ladder separate, 0 of 12 early-against-late comparisons separate, and 0 of 24 learner-against-learner comparisons separate.

Which modalities the model needs, and which it uses

Uncertainty and calibration

Against a fingerprint baseline and three D-MPNNs

ChemProp initialised from CheMeleon averages 0.376 R squared here, against 0.348 for the best of the thirty-three configurations, 0.335 for a multitask D-MPNN and 0.281 for LightGBM on Morgan fingerprints.

Two controls

Late fusion, with the meta-learner moved off the molecules its base learners were fit on: +0.013 R squared on average.

The GNN modality as a 200-wide learned readout rather than the 30-wide mean of raw atom features the released extractor returns: +0.040 R squared.

What it costs

References

1. Wasswa, J.; Kajjumba, G. W.; Ramsundar, B. Unimodal vs Multimodal Learning: A Systematic Evaluation of Fusion Strategies and Model Design for Molecular Property Prediction and Uncertainty Quantification. J. Chem. Inf. Model. 2026. doi:10.1021/acs.jcim.6c01878
2. Multimodal_Fusion, the authors' released code. MIT licensed. github.com/jwasswa2023/Multimodal_Fusion
3. Xiong, Z.; Wang, D.; Liu, X.; et al. Pushing the Boundaries of Molecular Representation for Drug Discovery with the Graph Attention Mechanism. J. Med. Chem. 2020, 63 (16), 8749-8760. doi:10.1021/acs.jmedchem.9b00959
4. Jaeger, S.; Fulle, S.; Turk, S. Mol2vec: Unsupervised Machine Learning Approach with Chemical Intuition. J. Chem. Inf. Model. 2018, 58 (1), 27-35. doi:10.1021/acs.jcim.7b00616
5. Graff, D. E.; Morgan, N. K.; Burns, J. W.; et al. Chemprop v2: An Efficient, Modular Machine Learning Package for Chemical Property Prediction. J. Chem. Inf. Model. 2026, 66 (1), 28-33. doi:10.1021/acs.jcim.5c02332
6. Burns, J. W.; Zalte, A. S.; Abreu, C. R. A.; et al. Deep Learning Foundation Models for Low-Data Regimes from Classical Molecular Descriptors. J. Chem. Inf. Model. 2026. doi:10.1021/acs.jcim.6c01546
7. Ash, J. R.; Wognum, C.; Rodríguez-Pérez, R.; et al. Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery. J. Chem. Inf. Model. 2025, 65 (18), 9398-9411. doi:10.1021/acs.jcim.5c01609
8. Fang, C.; Wang, Y.; Grater, R.; et al. Prospective Validation of Machine Learning Algorithms for ADME Prediction. J. Chem. Inf. Model. 2023, 63 (11), 3263-3274. doi:10.1021/acs.jcim.3c00160
9. OpenADMET-ExpansionRx Blind Challenge data. Contributed by Expansion Therapeutics, CC BY 4.0. huggingface.co/datasets/openadmet
10. expansion-ml-comparison, the folds, the splits and the four reference methods. model-validation-central/studies/expansion-ml-comparison