Causal inference on a learned embedding · anchored series · the paper's tasks and two ADME data sets

A counterfactual from one measured analog, against the encoder it sits on

Causal-Chemprop¹ fits a structural causal model to the last hidden layer of a trained Chemprop² and, given one measured member of a scaffold series, predicts the rest of the series as counterfactuals of it. Its paper reports better within-series ranking than Chemprop on eleven kinase scaffold groups. With the authors' released checkpoints that reproduces: Table 1 to two decimals in 65 of 66 values, ahead of their Chemprop on 9 of 11 tasks. It does not survive retraining. Refit end to end with the authors' own code, it loses within-task Spearman ρ on 10 of 11 tasks, by a median 0.61. And fitted per fold and per series on ExpansionRx and Biogen, its within-series ρ is behind that of its own encoder on 152 of 182 series on ExpansionRx and 20 of 22 series on Biogen, and behind Monroe + TabPFN 3^{5, 6} on 159 of 182 on ExpansionRx and 22 of 22 on Biogen.

The paper's eleven tasks

The release ships the eleven ABL1, AURKA and SRC tasks with their anchors, the trained encoders and the fitted SCMs. Run through the authors' own functions they give Table 1 to two decimals in 65 of 66 values; the exception is chemprop retrained on ABL1_2, ρ −0.08 against a printed 0.08, which looks like a dropped minus sign. The other arms are fitted here on the same training sets and anchors; +anchor adds the anchor to the training data. The table is within-task Spearman ρ, the paper's metric; the bottom row is how many tasks the released Causal-Chemprop is ahead and behind, and a Wilcoxon signed-rank test over the tasks.

Against the arms the paper ran, the released model holds up: ahead of its Chemprop on 9 of 11 tasks and of the retrained Chemprop on 8. Against arms it did not run the margin is similar in count and weaker in evidence: ahead of Monroe + TabPFN 3 on 8 (p = 0.12), of Monroe with the anchor on 9 (p = 0.08) and of fine-tuned Atom-JEPA on 7 (p = 0.15). With 7 to 22 molecules per task, few of these tests could be significant either way.

Refitting the release

The released numbers come from one trained encoder and one fitted SCM per task. Two refits, both through the authors' train, separate them. Refitting only the SCM on the released encoder (SCM refit) keeps most of the result: the released model is ahead of it on 4 and behind on none of 11 tasks. Retraining the encoder as well (encoder + SCM refit) does not: within-task ρ falls on 10 of 11 tasks, by a median 0.61, and the refit model is behind the released Chemprop on 9 of 11. Mean ρ over the 11 tasks is 0.51 released and −0.10 refit. The authors' training is seeded and set deterministic, but it does not regenerate their checkpoints here, and a new encoder is a new embedding: the SCM, the contributing nodes and the counterfactuals all change with it. The result in Table 1 belongs to those particular checkpoints.

ExpansionRx and Biogen

On 8 ExpansionRx endpoints⁹ there are 182 series with enough analogs, and on 4 Biogen endpoints¹⁰ 22. Each series is scored separately in each fold, and the table below counts series, with each series' ρ averaged over its folds: the comparison on the paper's own terms.

Causal-Chemprop is behind its own encoder on 152 of 182 ExpansionRx series and 20 of 22 Biogen series. The other ranking arms are further ahead of it, Monroe + TabPFN 3 most of all; giving Monroe the anchors in context changes its ρ by +0.009 on ExpansionRx and +0.002 on Biogen on average.

Tukey HSD over folds

The protocol's view, per endpoint, the mean within-series ρ of a fold as the metric. A method is on top if Tukey's correction cannot separate it from the best mean. With 5 folds the intervals are wide, and still Causal-Chemprop is on top of 0 of 8 ExpansionRx endpoints and 1 of 4 Biogen endpoints.

ExpansionRx

Biogen ADME

Where the ranking goes

The fit diagnostics say which of the authors' rules produced each series' predictions. When the SCM's plain prediction is used, for every molecule or through the low-variance fallback, Causal-Chemprop ranks about as well as the encoder underneath it. When every molecule gets the counterfactual, the ranking is close to random: ρ −0.02 against the encoder's 0.65 on ExpansionRx and 0.12 against the encoder's 0.69 on Biogen on the same series. The counterfactual pulls every member of a series towards the anchor and does not keep their order. No node clears the paper's contribution threshold in 47% of ExpansionRx fits and 1% of Biogen fits; there the authors' fallback chooses the nodes.

What the anchor buys

One measured analog is a noisy correction. Shifting Monroe + TabPFN 3 by its error on the anchor raises its mean MAE, by 0.051 on ExpansionRx and 0.118 on Biogen: the anchor's own error is part noise and part where it sits in the series, and the shift adds both to every member. Putting the anchors in its context lowers mean MAE instead, by 0.019 on ExpansionRx and 0.004 on Biogen. Causal-Chemprop has a higher mean MAE than its own encoder, shifted or not, on both data sets: its counterfactuals are worse in absolute terms as well as in rank. MAE is in the log units of each endpoint.

ExpansionRx

Biogen ADME

What it costs

Every series needs its own SCM in every fold, each on one CPU core: DAGMA on the analog set, a mechanism per node, then intrinsic causal contributions by Shapley sampling, which is most of the time. The encoders, one per endpoint and fold, are the cheap part. The fits ran on two machines, a 32-core workstation with an RTX 5070 Ti and a DGX Spark, in 527 CPU-core hours and 5 GPU hours.

What this does and does not say

Reported the way “Even More Thoughts on ML Method Comparisons” and the protocol paper it points at¹¹ argue comparisons should be: distributions over folds and series, simultaneous confidence intervals, paired tests, and no bolded maxima. Where two methods cannot be separated, both are reported as tied.

Code, per-fold and per-series metrics and every figure: studies/causal-chemprop.

References

1. Natajaya, C.; Attia, L.; Burns, J. W.; Doyle, P. S. Causal-Chemprop: Causal Machine Learning for Molecular Property Prediction and Design. J. Chem. Inf. Model. 2026. The method this page puts to the test, run with the authors' code and released checkpoints. 10.1021/acs.jcim.6c01429
2. Graff, D. E.; Morgan, N. K.; Burns, J. W.; et al. Chemprop v2: An Efficient, Modular Machine Learning Package for Chemical Property Prediction. J. Chem. Inf. Model. 2026, 66 (1), 28–33. The encoder Causal-Chemprop is built on. 10.1021/acs.jcim.5c02332
3. Bello, K.; Aragam, B.; Ravikumar, P. DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization. Advances in Neural Information Processing Systems 35, 2022. The structure learner. arXiv:2209.08037
4. Blöbaum, P.; Götz, P.; Budhathoki, K.; Mastakouri, A. A.; Janzing, D. DoWhy-GCM: An Extension of DoWhy for Causal Inference in Graphical Causal Models. J. Mach. Learn. Res. 2024, 25 (147), 1–7. The causal mechanisms and the intrinsic causal contributions. arXiv:2206.06821
5. Banaszewski, B.; Fitzgibbon, A. W. Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference. Preprint, 2026. The frozen encoder of the strongest arm on this site. arXiv:2608.18982
6. Grinsztajn, L.; Flöge, K.; Key, O.; et al. TabPFN-3: Technical Report. Preprint, 2026. The in-context head of the Monroe arms. arXiv:2605.13986
7. Petersen, K. H.; Tirsgaard, R. H.; Cornet, F. R. J.; Jordahn, M.; Schmidt, M. N. Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems. Preprint, 2026. The fine-tuned 3D encoder, run with the authors' code. arXiv:2610.08400
8. Burns, J. W.; Zalte, A. S.; Abreu, C. R. A.; et al. Deep Learning Foundation Models for Low-Data Regimes from Classical Molecular Descriptors. J. Chem. Inf. Model. 2026, articles ASAP. 10.1021/acs.jcim.6c01546
9. OpenADMET; Expansion Therapeutics. OpenADMET–ExpansionRx Blind Challenge Data. Hugging Face, 2026, CC BY 4.0. 10.57967/hf/9687
10. Fang, C.; Wang, Y.; Grater, R.; et al. Prospective Validation of Machine Learning Algorithms for Absorption, Distribution, Metabolism, and Excretion Prediction: An Industrial Perspective. J. Chem. Inf. Model. 2023, 63 (11), 3263–3274. 10.1021/acs.jcim.3c00160
11. Ash, J. R.; Wognum, C.; Rodríguez-Pérez, R.; et al. Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery. J. Chem. Inf. Model. 2025, 65 (18), 9398–9411. 10.1021/acs.jcim.5c01609