The review 03: the machine-learning/statistics expert (an independent review, 2026-09-28)

The reviewer persona: a senior researcher in matrix factorization / topic models / variational inference / conformal prediction / compositional data analysis, a methodological reviewer, rejection-first. The overall verdict: Major revision (if a paper) / the design is salvageable (if an engineering archive). The statistical claims of the four flagship MSU methods are generally 1–2 grades stronger than what the archive actually specifies — downgrade every adjective in the §5 table (“calibrated”, “distribution-free”, “lossless”, “first”) to the level the protocols actually support, and the table survives.

P0

1.P0 MSU-LowCount’s conformal “distribution-free coverage guarantee” is mathematically untenable in the cross-TMB scenario

• Split conformal requires only that the calibration and the test data are exchangeable; the total mutation count N of panel/ctDNA samples spans 1–2 orders of magnitude, and the residual distribution depends on N as ~1/√N — N=50 and N=500 are not exchangeable, the standard marginal interval is dominated by the high-N side and is systematically miscalibrated at the low-N end. “distribution-free” means free of the generative distribution, not free of exchangeability; if the calibration set comes from an NB simulator, the guarantee transfers only when “simulation = reality”. Neither the nonconformity score nor the prediction target (fraction? counts? presence set?) is defined.
• The fix: Mondrian conformal by TMB bin (a stratified per-stratum guarantee) + an N-scaled/studentized score; the prediction target and the score definition written into §5; the claim rewritten as “per-TMB-stratum conformal intervals with empirical coverage verified on stratified simulations and retrospective real panel data”; the quantile tables pre-computed per bin and shipped with the package + a dispersion-misspecification sensitivity curve.

2.P0 MSU-Fit’s coverage-calibrated CI: the four matters — estimand, interval type, boundary mass, calibration leakage — are all undefined

• 1. The bootstrap distributions of the fraction vs the absolute-count estimands are wholly different; (b) structural zeros → the percentile bootstrap coverage degenerates by construction, a strong bias regime for small exposures; (c) the percentile/basic/BCa boundary behaviors differ hugely, and the archive only writes “bootstrap CI”; (d) post-selection: the CI is support-conditional, the unconditional coverage is contaminated by the selection effect (winner’s curse), so the calibration must target the whole procedure (the LRT selection + the CI + the zeroing); (e) after zeroing the output is no longer a CI — label the truncation honestly or fold it into the point estimate and recalibrate; (f) there is no evidence that synthetic calibration transfers to real overdispersion/batch.
• The fix: estimand = the support-conditional absolute exposure (also report the compositional version); BCa or a calibrated-percentile stratified by N×Shannon entropy; the calibration object is the whole procedure, reporting the coverage as curves over N and flatness; a real-data sanity protocol (down-sample a high-TMB sample → does the CI cover the full-volume estimate at the nominal frequency); the claim downgraded to “empirically coverage-calibrated in simulation under NB-calibrated noise, with sensitivity to dispersion misspecification”.

3.P0 MSU-Corr: the log-linear embedding carries an exact gauge unidentifiability; the m-selection protocol is missing; the dual regularization is unargued

• Exact non-identifiability: for any c∈R^m, (α,L,U)→(α−Lc, L, U+c) leaves h unchanged, plus the (LQ,UQ) rotation freedom — the embedding vectors themselves are not interpretable, the Gaussian prior only selects a representative point; the identifiability of W depends on the strong structural assumption “log H low-rank + sparse-leaning”, and no theorem guarantees where the misspecification bias goes (the Cornet evidence is a synthetic benchmark, not an identifiability theorem); the MvNMF volume regularization by contrast has a Leplat–Gillis theorem lineage — the “joint” of the two carries no joint theory or ablation; the m=K default makes the cross-K held-out comparison happen between model classes of differing flexibility, in direct collision with SUITOR’s fixing of K; σ²=mean((L,U)²) plus the prior penalty form a self-referential feedback, and the stability of the L=U=0 stationary point is unknown.
• The fix: (a) the paper reports only gauge-fixed quantities (h, W, the reconstruction), direct interpretation of the embedding vectors forbidden; (b) m selected via held-out likelihood (reusing the entry-wise mask) or W stability, with m fixed during the K sweep; (c) the volume regularization and the embedding reported as separate ablation axes; (d) add the baseline matrix: plain KL-NMF on the recalibrated catalog, pure MvNMF, exact Cornet, correlation-aware refit, all compared under the Wu 2022 generator.

P1

4.P1 The multi-evidence K-selection table has no decision rule; the Wilcoxon is a pseudo-replication test; the rectangular-Hungarian semantics are undefined

• When the three evidences conflict the pipeline cannot proceed; the MuSiCal-style adjacent-K Wilcoxon treats the 50 replicates sharing the same data as independent evidence — the p-value is anti-conservative, usable only as a diagnostic; under the SBS5/40 near-duplicate clusters the cosine silhouette (which does not satisfy the triangle inequality) is unreliable; when the replicates’ effective Ks disagree, whether an unmatched pair counts as a split or a merge is undefined; the threshold calibration of the ARD 17/20 mis-reports goes unmentioned.
• The fix: a hierarchical decision rule (SUITOR argmin as the main rule → the stability veto → the Wilcoxon demoted to diagnostics); the Hungarian-unmatched semantics defined explicitly (split/merge counted separately, compatible with the Islam threshold sweep); the meta-rule itself evaluated over the full grid (the meta-rule can also overfit the benchmark).

5.P1 Benchmark: calibration-leakage channels and an adversarial-scenario gap; the greedy protocol rewards signature splitting

• The LTH/λ̃/σ²/conformal quantile/CI calibration/ARD thresholds are all calibrated on synthetic data — if that uses the same generator (or even the same seed family) as the evaluation, the benchmark is a training-set report; the hardest realistic cell is the joint adversarial one (catalog misspecification + high correlation + flat signatures + low TMB at once); under the Islam greedy protocol, splitting one true signature into two ≥0.90 near-duplicates counts both as TP — the split is rewarded.
• The fix: three-layer isolation — the calibration scenario, the model-selection scenario, the evaluation scenario (different generators: SigProfilerSimulator/SynSig/PCAWG-anchored realistic mixtures); add the joint-adversarial cell and the off-catalog-signature cell to the evaluation grid; the primary metric = the Hungarian one-to-one, the greedy + threshold sweep as a robustness appendix; an explicit penalty or a dedup statistic for the splitting behavior; freeze the protocol before publication (pre-registration style) and ship it with the package.

6.P1 The Fisher information interval degenerates at the boundary, and “LRT selects the support + Fisher CI on the support” is post-selection inference

• Wald/Fisher are invalid at the simplex boundary (negative lower bound + illusory precision), and are not the same thing as the LRT’s χ²₁ (boundary-asymptotic, correct) — the archive listing the two side by side as “dual modes” creates an illusion.
• The fix: the Fisher interval confined to interior support and explicitly labeled as conditional; the boundary gets LRT + bootstrap across the board; “boundary parameters get tests, not intervals” written into the documentation.

7.P1 The MSU-Catalog IR’s “lossless projection SBS96↔︎1536” bidirectional arrow fails in one direction

• 1536→96 is lossless aggregation; 96→1536 is unidentifiable at the catalog level (unless one goes back to variant-level recomputation); ID89 is a re-defined classification scheme, and the 83→89 mapping is not necessarily well-defined (the partitions do not refine one another).
• The fix: the wording becomes “lossless coarse-graining (fine→coarse) + variant-level recomputation (coarse→fine requires variants)”; an explicit ID83↔︎89 mapping table + labeling the unmappable channels; audit every bidirectional arrow pair by pair (SBS192/288 likewise suffer the missing chain information).

8.P1 MSU-Tensor: BIC is unreliable for latent-factor models, and tensor identifiability goes not even one word mentioned

• BIC is known to be inconsistent on latent-factor models (the parameter count ≠ the effective degrees of freedom); the standard replacements are Bai–Ng-type criteria or a held-out predictive likelihood; the Kruskal uniqueness condition (Σk_ranks ≥ 2R+2) is never discussed; the precedent TensorSignatures already died, and there is no structural reason the same model-complexity trap would not be re-fallen into.
• The fix: contract the scope — K migrated fixed from the SBS analysis, one new modality added at a time; the K/rank selection becomes a held-out predictive likelihood + stability, the BIC demoted to diagnostics; give the Kruskal condition check for the rank configuration; or honestly position it as an exploratory engine.

P2

9.P2 The zero channels and the ε strategy of the KL pipeline are unwritten

• The real mine is the 0/0 guard in the MU updates and the musicatk-style “+ε pseudo-count” (a low TMB systematically lifts the sparse channels and pollutes the exposure). The fix: an explicit “no pseudo-count at the catalog layer + NNDSVDa positive initial values + numerical guards enter only the divisions”, and write “no pseudo-count” into the comparison test against musicatk.

10.P2 The β≠1 “three-noise model” has no selection protocol

• β=1 as the default (= the multinomial MLE) and declared as such; if the β knob is kept, provide a CV/profile protocol; otherwise delete the marketing language. The overdispersion is handled by the NB (the fitting side), and the division of duties written clearly.

11.P2 MSU-Infer does not handle structural zeros

• The DM is natural for sampling zeros, but needs a hurdle/two-point distribution for structural zeros; Morrill 2025 does not solve it either — add zero-inflation/hurdle to the extension list explicitly; land the uncertainty propagation as a multiple-imputation protocol (a measurement-error regression over the bootstrap draws).

12.P2 The SUITOR replication’s fidelity

• The fold assignment by index modulo (not random — for a genome-clustered real catalog that is exactly the original design) and the ECM semantics of the conditional-mean imputation must be replicated bit for bit, otherwise the CV error is not comparable to the published calibration; the K×10 folds×300 restarts cost budget into the documentation + a golden cross-check against the original R package.

The conclusion and the highlights

• Major revision / the design is salvageable: the architecture layering, the S7 registry, the licensing discipline and the protocol replication are first-class; the problems concentrate in “the gap between the statistical claims and the statistical protocols”, all solvable by “rewrite the claim + supply the protocol + supply the baseline”, with no need to overturn the architecture.
• The three things done well: the term-level and protocol-level source-grade honesty (proactively correcting that MWPruning/MDL do not exist); the identifiability awareness goes the right direction (the volume regularization and the Cornet two legs + the degenerate property test); the explicit won’t-do list and the boundary discipline.