`ms_compare()` scores an estimated dictionary against a reference one: the pairwise similarity metrics on share-normalized columns, the three matching protocols (Hungarian one-to-one primary, Islam greedy, Jiang activity) and the null-distribution evaluations that calibrate the cosine ("zero-calibration": better than chance at this channel count and burden). The Jiang protocol additionally needs the exposure matrices (activity > 0 matching has no cosine face).
Usage
ms_compare(
estimated,
reference,
metric = "cosine",
protocol = c("hungarian", "islam", "jiang"),
thresholds = seq(0.8, 0.9, by = 0.01),
exposures_est = NULL,
exposures_ref = NULL,
catalogs = NULL,
signatures_est = NULL,
burden = NULL,
null_n = .ms_compare_defaults$null_n,
null_quantiles = .ms_compare_defaults$null_quantiles,
seed = 1L,
null_n_draws = 2000L
)Arguments
- estimated
m x k_est numeric matrix of signatures (columns sum to 1 after normalization; zero columns never match), or an [MsSignature].
- reference
m x k_ref numeric matrix with the same channel count, or an [MsSignature].
- metric
character vector among "cosine", "correlation", "jsd", "hellinger".
- protocol
character vector among "hungarian", "islam", "jiang".
- thresholds
numeric vector of match thresholds for the Hungarian/Islam sweeps (default seq(0.80, 0.90, by = 0.01), the robustness appendix convention).
- exposures_est, exposures_ref
optional k x n count-scale activity matrices (Jiang protocol; required when it is requested).
- catalogs
optional m x n count matrix (Jiang reconstruction flag; also the default burden source for the catalog null).
- signatures_est
optional m x k_est matrix for the Jiang reconstruction flag (the fit's signature dictionary).
- burden
positive burden for the catalog null; default = the mean truth burden (colSums of the reference exposures or catalogs), and the catalog null is skipped when neither is available.
- null_n
draws per null family (>= 200; default 20000).
- null_quantiles
quantiles reported for each null, in (0, 1).
- seed
master seed of the null Monte Carlo (PCG64 layout v1; identical seeds are bitwise-reproducible).
- null_n_draws
Draws per pairwise empirical null p (>= 200; default 2000) – the add-one tail estimate reported in the `p_null`/`q_bh` metric matrices.
Examples
# Compare estimated signatures against the truth that generated them,
# with the Jiang protocol calibrated by a small null Monte Carlo.
ref <- ms_refdb_bundled()$SBS_GRCh38@matrices[[1]]
S <- ref[, c("SBS1", "SBS5", "SBS13")]
colnames(S) <- paste0("CS", 1:3)
E <- matrix(rep(c(0.5, 0.3, 0.2), each = 24), 3, 24)
sim <- ms_simulate(S, exposures = E, burden = 4000, seed = 7)
ext <- ms_extract(sim, k = 3)
#> Warning: K selection is undecidable: no rank passes the stability veto; falling back to the CV argmin (evidence table attached to the result)
#> ℹ the stability veto rejected every rank in the evaluated grid
#> falling back to k = 3
#> call ms_select_k() explicitly for the full evidence table
cmp <- ms_compare(ext@signatures, S,
exposures_est = ext@exposures, exposures_ref = E,
null_n = 200, null_n_draws = 500, seed = 1L)
round(cmp@metrics$cosine, 3)
#> CS1 CS2 CS3
#> Sig1 0.736 0.283 0.024
#> Sig2 0.441 0.249 0.038
#> Sig3 0.111 0.159 0.734