Skip to contents

Comparing signatures and exposures

Source: vignettes/comparing-signatures.Rmd
comparing-signatures.Rmd

ms_compare() is the calibrated-similarity face of the package: it matches two signature sets under a chosen protocol, scores each match with a similarity metric, and attaches null distributions so a cosine is never read without its sampling context. This guide covers the comparison container, the exposure-level group test, and the clustering/embedding post-fitting analyses.

Two sets of signatures

A synthetic cohort generated under two exposure programs, extracted de novo, and compared against the generating reference — the everyday “did we recover the processes?” question:

ref <- ms_refdb_bundled()$SBS_GRCh38@matrices[[1]]
S   <- ref[, c(1, 5, 9)]
colnames(S) <- paste0("CS", 1:3)
E1  <- matrix(rep(c(0.5, 0.3, 0.2), each = 12), 3, 12)
E2  <- matrix(rep(c(0.2, 0.5, 0.3), each = 12), 3, 12)
E   <- cbind(E1, E2)
sim <- ms_simulate(S, exposures = E, burden = 3000, seed = 3)
ext <- suppressWarnings(ms_extract(sim, k = 3))

The comparison container

ms_compare() takes the estimated and reference signature matrices (the channel rows must match), the two exposure matrices for the activity protocol, and the null-distribution controls. The result is an MsComparison — a metrics table keyed by signature pair, with raw and BH-adjusted null probabilities:

cmp <- ms_compare(ext@signatures, S,
                  exposures_est = ext@exposures, exposures_ref = E,
                  null_n = 200, null_n_draws = 500, seed = 1L)
cmp
#> <MsComparison> metrics: cosine,p_null,q_bh | protocols: hungarian+islam+jiang

The metrics table carries the cosine per matched pair plus the p_null and q_bh columns; the protocol field names the match protocol family applied. metric selects the similarity measure (cosine by default), and protocol switches between the Hungarian assignment, the Islam greedy threshold sweep, and the Jiang activity protocol. Two practical notes: the Jiang activity face is only defined when both exposure matrices are supplied, and the default run evaluates the full protocol family — so a signatures-only comparison (no exposures) must restrict the protocol, e.g. to the Hungarian assignment:

cmp_isl <- ms_compare(ext@signatures, S, protocol = "islam",
                      exposures_est = ext@exposures, exposures_ref = E,
                      null_n = 200, null_n_draws = 500, seed = 1L)
cmp_isl
#> <MsComparison> metrics: cosine,p_null,q_bh | protocols: islam

The null faces: an analytic null (catalog-shape conditioned) and an empirical Monte-Carlo null controlled by null_n, null_n_draws, and the seed (the seed is gated to the same bounded range as every other FFI face). Comparing two extracted sets — not one against a reference — uses the same call with the second matrix in the reference position.

Group differences in exposure 

cmp_min <- ms_compare(ext@signatures, S, protocol = "hungarian",
                      null_n = 200, null_n_draws = 500, seed = 1L)
cmp_min
#> <MsComparison> metrics: cosine,p_null,q_bh | protocols: hungarian

ms_exposure_test() asks a per-signature question: does this signature’s activity differ between two sample groups? It returns a tidy table with a raw and a BH-adjusted Wilcoxon-style test per signature:

grp <- factor(rep(c("early", "late"), each = 12))
et  <- ms_exposure_test(ext@exposures, grp)
et
#>   signature median_group1 median_group2 statistic     p_raw     p_bh pass_bh
#> 1      Sig1    0.01102982   0.004305302      78.5 0.6961547 0.953075   FALSE
#> 2      Sig2    0.40405818   0.368587940      73.0 0.9530750 0.953075   FALSE
#> 3      Sig3    0.00000000   0.057513455      62.0 0.5287333 0.953075   FALSE

A signature with pass_bh = TRUE under a real design is a candidate etiological or temporal difference — the test is diagnostic, and its assumptions belong to the exposure scale you pass in.

Clustering and embedding

Two post-fitting analyses round out the comparison family. ms_cluster_signatures() groups signatures whose pairwise similarity exceeds a threshold (connected components over the threshold graph) and names a medoid per cluster:

cl <- ms_cluster_signatures(ext@signatures, threshold = 0.9)
cl
#>   signature cluster is_medoid cluster_size
#> 1      Sig1      C1      TRUE            1
#> 2      Sig2      C2      TRUE            1
#> 3      Sig3      C3      TRUE            1

ms_embed() projects the sample × signature exposure matrix into a low-dimensional space (k-means by default; UMAP when the suggested package is installed):

em <- ms_embed(ext@exposures, method = "kmeans", clusters = 2, seed = 1)
head(em)
#>      sample      embed_1   embed_2 cluster
#> sim1   sim1 0.0376281971 0.0000000       2
#> sim2   sim2 1.0000000000 0.0000000       2
#> sim3   sim3 1.0000000000 0.0000000       2
#> sim4   sim4 0.2509873319 0.7490127       1
#> sim5   sim5 0.0000000000 0.9889033       1
#> sim6   sim6 0.0003847709 0.9996152       1

Both results have dedicated figure types — plot_similarity_heatmap(), plot_exposure_heatmap(), and plot_embedding() — shown in the visualization gallery.

Where to go next