Drives every (scenario x engine) cell of the grid through the production extraction face and scores it: Hungarian precision/recall/ F1 at the 0.90 line, the D2 paired-L1 (absolute and compositional) on the tau-independent Hungarian assignment, and the extraction runtime. PROVISIONAL defaults per the design memo (D1 stage 1: in-house certified engines; D2: Hungarian-paired L1) – pending PI confirmation.
Arguments
- grid
A named list of [ms_benchmark_grid()] scenarios.
- engines
A named list of extract-mode engine specs (e.g. `list(nmf = ms_nmf())`); NULL = the three certified in-house defaults. Non-certified specs warn once through the registry.
- seed
Master seed (the scenario dictionary draw and the truth exposures derive from it deterministically).
Details
Single-cell isolation: extraction AND scoring live in one guarded block – a failure in either records a `cell_error` metric for that cell and the grid continues. Scenario-level generation failures (ms_simulate / the dictionary draw) abort the run: the v1 isolation boundary is the cell, not the scenario.
Create specs with [ms_benchmark_grid()], execute with `ms_run_benchmark()`, store via [ms_benchmark()] – the trio stays three separate functions by the U-M7-04 ergonomics ruling (no generic merge).
See also
Other benchmark:
ms_benchmark(),
ms_benchmark_grid()