
Summarize a DIF/bias screening simulation
Source:R/api-simulation.R
summary.mfrm_signal_detection.RdSummarize a DIF/bias screening simulation
Usage
# S3 method for class 'mfrm_signal_detection'
summary(object, digits = 3, ...)Arguments
- object
Output from
evaluate_mfrm_signal_detection().- digits
Number of digits used when printing numeric summaries.
- ...
Reserved for generic compatibility.
Value
An object of class summary.mfrm_signal_detection with:
overview: run-level overviewdetection_summary: full-precision rates by design, with design-variable aliases when applicable.DIF*,DIFClass*andBiasScreen*columns retainPlanned,Available,Unavailable,Positive,MCLower,MCUpper,AllTrialsLowerandAllTrialsUpperfor target outcomes. Monte Carlo bounds are exact 95% binomial intervals conditional on availability; all-trial bounds describe unresolved outcomes, not confidence intervals.DIFNonTarget*/BiasNonTarget*retain summedPlannedandAvailablecell counts andAvailableReps, the number of trials contributing a non-target proportion. Those proportions receive equal weight across contributing trials, even with unequal cell counts; their MCSE is the sample SD of the proportions divided by the square root ofAvailableReps.TargetContrastAvailable,TargetContrastAbsAvailable,TargetBiasAvailableandTargetBiasTAvailableare denominators for the corresponding finite point summaries. All-missing means returnNA.ademp: simulation-study metadata carried forward from the original objectfacet_names: public facet labels carried from the simulation specificationdesign_variable_aliases: accepted public aliases for design variablesdesign_descriptor: role-based design-variable metadataplanning_scope: explicit record of the current planning contractplanning_constraints: explicit record of mutable/locked design variablesplanning_schema: structured planning metadatastructural_design_review: deterministic structural review of the named-facet design grid; it reports design bookkeeping rather than signal-detection performancegpcm_boundary:GPCMcaveat row when presentnotes: short interpretation notes, including the bias-side screening caveat
Examples
# One saved replication illustrates output structure, not precise rates.
# Regeneration recipe: examples/simulation-results.R in the package.
saved <- readRDS(system.file("examples", "simulation-signal-detection.rds",
package = "mfrmr", mustWork = TRUE))
# Inspect saved$generation_warnings and the result's run/status tables.
sig_eval <- saved$result
summary(sig_eval)
#> mfrmr Signal Detection Summary
#>
#> Overview
#> Designs Replications SuccessfulRuns ConvergedRuns MeanElapsedSec
#> 1 1 1 0 0.345
#>
#> Detection summary (preview)
#> design_id n_person n_rater n_criterion raters_per_person DIFTargetLevel
#> S01 8 2 2 2 C02
#> BiasTargetRater BiasTargetCriterion Reps ConvergenceRate McseConvergenceRate
#> R02 C02 1 0 NA
#> DIFPower McseDIFPower DIFClassificationPower McseDIFClassificationPower
#> NA NA NA NA
#> MeanTargetContrast McseTargetContrast TargetContrastAvailable
#> NA NA 0
#> TargetContrastAbsAvailable MeanTargetContrastAbs McseTargetContrastAbs
#> 0 NA NA
#> McseDIFFalsePositiveRate DIFFalsePositiveRate BiasScreenRate
#> NA NA 0
#> McseBiasScreenRate TargetBiasAvailable TargetBiasTAvailable MeanTargetBias
#> NA 1 1 0.156
#> McseTargetBias MeanAbsTargetBias McseAbsTargetBias MeanTargetBiasT
#> NA 0.156 NA 0.278
#> McseTargetBiasT BiasScreenMetricAvailabilityRate
#> NA 1
#> McseBiasScreenMetricAvailabilityRate McseBiasScreenFalsePositiveRate
#> NA NA
#> BiasScreenFalsePositiveRate DIFNonTargetAvailableReps DIFNonTargetPlanned
#> 0 0 1
#> DIFNonTargetAvailable BiasNonTargetAvailableReps BiasNonTargetPlanned
#> 0 1 3
#> BiasNonTargetAvailable MeanElapsedSec McseElapsedSec DIFPlanned DIFAvailable
#> 3 0.345 NA 1 0
#> DIFUnavailable DIFPositive DIFMCLower DIFMCUpper DIFAllTrialsLower
#> 1 0 NA NA 0
#> DIFAllTrialsUpper DIFClassPlanned DIFClassAvailable DIFClassUnavailable
#> 1 1 0 1
#> DIFClassPositive DIFClassMCLower DIFClassMCUpper DIFClassAllTrialsLower
#> 0 NA NA 0
#> DIFClassAllTrialsUpper BiasScreenPlanned BiasScreenAvailable
#> 1 1 1
#> BiasScreenUnavailable BiasScreenPositive BiasScreenMCLower BiasScreenMCUpper
#> 0 0 0 0.975
#> BiasScreenAllTrialsLower BiasScreenAllTrialsUpper
#> 0 0
#>
#> Structural design review
#> review_available n_designs recommended_design_id view mode surface
#> FALSE 0 <NA> public brief digest
#> table_component RecommendedAppendixTables CompactAppendixTables NumericTables
#> grid 4 3 6
#> AnyNumericTable
#> TRUE
#>
#> ADEMP metadata
#> - aims
#> - data_generating_mechanism
#> - estimands
#> - methods
#> - performance_measures
#>
#> Notes
#> - Some design conditions did not converge in every replication.
#> - DIF detection rates are unavailable. Missing test statistics and descriptive residual differences are not negative screening outcomes.
#> - DIF contrasts are reference minus focal: residual means are in score units; refit means require documented common-scale comparability. Neither is automatically the injected linear-predictor shift.
#> - Bias-side rates are screening summaries derived from `estimate_bias()` output and should not be interpreted as formal power or alpha-calibrated false-positive rates.
#> - Target rates condition on available screens. Planned/available counts, exact 95% Monte Carlo bounds and all-trial unresolved-outcome bounds are retained; these do not establish general diagnostic accuracy.
#> - Non-target rates average within-replication proportions over available cells; they are not the probability of any false flag across a family.
#> - MCSE columns summarize finite-replication uncertainty around the reported means and rates.
#> - Planning helpers vary one person count and two named non-person facet roles (Rater and Criterion). Estimation may contain additional facets, but planning and forecasting are limited to this role-based design.
#> - Current scalar-argument planning paths allow `n_person`, `n_rater`, `n_criterion`, and `raters_per_person` to vary subject to `raters_per_person <= n_rater`.
#> - Named-facet structural design metadata for person count, non-person facet counts, and assignments per person. These deterministic design summaries do not establish arbitrary-facet simulation support or parameter-recovery performance.
#> - The structural design review reports deterministic bookkeeping and conservative design guidance, not DIF/bias detection power.