Summarize a design-simulation study
Usage
# S3 method for class 'mfrm_design_evaluation'
summary(object, digits = 3, ...)Arguments
- object
Output from
evaluate_mfrm_design().- digits
Number of digits used in the returned numeric summaries.
- ...
Reserved for generic compatibility.
Value
An object of class summary.mfrm_design_evaluation with components:
overview: run-level overviewdesign_summary: aggregated design-by-facet metricsademp: simulation-study metadata carried forward from the original objectnotes: short interpretation notes
Details
The summary emphasizes condition-level averages that are useful for practical design planning, especially:
convergence rate
separation and reliability by facet
severity recovery RMSE
mean misfit rate
Examples
sim_eval <- evaluate_mfrm_design(
n_person = c(30, 50),
n_rater = 4,
n_criterion = 4,
raters_per_person = 2,
reps = 1,
maxit = 15,
seed = 123
)
#> Warning: Optimizer did not fully converge (code = 1). Consider increasing maxit (current: 15) or relaxing reltol (current: 1e-06).
#> Warning: Optimizer did not fully converge (code = 1). Consider increasing maxit (current: 15) or relaxing reltol (current: 1e-06).
summary(sim_eval)
#> $overview
#> # A tibble: 1 × 5
#> Designs Replications SuccessfulRuns ConvergedRuns MeanElapsedSec
#> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 2 2 2 0 0.87
#>
#> $design_summary
#> # A tibble: 6 × 34
#> design_id Facet n_person n_rater n_criterion raters_per_person Reps
#> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 D01 Criterion 30 4 4 2 1
#> 2 D02 Criterion 50 4 4 2 1
#> 3 D01 Person 30 4 4 2 1
#> 4 D02 Person 50 4 4 2 1
#> 5 D01 Rater 30 4 4 2 1
#> 6 D02 Rater 50 4 4 2 1
#> # ℹ 27 more variables: ConvergenceRate <dbl>, McseConvergenceRate <dbl>,
#> # MeanSeparation <dbl>, SdSeparation <dbl>, McseSeparation <dbl>,
#> # MeanReliability <dbl>, McseReliability <dbl>, MeanInfit <dbl>,
#> # McseInfit <dbl>, MeanOutfit <dbl>, McseOutfit <dbl>, MeanMisfitRate <dbl>,
#> # McseMisfitRate <dbl>, MeanSeverityRMSE <dbl>, McseSeverityRMSE <dbl>,
#> # MeanSeverityBias <dbl>, McseSeverityBias <dbl>, MeanSeverityRMSERaw <dbl>,
#> # McseSeverityRMSERaw <dbl>, MeanSeverityBiasRaw <dbl>, …
#>
#> $notes
#> [1] "Some design conditions did not converge in every replication."
#> [2] "MCSE columns summarize finite-replication uncertainty around the reported means and rates."
#>
#> $ademp
#> $ademp$aims
#> [1] "Assess many-facet design conditions via repeated parametric simulation under explicit data-generating assumptions."
#>
#> $ademp$data_generating_mechanism
#> $ademp$data_generating_mechanism$source
#> [1] "scalar_arguments"
#>
#> $ademp$data_generating_mechanism$model
#> [1] "RSM"
#>
#> $ademp$data_generating_mechanism$step_facet
#> [1] NA
#>
#> $ademp$data_generating_mechanism$assignment
#> [1] "design_dependent"
#>
#> $ademp$data_generating_mechanism$latent_distribution
#> [1] "normal"
#>
#> $ademp$data_generating_mechanism$threshold_mode
#> [1] "generated_common"
#>
#> $ademp$data_generating_mechanism$threshold_step_facet
#> [1] NA
#>
#> $ademp$data_generating_mechanism$design_variables
#> [1] "n_person" "n_rater" "n_criterion"
#> [4] "raters_per_person"
#>
#>
#> $ademp$estimands
#> [1] "Facet separation, reliability, and strata"
#> [2] "Mean infit/outfit and misfit rate"
#> [3] "Aligned facet-parameter recovery RMSE and bias"
#> [4] "Convergence rate and elapsed time"
#>
#> $ademp$methods
#> $ademp$methods$fit_method
#> [1] "JML"
#>
#> $ademp$methods$fitted_model
#> [1] "RSM"
#>
#> $ademp$methods$maxit
#> [1] 15
#>
#> $ademp$methods$quad_points
#> [1] NA
#>
#> $ademp$methods$residual_pca
#> [1] "none"
#>
#>
#> $ademp$performance_measures
#> [1] "Mean performance across replications"
#> [2] "MCSE for means and rates"
#> [3] "Convergence rate"
#> [4] "Sparse-category warning rate"
#>
#>
#> attr(,"class")
#> [1] "summary.mfrm_design_evaluation"