Forecast population-level MFRM operating characteristics for one future design

Usage

predict_mfrm_population(
  fit = NULL,
  sim_spec = NULL,
  n_person = NULL,
  n_rater = NULL,
  n_criterion = NULL,
  raters_per_person = NULL,
  reps = 50,
  fit_method = NULL,
  model = NULL,
  maxit = 25,
  quad_points = 7,
  residual_pca = c("none", "overall", "facet", "both"),
  seed = NULL
)

Arguments

fit: Optional output from fit_mfrm() used to derive a fit-based simulation specification.
sim_spec: Optional output from build_mfrm_sim_spec() or extract_mfrm_sim_spec(). Supply exactly one of fit or sim_spec.
n_person: Number of persons/respondents in the future design. Defaults to the value stored in the base simulation specification.
n_rater: Number of rater facet levels in the future design. Defaults to the value stored in the base simulation specification.
n_criterion: Number of criterion/item facet levels in the future design. Defaults to the value stored in the base simulation specification.
raters_per_person: Number of raters assigned to each person in the future design. Defaults to the value stored in the base simulation specification.
reps: Number of replications used in the forecast simulation.
fit_method: Estimation method used inside the forecast simulation. When fit is supplied, defaults to that fit's estimation method; otherwise defaults to "MML".
model: Measurement model used when refitting the forecasted design. Defaults to the model recorded in the base simulation specification.
maxit: Maximum iterations passed to fit_mfrm() in each replication.
quad_points: Quadrature points for fit_method = "MML".
residual_pca: Residual PCA mode passed to diagnose_mfrm().
seed: Optional seed for reproducible replications.

Value

An object of class mfrm_population_prediction with components:

design: requested future design
forecast: facet-level forecast table
overview: run-level overview
simulation: underlying evaluate_mfrm_design() result
sim_spec: simulation specification used for the forecast
settings: forecasting settings
ademp: simulation-study metadata
notes: interpretation notes

Details

predict_mfrm_population() is a scenario-level forecasting helper built on top of evaluate_mfrm_design(). It is intended for questions such as:

what separation/reliability would we expect if the next administration had 60 persons, 4 raters, and 2 ratings per person?
how much Monte Carlo uncertainty remains around those expected summaries?

The function deliberately returns aggregate operating characteristics (for example mean separation, reliability, recovery RMSE, convergence rate) rather than future individual true values for one respondent or one rater.

If fit is supplied, the function first constructs a fit-derived parametric starting point with extract_mfrm_sim_spec() and then evaluates the requested future design under that explicit data-generating mechanism. This should be interpreted as a fit-based forecast under modeling assumptions, not as a guaranteed out-of-sample prediction.

Interpreting output

forecast contains facet-level expected summaries for the requested future design.
Mcse* columns quantify Monte Carlo uncertainty from using a finite number of replications.
simulation stores the full design-evaluation object in case you want to inspect replicate-level behavior.

What this does not justify

This helper does not produce definitive future person measures or rater severities for one concrete sample. It forecasts design-level behavior under the supplied or derived parametric assumptions.

References

The forecast is implemented as a one-scenario Monte Carlo / operating- characteristic study following the general guidance of Morris, White, and Crowther (2019) and the ADEMP-oriented reporting framework discussed by Siepe et al. (2024). In mfrmr, this function is a practical wrapper for future-design planning rather than a direct implementation of a published many-facet forecasting procedure.

Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074-2102.
Siepe, B. S., Bartoš, F., Morris, T. P., Boulesteix, A.-L., Heck, D. W., & Pawel, S. (2024). Simulation studies for methodological research in psychology: A standardized template for planning, preregistration, and reporting. Psychological Methods.

Examples

spec <- build_mfrm_sim_spec(
  n_person = 40,
  n_rater = 4,
  n_criterion = 4,
  raters_per_person = 2,
  assignment = "rotating"
)
pred <- predict_mfrm_population(
  sim_spec = spec,
  n_person = 60,
  reps = 2,
  maxit = 10,
  seed = 123
)
#> Warning: Optimizer did not fully converge (code = 1). Consider increasing maxit (current: 10) or relaxing reltol (current: 1e-06).
#> Warning: Optimizer did not fully converge (code = 1). Consider increasing maxit (current: 10) or relaxing reltol (current: 1e-06).
s_pred <- summary(pred)
s_pred$forecast[, c("Facet", "MeanSeparation", "McseSeparation")]
#> # A tibble: 3 × 3
#>   Facet     MeanSeparation McseSeparation
#>   <chr>              <dbl>          <dbl>
#> 1 Criterion          0.902          0.902
#> 2 Person             1.84           0.13 
#> 3 Rater              2.19           1.00