Main-effects generalizability approximation for an MFRM design

Re-fits the rating data underlying an mfrm_fit as a crossed random-effects model Score ~ 1 + (1 | Person) + (1 | Facet1) + ... + Residual via lme4::lmer, and returns main-effect variance components, residual variance, and G / Phi coefficients under this simplified model. Interactions are not estimated separately. For a G-study including the supported interactions, use mfrm_g_study() with one or two common random facets; it also accepts a single score column.

Usage

mfrm_generalizability(
  fit,
  data = NULL,
  object_facet = "Person",
  random_facets = NULL,
  reml = TRUE,
  missing = c("error", "omit")
)

Arguments

fit: An mfrm_fit from fit_mfrm().
data: Optional data frame. When NULL, the rating data stored on fit$prep$data is used. Required columns are the selected facets and Score. Scores must be numeric or numeric character/factor labels; nonnumeric labels and infinite values are refused. Use NA for missing values. Facet labels must be nonblank; Score and Residual are reserved and cannot be facet names.
object_facet: Facet that plays the role of the "object of measurement" – typically "Person" (default).
random_facets: Character vector of other facets to treat as random conditions of measurement. The default uses Person and the configured MFRM facets, excluding object_facet. For example, when raters are the objects, persons remain a random condition.
reml: Logical, passed to lme4::lmer() (default TRUE).
missing: Either "error" (default) or "omit". Missing scores or selected facet values stop the analysis by default. Explicit omission fits only complete rows and records the excluded row positions and missing columns. Missingness in unselected columns does not exclude a row.

Value

An object of class mfrm_generalizability with:

variance_components: One row per random effect plus residual, with columns Source, Variance, ProportionVariance, and Identified. When Identified is false, the fitted allocation and its proportion are not uniquely determined by the source design.
coefficients: One-row data frame with G (generalizability coefficient, relative decision) and Phi (dependability coefficient, absolute decision), coefficient status labels, and the identification status of the fitted random-effects model. GReason and PhiReason explain missing coefficients; an empty reason means the point is calculable. ComponentIdentificationChecked records the design check.
design: Description of the crossed-random model. component_identification retains the covariance-kernel Gram matrix, rank, scaled eigenvalues, null space and numerical tolerance. fit_status retains the separate lme4 boundary/convergence assessment.
data_usage: Input source, omission policy, named counts (InputRows, UsedRows, ExcludedRows), excluded_rows, and missing_cells (InputRow, Column). Row positions refer to the supplied data, or stored fitted rows when data = NULL. They cannot recover rows previously removed during MFRM fitting. Counts also accompany the coefficient table and D-study projections, including tabular exports; GStudyDataSource identifies the scope of those counts.

Details

The decomposition is on the observed numeric Score scale. It does not use the fitted MFRM latent scale or estimate a latent ordinal G/Phi coefficient.

Interpretation

G is appropriate for relative decisions (rank-ordering persons): G = sigma2(p) / (sigma2(p) + sigma2(Residual)).
The reported Phi describes dependability for absolute decisions: Phi = sigma2(p) / (sigma2(p) + sigma2(facet main effects) + sigma2(Residual)), before D-study scaling. It does not estimate the probability of correct classification at a particular cut score.
Use mfrm_d_study() to project G / Phi under planned numbers of raters, items, criteria, or other random measurement facets.
Values of 0.70 and 0.80 are displayed as familiar planning references, not as universal decision rules. Required dependability depends on the decision, consequences, population, and evidence beyond a single coefficient.
For ordered categories, Score is treated as a numeric observed response in a Gaussian linear mixed model; thresholding is not modeled.

Component identification and boundary estimates

A zero variance estimate can be a valid constrained ML/REML solution. It is distinct from a design that cannot separate two variance components. For example, if every rater identifier always corresponds to the same task identifier, their main-effect covariance kernels coincide. Their sum may be identified while the individual allocations are not. isSingular() alone does not test this design property.

The design check uses linear independence of the observation covariance kernels (after projecting out the intercept for REML). Only variance combinations orthogonal to the numerical null space enter coefficients. Thus an identified sum can still supply Phi, while a future plan assigning unequal weights to its confounded components cannot. Individual fitted allocations remain available with Identified = FALSE for diagnosis. A full numerical rank does not establish correct model assumptions, optimizer accuracy or finite-sample precision. This is a linear design check; fitted boundary zeros are not used to resolve otherwise confounded components.

If object variance is zero and the relevant error variance is positive, G or Phi equals zero and is retained with boundary diagnostics. A zero total variance gives an undefined ratio and zero_total_variance. The historical IdentificationStatus field summarizes design and fit cautions; inspect design$component_identification and design$fit_status to distinguish them. This function does not supply coefficient intervals.

Limitations

This helper formulates the random-effects model with main effects only (Score ~ 1 + (1|Person) + (1|Facet1) + ... + Residual); no explicit (1 | Person:Rater), (1 | Person:Criterion), or (1 | Rater:Criterion) interaction terms are estimated. All interactions are omitted. The residual combines unexplained variation; omitted interactions may also affect the fitted main-effect components. Their separate variances and correct averaging rates are not recovered. This function reports the one-observation-per-cell baseline. mfrm_d_study() applies D-study scaling, including residual-scaling sensitivity checks, to the same simplified variance-component decomposition. Because person-by-facet interaction terms are not estimated separately, D-study projections remain practical planning evidence rather than a replacement for a fully specified G-theory design. Counts held constant in a D-study do not turn a random facet into a fixed-facet universe. Variance estimation uncertainty is not propagated to G/Phi. Component values and G/Phi are stored at full precision; rounding is only for display. Recreate older G/D results from the existing MFRM fit before reuse. Boundary or singular lme4 fits are retained as diagnostic evidence but are not treated as decision-ready G/D-study evidence. New results record component separation explicitly. Earlier saved G-studies without this record can still supply conditional D-study calculations, with ComponentIdentificationChecked = FALSE; recreate the G-study from the original ratings to obtain the check. This needs no MFRM refit.

Omission does not correct missing-data bias or identify why ratings are absent. Entirely absent assignments are not reconstructed. Review the rating design and missingness assumptions before interpreting G/D results. Earlier versions silently omitted incomplete rows and unparseable scores. To reproduce complete-row selection, clean invalid labels explicitly and choose missing = "omit". Older saved results remain usable when their calculation version is current, but unavailable row counts are not guessed; rerun the G-study with the original data to obtain row accounting.

References

Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01 . Gaussian ML/REML model and optimization. See also https://lme4.github.io/lme4/reference/isSingular.html for the meaning of a singular fitted covariance matrix.

Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. Wiley.

Brennan, R. L. (2001). Generalizability theory. Springer. doi:10.1007/978-1-4757-3456-0 . General G/D framework; the main-effects restriction and residual-scaling choices here are package-specific limits.

Examples

# Saved synthetic JML/RSM analysis of example_core.
# For your own data, fit a model first; see ?fit_mfrm.
example_fit <- readRDS(system.file("examples", "core-jml.rds", package = "mfrmr"))
fit <- example_fit$fit
if (requireNamespace("lme4", quietly = TRUE)) {
  gt <- mfrm_generalizability(fit)
  gt$variance_components
  # Look for: a Person variance component well above any single
  #   non-person facet's variance share. Large rater or criterion
  #   variance shares mean those conditions add measurement error
  #   relative to person spread.
  gt$coefficients
  # Compare G and Phi with study-specific requirements; 0.70 and 0.80
  #   are reference guides only. Phi < G means absolute decisions are noisier than relative
  #   decisions; review whether facet main effects need anchoring.
  # Always check IdentificationStatus before using the bands:
  gt$coefficients[, c("G", "Phi", "GStatus", "PhiStatus",
                      "IdentificationStatus")]
  gt$design$identification_note
  # If IdentificationStatus is not "identified", treat G/Phi as
  # design-review evidence rather than decision-ready reliability.
}
#> [1] "No lme4 boundary or singular fit was detected."