Facet-quality dashboard for facet-level screening

Build a compact dashboard for one facet at a time, combining facet severity, misfit, central-tendency screening, and optional bias counts.

Usage

facet_quality_dashboard(
  fit,
  diagnostics = NULL,
  facet = NULL,
  bias_results = NULL,
  severity_warn = 1,
  misfit_warn = 1.5,
  central_tendency_max = 0.25,
  bias_count_warn = 1L,
  bias_abs_t_warn = 2,
  bias_abs_size_warn = 0.5,
  bias_p_max = 0.05
)

Arguments

fit: Output from fit_mfrm().
diagnostics: Optional output from diagnose_mfrm().
facet: Optional facet name. When NULL, the function tries to infer a rater-like facet and otherwise falls back to the first modeled facet.
bias_results: Optional output from estimate_bias() or a named list of such outputs. Non-matching bundles are skipped quietly.
severity_warn: Absolute estimate cutoff used to flag severity outliers.
misfit_warn: Mean-square cutoff used to flag misfit. Values above this cutoff or below its reciprocal are flagged.
central_tendency_max: Absolute estimate cutoff used to flag central tendency. Levels near zero are marked.
bias_count_warn: Minimum flagged-bias row count required to flag a level.
bias_abs_t_warn: Absolute t cutoff used when deriving bias-row flags from a raw bias bundle.
bias_abs_size_warn: Absolute bias-size cutoff used when deriving bias-row flags from a raw bias bundle.
bias_p_max: Probability cutoff used when deriving bias-row flags from a raw bias bundle.

Details

The dashboard screens individual facet elements across four complementary criteria:

Severity: elements with \(|\mathrm{Estimate}| >\) severity_warn logits are flagged as unusually harsh or lenient.
Misfit: elements with Infit or Outfit MnSq outside \([1/\)misfit_warn\(,\;\)misfit_warn\(]\) (default 0.67–1.5) are flagged.
Central tendency: elements with \(|\mathrm{Estimate}| <\) central_tendency_max logits are flagged. Near-zero estimates may indicate a rater who avoids extreme categories, producing artificially narrow score ranges.
Bias: elements involved in \(\ge\) bias_count_warn screen-positive interaction cells (from estimate_bias()) are flagged.

A flag density score counts how many of the four criteria each element triggers. Elements flagged on multiple criteria warrant priority review (e.g., rater retraining, data exclusion).

Default thresholds are designed for moderate-stakes rating contexts. Adjust for your application: stricter thresholds for high-stakes certification, more lenient for formative assessment.

Output

The returned object is a bundle-like list with class mfrm_facet_dashboard and components such as:

overview: one-row structural overview
summary: one-row screening summary
detail: level-level detail table
ranked: detail ordered by flag density / severity
flagged: flagged levels only
bias_sources: per-bundle bias aggregation metadata
settings: resolved threshold settings
notes: short interpretation notes

Examples

toy <- load_mfrmr_data("example_core")
fit <- fit_mfrm(toy, "Person", c("Rater", "Criterion"), "Score", method = "JML", maxit = 25)
diag <- diagnose_mfrm(fit, residual_pca = "none")
dash <- facet_quality_dashboard(fit, diagnostics = diag)
summary(dash)
#> mfrmr Facet Quality Dashboard Summary
#> 
#> Overview
#>  Facet FacetSource Levels FlaggedLevels BiasSourceBundles
#>  Rater    inferred      4             2                 0
#> 
#> Summary
#>  Facet Levels MeanEstimate    SD MinEstimate MaxEstimate MeanInfit MeanOutfit
#>  Rater      4            0 0.313      -0.329       0.334     0.994      1.019
#>  SeverityFlagged MisfitFlagged CentralTendencyFlagged BiasFlagged AnyFlagged
#>                0             0                      2           0          2
#>  BiasRows
#>         0
#> 
#> Flagged levels
#>  Facet Level Estimate N.x    SE ModelSE RealSE                     SE_Method
#>  Rater   R01   -0.196 192 0.097   0.097  0.100 Observation-table information
#>  Rater   R03    0.191 192 0.097   0.097  0.097 Observation-table information
#>  Converged PrecisionTier SupportsFormalInference          SEUse
#>       TRUE   exploratory                   FALSE screening_only
#>       TRUE   exploratory                   FALSE screening_only
#>                                                CIBasis          CIUse N.y Infit
#>  Normal interval from exploratory observation-table SE screening_only 192 1.051
#>  Normal interval from exploratory observation-table SE screening_only 192 0.965
#>  Outfit InfitZSTD OutfitZSTD DF_Infit DF_Outfit N.x.x ObservedAverage
#>   1.045     0.410      0.468  105.575       192   192           2.609
#>   0.970    -0.213     -0.258  105.705       192   192           2.396
#>  ExpectedAverage Bias MeanResidual MeanStdResidual MeanAbsStdResidual   ChiSq
#>            2.609    0            0          -0.009              0.842 200.635
#>            2.396    0            0           0.001              0.812 186.334
#>  ChiDf  ChiP SE_Residual t_Residual p_Residual SE_StdResidual t_StdResidual
#>    191 0.302       0.054     -0.002      0.998          0.072        -0.124
#>    191 0.582       0.054     -0.004      0.997          0.072         0.010
#>  p_StdResidual  DF PTMEA N.y.y CI_Lower CI_Upper CIEligible
#>          0.901 191 0.623   192   -0.387   -0.005      FALSE
#>          0.992 191 0.658   192    0.000    0.382      FALSE
#>                               CILabel   N AbsEstimate SeverityFlag MisfitFlag
#>  Approximate interval; screening only 192       0.196        FALSE      FALSE
#>  Approximate interval; screening only 192       0.191        FALSE      FALSE
#>  CentralTendencyFlag BiasCount BiasSources BiasFlag FlagCount AnyFlag FlagLabel
#>                 TRUE         0           0    FALSE         1    TRUE   central
#>                 TRUE         0           0    FALSE         1    TRUE   central
#>  .AbsEstimate
#>         0.196
#>         0.191
#> 
#> Settings
#>               Setting    Value
#>                 facet    Rater
#>          facet_source inferred
#>         severity_warn        1
#>           misfit_warn      1.5
#>  central_tendency_max     0.25
#>       bias_count_warn        1
#>       bias_abs_t_warn        2
#>    bias_abs_size_warn      0.5
#>            bias_p_max     0.05
#>   bias_source_bundles        0
#> 
#> Notes
#>  - Dashboard constructed successfully.

Usage

Arguments

Details

Output

See also

Examples