Build an unexpected-response screening report

Usage

unexpected_response_table(
  fit,
  diagnostics = NULL,
  abs_z_min = 2,
  prob_max = 0.3,
  top_n = 100,
  rule = c("either", "both")
)

Arguments

fit: Output from fit_mfrm().
diagnostics: Optional output from diagnose_mfrm().
abs_z_min: Absolute standardized-residual cutoff.
prob_max: Maximum observed-category probability cutoff.
top_n: Maximum number of rows to return.
rule: Flagging rule: "either" (default) or "both".

Value

A named list with:

table: flagged response rows
summary: one-row overview
thresholds: applied thresholds

Details

A response is flagged as unexpected when:

rule = "either": |StdResidual| >= abs_z_min OR ObsProb <= prob_max
rule = "both": both conditions must be met.

The table includes row-level observed/expected values, residuals, observed-category probability, most-likely category, and a composite severity score for sorting.

Interpreting output

summary: prevalence of unexpected responses under current thresholds.
table: ranked row-level diagnostics for case review.
thresholds: active cutoffs and flagging rule.

Compare results across rule = "either" and rule = "both" to assess how conservative your screening should be.

Typical workflow

Start with rule = "either" for broad screening.
Re-run with rule = "both" for strict subset.
Inspect top rows and visualize with plot_unexpected().

Further guidance

For a plot-selection guide and a longer walkthrough, see mfrmr_visual_diagnostics and vignette("mfrmr-visual-diagnostics", package = "mfrmr").

Output columns

The table data.frame contains:

Row: Original row index in the prepared data.
Person: Person identifier (plus one column per facet).
Score: Observed score category.
Observed, Expected: Observed and model-expected score values.
Residual, StdResidual: Raw and standardized residuals.
ObsProb: Probability of the observed category under the model.
MostLikely, MostLikelyProb: Most probable category and its probability.
Severity: Composite severity index (higher = more unexpected).
Direction: "Higher than expected" or "Lower than expected".
FlagLowProbability, FlagLargeResidual: Logical flags for each criterion.

The summary data.frame contains:

TotalObservations: Total observations analyzed.
UnexpectedN, UnexpectedPercent: Count and share of flagged rows.
AbsZThreshold, ProbThreshold: Applied cutoff values.
Rule: "either" or "both".

Examples

toy <- load_mfrmr_data("example_core")
fit <- fit_mfrm(toy, "Person", c("Rater", "Criterion"), "Score", method = "JML", maxit = 25)
t4 <- unexpected_response_table(fit, abs_z_min = 1.5, prob_max = 0.4, top_n = 10)
summary(t4)
#> mfrmr Unexpected Response Summary 
#>   Class: mfrm_unexpected
#>   Components (3): table, summary, thresholds
#> 
#> Threshold summary
#>  TotalObservations UnexpectedN UnexpectedPercent LowProbabilityN LargeResidualN
#>                768          10             1.302              10             10
#>    Rule AbsZThreshold ProbThreshold
#>  either           1.5           0.4
#> 
#> Flagged responses: table
#>  Row Person Rater    Criterion Weight Score Observed Expected Residual
#>   71   P023   R02      Content      1     2        2    3.879   -1.879
#>  199   P007   R01 Organization      1     1        1    3.145   -2.145
#>  628   P004   R02     Accuracy      1     1        1    3.119   -2.119
#>  236   P044   R01 Organization      1     1        1    3.083   -2.083
#>  166   P022   R04      Content      1     4        4    1.908    2.092
#>  132   P036   R03      Content      1     2        2    3.594   -1.594
#>  609   P033   R01     Accuracy      1     4        4    2.021    1.979
#>  574   P046   R04     Language      1     3        3    1.435    1.565
#>  619   P043   R01     Accuracy      1     4        4    2.084    1.916
#>  362   P026   R04 Organization      1     3        3    1.445    1.555
#>  StdResidual ObsProb MostLikely MostLikelyProb CategoryGap Surprise
#>       -5.569   0.003          4          0.882           2    2.457
#>       -2.913   0.015          3          0.483           2    1.811
#>       -2.856   0.017          3          0.486           2    1.768
#>       -2.778   0.020          3          0.490           2    1.709
#>        2.747   0.019          2          0.468           2    1.730
#>       -2.785   0.039          4          0.635           2    1.408
#>        2.532   0.027          2          0.471           2    1.562
#>        2.625   0.050          1          0.619           2    1.304
#>        2.425   0.033          2          0.469           2    1.477
#>        2.585   0.052          1          0.611           2    1.285
#>             Direction FlagLowProbability FlagLargeResidual Severity
#>   Lower than expected               TRUE              TRUE    9.026
#>   Lower than expected               TRUE              TRUE    5.724
#>   Lower than expected               TRUE              TRUE    5.624
#>   Lower than expected               TRUE              TRUE    5.487
#>  Higher than expected               TRUE              TRUE    5.477
#>   Lower than expected               TRUE              TRUE    5.193
#>  Higher than expected               TRUE              TRUE    5.094
#>  Higher than expected               TRUE              TRUE    4.929
#>  Higher than expected               TRUE              TRUE    4.902
#>  Higher than expected               TRUE              TRUE    4.870
#> 
#> Settings
#>    Setting  Value
#>  abs_z_min    1.5
#>   prob_max    0.4
#>       rule either
#> 
#> Notes
#>  - Unexpected-response summary for quick residual screening.
p_t4 <- plot(t4, draw = FALSE)
class(p_t4)
#> [1] "mfrm_plot_data" "list"