Skip to contents

mfrmr standardized/aggregated-residual Q3-style screen

Source: R/api-q3.R
q3_statistic.Rd

Computes mfrmr's Q3-style screening index: the Pearson correlation of standardized residuals, after mean-aggregation to one residual per Person-by-level cell, between every pair of levels of a chosen facet. Large absolute values identify pairs whose residuals show stronger positive co-movement or negative counter-movement than the main-effects model expects. This is not the raw-residual Yen (1984) Q3 statistic.

Usage

q3_statistic(
  fit,
  diagnostics = NULL,
  facet = "Rater",
  min_pairs = 5L,
  yen_threshold = 0.2,
  marais_threshold = 0.3,
  relative_offset = 0.2
)

Arguments

fit

An mfrm_fit from fit_mfrm().

diagnostics

Optional diagnose_mfrm() output. Computed on demand when omitted.

facet

Facet whose levels are paired (default "Rater").

min_pairs

Minimum number of persons with finite aggregated residuals at both levels required to retain a pair; a single integer of at least three. Pairs below the threshold drop out of the table (mirrors plot_local_dependence_heatmap()).

yen_threshold

Legacy-named argument retained for compatibility. It applies the strict absolute heuristic |Q3-style| > threshold (default 0.20). Yen (1984) did not propose this as a general cutoff, and it is not calibrated for mfrmr's standardized/aggregated-residual index.

marais_threshold

Stricter community-convention threshold (default 0.30). Marais (2013, p. 121) reports this as a value "often considered" in the literature, not as her own recommendation; her actual recommendation is the relative comparison implemented by relative_offset.

relative_offset

Screening offset for the relative-flag rule |Q3 - mean(Q3)| > relative_offset (default 0.20). This is a simplified screening approximation of the relative comparison advocated by Marais (2013) and operationalized by Christensen et al. (2017) as Q3_* = Q3_max - mean(Q3). Christensen et al. (2017) demonstrate empirically that no single critical value is appropriate across designs and recommend a parametric bootstrap; the fixed 0.20 here is a screening default, not a substitute for that bootstrap.

Value

An object of class mfrm_q3 containing:

pairs

A data frame with one row per facet-level pair and columns Level1, Level2, Q3, N, AbsQ3, YenFlag, MaraisFlag, RelativeFlag, and a textual Interpretation summarising which thresholds were exceeded. N is the number of persons with finite aggregated residuals at both levels. YenFlag is a compatibility name and does not imply that Yen (1984) proposed the fixed default.

summary

One-row tibble with MeanQ3, MaxAbsQ3, and the three flagged-pair counts.

thresholds

The thresholds used, for reproducibility. The yen name is retained for compatibility.

facet

The facet whose levels were paired.

Statistic definition and distinction from Yen (1984)

The name Q3-style is deliberate. This implementation differs from Yen's (1984) original Q3 definition in two respects that together affect threshold interpretation.

(1) Standardized vs raw residuals. Yen (1984, eqs. 7-8, p. 127) defines Q3 = cor(d_i, d_j) where d_{ik} = u_{ik} - P_hat_{ik} is the raw residual. mfrmr uses standardized residuals Z = (u - P_hat) / sqrt(Var(u)) because that is what diagnose_mfrm() stores. Because model-based residual scale factors can vary across observations, the resulting pair correlations and their null distribution differ from raw-residual Q3.

(2) Mean-aggregation. When the facet being paired (e.g. Rater) has multiple residual rows per (Person, Level) cell because of additional facets in the design (e.g. multiple Criterion rows per Person-Rater cell), the standardized residuals are first mean-aggregated to one value per (Person, Level) cell, and the Pearson correlation is taken over those mean-aggregated residuals. Yen's original formulation takes the correlation directly over per-(Person, Item) residuals, without aggregation. Aggregation changes the analysis unit to persons. Each correlation is based on persons observed at both levels, so its magnitude and null distribution are not directly comparable with raw-row Q3.

For both reasons, treat the values returned here as a screening summary rather than a direct substitute for the published Q3 thresholds. A formal raw-residual Q3 procedure would require a separately implemented and validated design-specific bootstrap; mfrmr 0.2.2 does not provide that procedure.

References

  • Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145. doi:10.1177/014662168400800201

  • Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265-289. doi:10.3102/10769986022003265

  • Marais, I. (2013). Local dependence. In K. B. Christensen, S. Kreiner, & M. Mesbah (Eds.), Rasch models in health (pp. 111-130). London: ISTE / Wiley.

  • Christensen, K. B., Makransky, G., & Horton, M. (2017). Critical values for Yen's Q3: Identification of local dependence in the Rasch model using residual correlations. Applied Psychological Measurement, 41(3), 178-194. doi:10.1177/0146621616677520

  • Nason, K., & DeMars, C. (2025). Another look at Yen's Q3: Is .2 an appropriate cut-off? Journal of Educational Measurement, 62(2), 345-359. doi:10.1111/jedm.12432

See also

plot_local_dependence_heatmap(), diagnose_mfrm()

Examples

# \donttest{
toy <- load_mfrmr_data("example_core")
fit <- fit_mfrm(toy, "Person", c("Rater", "Criterion"), "Score",
                method = "JML", maxit = 30)
#> Warning: Optimization convergence review did not produce an inference-ready numerical solution (code = 1, status = iteration_limit). Optimizer reached the iteration limit before the terminal gradient became small enough for review-only acceptance. Inspect the model specification, data support, and starting values. Do not interpret estimates until the review is resolved.
q3 <- q3_statistic(fit)
q3$summary
#>       MeanQ3  MaxAbsQ3 YenFlagged MaraisFlagged RelativeFlagged
#> 1 -0.3277129 0.4631436         10             8               2
# Look for: use MaxAbsQ3 and the pair table to rank follow-up. The default
#   0.20/0.30 absolute rules are uncalibrated heuristics for this Q3-style
#   index, not standalone local-independence tests.
head(q3$pairs)
#>    Level1 Level2         Q3  N     AbsQ3 YenFlag MaraisFlag RelativeFlag
#> 3     R01    R04 -0.4631436 48 0.4631436    TRUE       TRUE        FALSE
#> 10    R04    R01 -0.4631436 48 0.4631436    TRUE       TRUE        FALSE
#> 2     R01    R03 -0.4382773 48 0.4382773    TRUE       TRUE        FALSE
#> 7     R03    R01 -0.4382773 48 0.4382773    TRUE       TRUE        FALSE
#> 6     R02    R04 -0.4243919 48 0.4243919    TRUE       TRUE        FALSE
#> 11    R04    R02 -0.4243919 48 0.4243919    TRUE       TRUE        FALSE
#>                            Interpretation
#> 3  stricter heuristic (|Q3-style| > 0.30)
#> 10 stricter heuristic (|Q3-style| > 0.30)
#> 2  stricter heuristic (|Q3-style| > 0.30)
#> 7  stricter heuristic (|Q3-style| > 0.30)
#> 6  stricter heuristic (|Q3-style| > 0.30)
#> 11 stricter heuristic (|Q3-style| > 0.30)
# }