Skip to contents

mfrmr standardized/aggregated-residual Q3-style screen

Source: R/api-q3.R
q3_statistic.Rd

Computes mfrmr's Q3-style screening index: the Pearson correlation of standardized residuals, after mean-aggregation to one residual per Person-by-level cell, between every pair of levels of a chosen facet. Large absolute values identify pairs whose residuals show stronger positive co-movement or negative counter-movement than the main-effects model expects. This is not the raw-residual Yen (1984) Q3 statistic.

Usage

q3_statistic(
  fit,
  diagnostics = NULL,
  facet = "Rater",
  min_pairs = 5L,
  yen_threshold = 0.2,
  marais_threshold = 0.3,
  relative_offset = 0.2
)

Arguments

fit

An mfrm_fit from fit_mfrm().

diagnostics

Optional diagnose_mfrm() output. Computed on demand when omitted.

facet

Facet whose levels are paired (default "Rater").

min_pairs

Minimum number of persons with finite aggregated residuals at both levels required to retain a pair; a single integer of at least three. Unavailable pairs remain in the table with NA correlations and flags (mirrors plot_local_dependence_heatmap()).

yen_threshold

Legacy-named argument retained for compatibility. It applies the strict absolute heuristic |Q3-style| > threshold (default 0.20). Yen (1984) did not propose this as a general cutoff, and it is not calibrated for mfrmr's standardized/aggregated-residual index.

marais_threshold

Stricter community-convention threshold (default 0.30). Marais (2013, p. 121) reports this as a value "often considered" in the literature, not as her own recommendation; her actual recommendation is the relative comparison implemented by relative_offset.

relative_offset

Screening offset for the relative-flag rule |Q3 - mean(Q3)| > relative_offset (default 0.20). This is a simplified screening approximation of the relative comparison advocated by Marais (2013) and operationalized by Christensen et al. (2017) as Q3_* = Q3_max - mean(Q3). Christensen et al. (2017) demonstrate empirically that no single critical value is appropriate across designs and recommend a parametric bootstrap; the fixed 0.20 here is a screening default, not a substitute for that bootstrap.

Value

An object of class mfrm_q3 containing:

pairs

A data frame with one row per unordered facet-level pair and columns Level1, Level2, Q3, N, AbsQ3, YenFlag, MaraisFlag, RelativeFlag, and a textual Interpretation summarising which thresholds were exceeded. N is the number of persons with finite aggregated residuals at both levels. Unavailable pairs have NA correlations and flags, with an explanation in Interpretation. YenFlag is a compatibility name and does not imply that Yen (1984) proposed the fixed default.

summary

One-row tibble with MeanQ3, MaxAbsQ3, the three flagged-pair counts, CandidatePairs, AvailablePairs, and UnavailablePairs. Flagged counts are unavailable if no pair can be evaluated.

thresholds

The thresholds used, for reproducibility. The yen name is retained for compatibility.

facet

The facet whose levels were paired.

Statistic definition and distinction from Yen (1984)

The name Q3-style is deliberate. This implementation differs from Yen's (1984) original Q3 definition in two respects that together affect threshold interpretation.

(1) Standardized vs raw residuals. Yen (1984, eqs. 7-8, p. 127) defines Q3 = cor(d_i, d_j) where d_{ik} = u_{ik} - P_hat_{ik} is the raw residual. mfrmr uses standardized residuals Z = (u - P_hat) / sqrt(Var(u)) because that is what diagnose_mfrm() stores. Because model-based residual scale factors can vary across observations, the resulting pair correlations and their null distribution differ from raw-residual Q3.

(2) Mean-aggregation. When the facet being paired (e.g. Rater) has multiple residual rows per (Person, Level) cell because of additional facets in the design (e.g. multiple Criterion rows per Person-Rater cell), the standardized residuals are first mean-aggregated to one value per (Person, Level) cell, and the Pearson correlation is taken over those mean-aggregated residuals. Yen's original formulation takes the correlation directly over per-(Person, Item) residuals, without aggregation. Aggregation changes the analysis unit to persons. Each correlation is based on persons observed at both levels, so its magnitude and null distribution are not directly comparable with raw-row Q3.

For both reasons, treat the values returned here as a screening summary rather than a direct substitute for the published Q3 thresholds. A formal raw-residual Q3 procedure would require a separately implemented and validated design-specific bootstrap; mfrmr does not currently provide that procedure.

Dimensionality and network interpretation

Choose facet for the question: pairing Rater levels describes rater residual associations, not a count of rubric dimensions. Associations may reflect additional ability structure, shared performance effects, rater effects or assignment patterns. A thresholded correlation graph is not a partial-correlation network or a fitted residual-network model; its community count is not a dimensionality test. Missing pairs are not zero edges. See analyze_residual_pca() and vignette("mfrmr-visual-diagnostics") for complementary checks and the limits of exploratory network analysis.

References

  • Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145. doi:10.1177/014662168400800201

  • Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265-289. doi:10.3102/10769986022003265

  • Marais, I. (2013). Local dependence. In K. B. Christensen, S. Kreiner, & M. Mesbah (Eds.), Rasch models in health (pp. 111-130). London: ISTE / Wiley.

  • Christensen, K. B., Makransky, G., & Horton, M. (2017). Critical values for Yen's Q3: Identification of local dependence in the Rasch model using residual correlations. Applied Psychological Measurement, 41(3), 178-194. doi:10.1177/0146621616677520

  • Nason, K., & DeMars, C. (2025). Another look at Yen's Q3: Is .2 an appropriate cut-off? Journal of Educational Measurement, 62(2), 345-359. doi:10.1111/jedm.12432

See also

plot_local_dependence_heatmap(), diagnose_mfrm()

Examples

# \donttest{
toy <- load_mfrmr_data("example_core")
fit <- fit_mfrm(toy, "Person", c("Rater", "Criterion"), "Score",
                method = "JML", maxit = 300)
q3 <- q3_statistic(fit)
q3$summary
#>   CandidatePairs AvailablePairs UnavailablePairs     MeanQ3  MaxAbsQ3
#> 1              6              6                0 -0.3277176 0.4631367
#>   YenFlagged MaraisFlagged RelativeFlagged
#> 1          5             4               1
# Look for: use MaxAbsQ3 and the pair table to rank follow-up. The default
#   0.20/0.30 absolute rules are uncalibrated heuristics for this Q3-style
#   index, not standalone local-independence tests.
head(q3$pairs)
#>   Level1 Level2          Q3  N      AbsQ3 YenFlag MaraisFlag RelativeFlag
#> 3    R01    R04 -0.46313671 48 0.46313671    TRUE       TRUE        FALSE
#> 2    R01    R03 -0.43828943 48 0.43828943    TRUE       TRUE        FALSE
#> 5    R02    R04 -0.42439987 48 0.42439987    TRUE       TRUE        FALSE
#> 4    R02    R03 -0.36331502 48 0.36331502    TRUE       TRUE        FALSE
#> 6    R03    R04 -0.26385969 48 0.26385969    TRUE      FALSE        FALSE
#> 1    R01    R02 -0.01330509 48 0.01330509   FALSE      FALSE         TRUE
#>                            Interpretation
#> 3  stricter heuristic (|Q3-style| > 0.30)
#> 2  stricter heuristic (|Q3-style| > 0.30)
#> 5  stricter heuristic (|Q3-style| > 0.30)
#> 4  stricter heuristic (|Q3-style| > 0.30)
#> 6 fixed absolute rule (|Q3-style| > 0.20)
#> 1        relative-to-mean (offset > 0.20)
# }