Function to summarize results from TDCM analyses.
Usage
tdcm.summary(
model,
num.time.points,
transition.option = 1,
classthreshold = 0.5,
attribute.names = c()
)Arguments
- model
a
gdinaobject returned from thetdcmfunction.- num.time.points
the number of time points (i.e., measurement/testing occasions), integer \(\ge 2\).
- transition.option
option for reporting results.
= 1compares the first time point to the last.= 2compares the first time point to every other time point.= 3compares successive time points. Default = 1.- classthreshold
probability threshold for establishing proficiency from examinee posterior probabilities. Default is .50, which maximizes overall classification accuracy. It can be set to a lower value to minimize false negatives (i.e., misclassifying proficient examinees as non-proficient) or set to a higher value to minimize false positives (i.e., misclassifying non-proficient examinees as proficient).
- attribute.names
optional vector of attribute names to include in results output.
Value
A list with the following items:
$item.parameters: LCDM item parameter estimates from the specified DCM.$growth: proficiency proportions for each time point and each attribute$transition.probabilities: conditional attribute proficiency transition probability matrices$posterior.probabilities: examinee marginal attribute posterior probabilities of proficiency$transition.posteriors: examinee marginal attribute transition posterior probabilities$most.likely.transitions: examinee most likely transitions for each attribute and transition$classifications: examinee classifications determined by the specified threshold applied to the posterior probabilities$reliability: estimated transition reliability metrics for each attribute for the specified transitions. “pt bis” = longitudinal point biserial metric; “info gain” = longitudinal information gain metric; “polychor” = longitudinal tetrachoric metric; “ave max tr” = average maximum transition posterior metric; “P(t>k)” = proportion of examinee marginal attribute transition posteriors greater than k; “wt pt bis” = weighted longitudinal point biserial; “wt info gain” = weighted longitudinal information gain.$att.corr: estimated attribute correlation matrix$model.fit: Several model fit indices and tests are output including item root mean square error of approximation (RMSEA; von Davier, 2005), mean RMSEA, bivariate item fit statistics (Chen et al., 2013), and absolute fit statistics such as mean absolute deviation for observed and expected item correlations (MADcor; DiBello, Roussons, & Stout, 2007), and standardized root mean square root of squared residuals (SRMSR; Maydeu-Olivares, 2013)
Details
Provides a summary of TDCM results including item parameters, attribute posterior probabilities, transition posterior probabilities, classifications, growth, transition probabilities, attribute correlations, several transition reliability metrics, and model fit. Includes longitudinal DCM reliability metrics developed by Schellman and Madison (2021).
References
Chen, J., de la Torre, J. ,& Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123-140.
DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao and S. Sinharay (Eds.), Handbook of Statistics, Vol. 26 (pp.979–1030). Amsterdam: Elsevier.
Johnson, M. S., & Sinharay, S. (2020). The reliability of the posterior probability of skill attainment in diagnostic classification models. Journal of Educational Measurement, 47(1), 5 – 31.
Madison, M. J. (2019). Reliably assessing growth with longitudinal diagnostic classification models. Educational Measurement: Issues and Practice, 38(2), 68-78.
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137.
Schellman, M., & Madison, M. J. (2021, July). Estimating the reliability of skill transition in longitudinal DCMs. Paper presented at the 2021 International Meeting of the Psychometric Society.
Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251-275.
von Davier M. (2008). A general diagnostic model applied to language testing data. The British journal of mathematical and statistical psychology, 61(2), 287–307.
Examples
# \donttest{
## Example 1: T = 2, A = 4
data(data.tdcm01, package = "TDCM")
dat1 <- data.tdcm01$data
qmat1 <- data.tdcm01$q.matrix
# estimate TDCM with invariance assumed and full LCDM
m1 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "GDINA")
# summarize results with tdcm.summary function
results1 <- TDCM::tdcm.summary(m1, num.time.points = 2)
results1$item.parameters
#> λ0 λ1,1 λ1,2 λ1,3 λ1,4 λ2,12 λ2,13 λ2,14 λ2,23 λ2,24
#> Item 1 -1.923 2.616 -- -- -- -- -- -- -- --
#> Item 2 -2.071 2.506 -- -- -- -- -- -- -- --
#> Item 3 -1.936 2.506 -- -- -- -- -- -- -- --
#> Item 4 -1.891 1.051 1.471 -- -- 1.115 -- -- -- --
#> Item 5 -2.157 1.705 -- 1.732 -- -- 0.835 -- -- --
#> Item 6 -1.841 -- 2.175 -- -- -- -- -- -- --
#> Item 7 -1.841 -- 2.272 -- -- -- -- -- -- --
#> Item 8 -1.965 -- 2.475 -- -- -- -- -- -- --
#> Item 9 -2.029 -- 1.242 1.531 -- -- -- -- 1.628 --
#> Item 10 -2.004 -- 1.921 -- 1.239 -- -- -- -- 0.999
#> Item 11 -1.851 -- -- 2.349 -- -- -- -- -- --
#> Item 12 -2.045 -- -- 2.566 -- -- -- -- -- --
#> Item 13 -2.083 -- -- 2.576 -- -- -- -- -- --
#> Item 14 -2.125 -- -- 1.739 2.104 -- -- -- -- --
#> Item 15 -1.805 0.777 -- 1.31 -- -- 1.896 -- -- --
#> Item 16 -2.156 -- -- -- 2.736 -- -- -- -- --
#> Item 17 -2.089 -- -- -- 2.679 -- -- -- -- --
#> Item 18 -2.087 -- -- -- 2.476 -- -- -- -- --
#> Item 19 -2.11 2.219 -- -- 1.46 -- -- 0.558 -- --
#> Item 20 -2.047 -- 2.408 -- 1.49 -- -- -- -- 0.154
#> λ2,34
#> Item 1 --
#> Item 2 --
#> Item 3 --
#> Item 4 --
#> Item 5 --
#> Item 6 --
#> Item 7 --
#> Item 8 --
#> Item 9 --
#> Item 10 --
#> Item 11 --
#> Item 12 --
#> Item 13 --
#> Item 14 0.493
#> Item 15 --
#> Item 16 --
#> Item 17 --
#> Item 18 --
#> Item 19 --
#> Item 20 --
results1$growth
#> T1[1] T2[1]
#> Attribute 1 0.201 0.372
#> Attribute 2 0.327 0.492
#> Attribute 3 0.397 0.573
#> Attribute 4 0.252 0.696
results1$transition.probabilities
#> , , Attribute 1: Time 1 to Time 2
#>
#> T2 [0] T2 [1]
#> T1 [0] 0.678 0.322
#> T1 [1] 0.432 0.568
#>
#> , , Attribute 2: Time 1 to Time 2
#>
#> T2 [0] T2 [1]
#> T1 [0] 0.578 0.422
#> T1 [1] 0.363 0.637
#>
#> , , Attribute 3: Time 1 to Time 2
#>
#> T2 [0] T2 [1]
#> T1 [0] 0.55 0.45
#> T1 [1] 0.24 0.76
#>
#> , , Attribute 4: Time 1 to Time 2
#>
#> T2 [0] T2 [1]
#> T1 [0] 0.365 0.635
#> T1 [1] 0.123 0.877
#>
results1$reliability
#> pt bis info gain polychor ave max tr P(t>.6) P(t>.7) P(t>.8)
#> Attribute 1 0.788 0.511 0.925 0.922 0.964 0.929 0.856
#> Attribute 2 0.769 0.546 0.905 0.896 0.923 0.868 0.821
#> Attribute 3 0.728 0.527 0.905 0.878 0.918 0.856 0.750
#> Attribute 4 0.715 0.485 0.891 0.899 0.936 0.883 0.801
#> P(t>.9) wt pt bis wt info gain
#> Attribute 1 0.747 0.822 0.591
#> Attribute 2 0.694 0.792 0.581
#> Attribute 3 0.625 0.757 0.568
#> Attribute 4 0.709 0.772 0.585
head(results1$most.likely.transitions)
#> Attribute 1: T1 to T2 Attribute 2: T1 to T2 Attribute 3: T1 to T2
#> 1 01 10 01
#> 2 10 00 10
#> 3 00 11 01
#> 4 00 01 01
#> 5 00 01 01
#> 6 00 10 01
#> Attribute 4: T1 to T2
#> 1 11
#> 2 11
#> 3 10
#> 4 01
#> 5 11
#> 6 01
results1$model.fit$Item.RMSEA
#> Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7
#> 0.1072683 0.1273641 0.1082175 0.1191112 0.1255691 0.1401920 0.1357562
#> Item 8 Item 9 Item 10 Item 11 Item 12 Item 13 Item 14
#> 0.1130181 0.1208428 0.1135816 0.1318266 0.1110083 0.1104637 0.1190979
#> Item 15 Item 16 Item 17 Item 18 Item 19 Item 20 Item 21
#> 0.1274363 0.1244858 0.1133979 0.1235323 0.1142053 0.1400732 0.1122910
#> Item 22 Item 23 Item 24 Item 25 Item 26 Item 27 Item 28
#> 0.1072758 0.1246946 0.1223008 0.1262111 0.1286012 0.1267738 0.1105200
#> Item 29 Item 30 Item 31 Item 32 Item 33 Item 34 Item 35
#> 0.1164483 0.1158832 0.1303099 0.1289143 0.1225251 0.1096947 0.1357179
#> Item 36 Item 37 Item 38 Item 39 Item 40
#> 0.1320643 0.1304714 0.1168578 0.1116256 0.1156059
# }