The ITEA Journal of Test and Evaluation on Test Science Research Document Library https://research.testscience.org/venues/the--itea-journal-of-test-and-evaluation/ Recent content in The ITEA Journal of Test and Evaluation on Test Science Research Document Library Hugo -- 0.129.0 en-us Institute for Defense Analyses Tue, 01 Jan 2019 00:00:00 +0000 The Effect of Extremes in Small Sample Size on Simple Mixed Models- A Comparison of Level-1 and Level-2 Size https://research.testscience.org/post/2019-the-effect-of-extremes-in-small-sample-size-on-simple-mixed-models-a-comparison-of-level-1-and-level-2-size/ Tue, 01 Jan 2019 00:00:00 +0000 https://research.testscience.org/post/2019-the-effect-of-extremes-in-small-sample-size-on-simple-mixed-models-a-comparison-of-level-1-and-level-2-size/ We present a simulation study that examines the impact of small sample sizes in both observation and nesting levels of the model on the fixed effect bias, type I error, and the power of a simple mixed model analysis. Despite the need for adjustments to control for type I error inflation, our findings indicate that smaller samples than previously recognized can be used for mixed models under certain conditions prevalent in applied research. We present a simulation study that examines the impact of small sample sizes in both observation and nesting levels of the model on the fixed effect bias, type I error, and the power of a simple mixed model analysis. Despite the need for adjustments to control for type I error inflation, our findings indicate that smaller samples than previously recognized can be used for mixed models under certain conditions prevalent in applied research.

Suggested Citation

Carter, Kristina A, Heather M Wojton, and Stephanie T Lane. “The Effect of Extremes in Small Sample Size on Simple Mixed Models: A Comparison of Level-1 and Level-2 Size.” The ITEA Journal of Test and Evaluation 40, no. 1 (2019): 16–29.

Paper:

]]> The Purpose of Mixed-Effects Models in Test and Evaluation https://research.testscience.org/post/2019-the-purpose-of-mixed-effects-models-in-test-and-evaluation/ Tue, 01 Jan 2019 00:00:00 +0000 https://research.testscience.org/post/2019-the-purpose-of-mixed-effects-models-in-test-and-evaluation/ Mixed-effects models are the standard technique for analyzing data with grouping structure. In defense testing, these models are useful because they allow us to account for correlations between observations, a feature common in many operational tests. In this article, we describe the advantages of modeling data from a mixed-effects perspective and discuss an R package—ciTools—that equips the user with easy methods for presenting results from this type of model. Suggested Citation Haman, John, Matthew Avery, and Heather Wojton. Mixed-effects models are the standard technique for analyzing data with grouping structure. In defense testing, these models are useful because they allow us to account for correlations between observations, a feature common in many operational tests. In this article, we describe the advantages of modeling data from a mixed-effects perspective and discuss an R package—ciTools—that equips the user with easy methods for presenting results from this type of model.

Suggested Citation

Haman, John, Matthew Avery, and Heather Wojton. “The Purpose of Mixed-Effects Models in Test and Evaluation.” The ITEA Journal of Test and Evaluation 40, no. 4 (2019): 249–55.

Slides:

Paper:

]]>