Data Analysis on Test Science Research Document Library https://research.testscience.org/keywords/data-analysis/ Recent content in Data Analysis on Test Science Research Document Library Hugo -- 0.129.0 en-us Institute for Defense Analyses Sun, 01 Jan 2017 00:00:00 +0000 Thinking About Data for Operational Test and Evaluation https://research.testscience.org/post/2017-thinking-about-data-for-operational-test-and-evaluation/ Sun, 01 Jan 2017 00:00:00 +0000 https://research.testscience.org/post/2017-thinking-about-data-for-operational-test-and-evaluation/ While the human brain is powerful tool for quickly recognizing patterns in data, it will frequently make errors in interpreting random data. Luckily, these mistakes occur in systematic and predictable ways. Statistical models provide an analytical framework that helps us avoid these error-prone heuristics and draw accurate conclusions from random data. This non-technical presentation highlights some tricks of the trade learned by studying data and the way the human brain processes. While the human brain is powerful tool for quickly recognizing patterns in data, it will frequently make errors in interpreting random data. Luckily, these mistakes occur in systematic and predictable ways. Statistical models provide an analytical framework that helps us avoid these error-prone heuristics and draw accurate conclusions from random data. This non-technical presentation highlights some tricks of the trade learned by studying data and the way the human brain processes. First, we introduce statistics as the science of data, and discuss how the popular conception of randomness differs from its technical definition. Later sections highlight the human brain as a pattern recognition machine. Examples from published literature and media highlight systematic and predicable errors in human cognition as well as how poor data analysis and graphical displays can cause critical errors in analysis. Finally, we’ll talk about using statistical models for analysis, including how violations of model assumptions should effect our analyses.

Suggested Citation

Thomas, Dean, and Matthew Avery. Thinking About Data for Operational Test and Evaluation. IDA Document NS D-8729. Alexandria, VA: Institute for Defense Analyses, 2017.

Slides:

]]> A First Step into the Bootstrap World https://research.testscience.org/post/2016-a-first-step-into-the-bootstrap-world/ Fri, 01 Jan 2016 00:00:00 +0000 https://research.testscience.org/post/2016-a-first-step-into-the-bootstrap-world/ Bootstrapping is a powerful nonparametric tool for conducting statistical inference with many applications to data from operational testing. Bootstrapping is most useful when the population sampled from is unknown or complex or the sampling distribution of the desired statistic is difficult to derive. Careful use of bootstrapping can help address many challenges in analyzing operational test data. Suggested Citation Avery, Matthew R. A First Step into the Bootstrap World. IDA Document NS D-5816. Bootstrapping is a powerful nonparametric tool for conducting statistical inference with many applications to data from operational testing. Bootstrapping is most useful when the population sampled from is unknown or complex or the sampling distribution of the desired statistic is difficult to derive. Careful use of bootstrapping can help address many challenges in analyzing operational test data.

Suggested Citation

Avery, Matthew R. A First Step into the Bootstrap World. IDA Document NS D-5816. Alexandria, VA: Institute for Defense Analyses, 2016.

Slides:

]]> Validating the PRA Testbed Using a Statistically Rigorous Approach https://research.testscience.org/post/2015-validating-the-pra-testbed-using-a-statistically-rigorous-approach/ Thu, 01 Jan 2015 00:00:00 +0000 https://research.testscience.org/post/2015-validating-the-pra-testbed-using-a-statistically-rigorous-approach/ For many systems, testing is expensive and only a few live test events are conducted. When this occurs, testers frequently use a model to extend the test results. However, testers must validate the model to show that it is an accurate representation of the real world from the perspective of the intended uses of the model. This raises a problem when only a small number of live test events are conducted, only limited data are available to validate the model, and some testers struggle with model validation. For many systems, testing is expensive and only a few live test events are conducted. When this occurs, testers frequently use a model to extend the test results. However, testers must validate the model to show that it is an accurate representation of the real world from the perspective of the intended uses of the model. This raises a problem when only a small number of live test events are conducted, only limited data are available to validate the model, and some testers struggle with model validation. This article describes a statistically rigorous approach for validating a model with only a small number of live test results. We discuss a specific application for validating a model of a naval surface combatant defending itself against a cruise missile attack. The approach takes into account potential correlation in the data and other factors that may drive system performance.

Suggested Citation

Thomas, Dean, and Rebecca Dickinson. “Validating the Probability of Raid Annihilation Testbed Using a Statistical Approach.” The ITEA Journal of Test and Evaluation 36, no. 2 (June 2015).

Paper:

]]>