Verification and Validation on Test Science Research Document Library https://research.testscience.org/keywords/verification-and-validation/ Recent content in Verification and Validation on Test Science Research Document Library Hugo -- 0.129.0 en-us Institute for Defense Analyses Sun, 01 Jan 2017 00:00:00 +0000 Comparing Live Missile Fire and Simulation https://research.testscience.org/post/2017-comparing-live-missile-fire-and-simulation/ Sun, 01 Jan 2017 00:00:00 +0000 https://research.testscience.org/post/2017-comparing-live-missile-fire-and-simulation/ Modeling and Simulation is frequently used in Test and Evaluation (T&E) of air-to-air weapon systems to evaluate the effectiveness of a weapons. The AirIntercept Missile-9X (AIM-9X) program uses modeling and simulationextensively to evaluate missile miss distances. Since flight testing isexpensive, the test program uses relatively few flight tests and supplementsthose data with large numbers of miss distances from simulated tests acrossthe weapons operational space. However, before modeling and simulation canbe used to predict performance it must first be validated. Modeling and Simulation is frequently used in Test and Evaluation (T&E) of air-to-air weapon systems to evaluate the effectiveness of a weapons. The AirIntercept Missile-9X (AIM-9X) program uses modeling and simulationextensively to evaluate missile miss distances. Since flight testing isexpensive, the test program uses relatively few flight tests and supplementsthose data with large numbers of miss distances from simulated tests acrossthe weapons operational space. However, before modeling and simulation canbe used to predict performance it must first be validated. Validation isespecially challenging when working with a limited number of live test data. Inthis presentation, we show that even with a limited number of live test points(e.g., 16 missile fires), we can still perform a statistical analysis for thevalidation. We introduce a validation technique known as Fisher’s CombinedProbability Test and show how to apply Fisher’s test to validate the AIM-9Xmodel and simulation.

Suggested Citation

Medlin, Rebecca, Pamela Rambow, and Douglas Peek. Comparing Live Missile Fire and Simulation. IDA Document NS D-8443. Alexandria, VA: Institute for Defense Analyses, 2017.

Slides:

]]> Best Practices for Statistically Validating Modeling and Simulation (M&S) Tools Used in Operational Testing https://research.testscience.org/post/2015-best-practices-for-statistically-validating-modeling-and-simulation-m-s-tools-used-in-operational-testing/ Thu, 01 Jan 2015 00:00:00 +0000 https://research.testscience.org/post/2015-best-practices-for-statistically-validating-modeling-and-simulation-m-s-tools-used-in-operational-testing/ In many situations, collecting sufficient data to evaluate system performance against operationally realistic threats is not possible due to cost and resource restrictions, safety concerns, or lack of adequate or representative threats. Modeling and simulation tools that have been verified, validated, and accredited can be used to supplement live testing in order to facilitate a more complete evaluation of performance. Two key questions that frequently arise when planning an operational test are (1) which (and how many) points within the operational space should be chosen in the simulation space and the live space for optimal ability to verify and validate the M&S, and (2) once that data is collected, what is the best way to compare the live trials to the simulated trials for the purpose of validating the M&S? In many situations, collecting sufficient data to evaluate system performance against operationally realistic threats is not possible due to cost and resource restrictions, safety concerns, or lack of adequate or representative threats. Modeling and simulation tools that have been verified, validated, and accredited can be used to supplement live testing in order to facilitate a more complete evaluation of performance. Two key questions that frequently arise when planning an operational test are (1) which (and how many) points within the operational space should be chosen in the simulation space and the live space for optimal ability to verify and validate the M&S, and (2) once that data is collected, what is the best way to compare the live trials to the simulated trials for the purpose of validating the M&S? This conference presentation addresses various strategies for addressing these two questions. The best methodologies for designing and analyzing will vary depending on the goal of operational test, the type of model used in the simulation, and the amount of live and simulated data available.

Suggested Citation

Avery, Kelly, Laura Freeman, and Rebecca Medlin. Best Practices for Statistically Validating Modeling and Simulation (M&S) Tools Used in Operational Testing. IDA Document NS D-5582. Alexandria, VA: Institute for Defense Analyses, 2015.

Slides:

]]>