U. S. NAVY AIR DEFENSE THREAT SIMULATOR VALIDATION PROCEDURES MANUAL TM 7489-3 Appendices

TM 7489-3 TABLE of CONTENTS

Introduction
Organizational Responsibilities
Validation Guidelines

Appendixes

A. Sample Correspondence
B. OPNAV Instruction 3960.15 B-1
C. NAVAIR Instruction 3960.00 C-1
D. Acronyms D-1
E. Acronyms E-1

Figure A-1. Sample Request For Threat Data Review (below)

3900                        
Ser Ser 53C000D/

Secret--Unclassified upon removal of enclosure (1)

From: Commander, Naval Air Warfare Center Weapons Division

     To: Director, Defense Investigative Agency, Missile & Space Intelligence Center, (MSC-1B,
           Attn: Mr. Clay Peake and Mr. Rick Brown), Redstone Arsenal, AL 35898-5500

   Subj: INTELLIGENCE DATA REVIEW

Encl: (1)"Emitter"-only section of the Intelligence data printout

1. Enclosure (1) provides a copy of the "Emitter only" section of the intelligence data to be used in the threat parametric data tables and threat description sections of the Navy validation report. This data was extracted from the Navy Validation Data Base and includes many of the parameters used in the CROSSBOW Radar Threat Definition Document tree stucture. Data pertaining to the threat.has been composed of extracted Electronic Warfare Integrated Reprogrammable (EWIR) and the latest available exploitation documents.

2. In order to expedite the ESS Basic Navy Threat Radar Simulator Validation Report through the Initial Operational Capability phase, your immediate review and comments on this material are requested on or before 23 January 1995. Please include confidence levels on all threat parameters with known data.

3. Should you have any questions or comments concerning this letter or the data printout, please contact Mr. Stephen Wireman, DSN 437-0307, or COMM (619) 939-0307.

R. L. BALLENGER
By direction

Figure A-1. Sample Request For Threat Data Review (above)

Figure A-2. Sample Validation Report Cover Letter (below)

3900                
Ser 53C000D/

SECRET--Unclassified upon removal of enclosure (1)

From: Commander, Naval Air Warfare Center Weapons Division

     To: Director, Office of Chief of Naval Operations (N912R4, CDR Mark Hansen),
            Washington, DC 20350-2000

  Subj: NAVY TEST AND EVALUATION RANGE SIMULATOR VALIDATION REPORT

   Encl: (1) J-4 Threat Radar Simulator Operational Validation Report (2 cys)

1. The Navy Validation Coordination Office is pleased to provide you with two copies of the J-4 Threat Radar Simulator Operational Validation Report. The simulator is located at the Electronic Combat Range Department (ECRD), NAWCWPNS China Lake, California. One copy is for your retention. The other copy is sent for your review/approval and requires forwarding to the CROSSBOW-S Management Office for further DOD validation subcommittee review and approval.

2. Other copies of this report are being forwarded to Mr. Mike Lewis (PMA-272E2), Program Executive Officer, Tactical Aircraft Programs, Arlington, Virginia; Mr. Dennis McKeen (529430D), Mr. Dave Dembeck (529410D), and Ms. Jerry Broaker (4J4000D), NAWCWPNS, China Lake.

3. Should you have any questions or comments concerning this letter or the report, please contact Mr. Stephen Wireman, DSN 437-0307 or COMM (619) 939-0307.

R. L. BALLENGER
By direction

Copy to:
COMNAVAIRSYSCOM (PMA-272E2, Mr. M. Lewis) cy 4
NAWCWPNS (529430D, Mr.D. Mckeen) cy 5
NAWCWPNS (529410D, Mr. D. Dembeck) cy 6
NAWCWPNS (4J4000D, Ms. J. Broaker) cy 7

Figure A-2. Sample Validation Report Cover Letter (above)

Commander, Naval Air Warfare Center, Weapons Division
China Lake, CA 93555-6001 (Code 53C000D)
ATTN: Mr. Steve Wireman

Defense Intelligence Agency
Missile & Space Intelligence Center
Redstone Arsenal, AL 35898-5500
ATTN: (MSC-7) Mr. John Smith

Director, Test & Evaluation Division
Office of CNO, Code (N912)
Room 5C735, Pentagon
Washington, DC 20350-2000

Commander, Naval Air Systems Command
(Code PMA-248D), Room 801
Crystal Gateway 2, Washington, DC 20361-1248
ATTN: Lt. Lee Priest

Program Executive Officer
PMA-272E2
ATTN: Mr. Mike Lewis

Tactical Aircraft Programs
1421 Jefferson Davis Highway
Arlington, VA 22243

Commander, Naval Air Warfare Center, Weapons Division
(Code PO3 IN, P2344)
Point Mugu, CA 93042

Commander, Naval Air Warfare Center-Aircraft Division
(Code SYO4 EW, CT-230)
Patuxent River, MD 20670

Commander, Operational Test Force
(Code 26, 54)
Norfolk, VA 23511-6388

Appendix B

OPNAVINST 3960.15
OP-913
6 MAY 1991
OPNAV INSTRUCTION 3960.15

From: Chief of Naval Operations

Subj: VALIDATION OF NAVY AIR DEFENSE THREAT SIMULATORS

Encl:
(1) U.S. Navy Electronic Warfare Training Ranges

(2) U.S. Navy Electronic Warfare Test and Evaluation Ranges/Laboratories

1. Purpose. To establish an approach and assign responsibilities for validation of Navy air defense threat simulators. The scope of this instruction will be expanded through revisions to address validation of the complete spectrum of Navy threat simulators.

2. Background

a. All three services develop and use air defense simulators. DOD oversight responsibility is carried out through the DOD Executive Committee on Threat Simulators (EXCOM) and the Joint-Service CROSSBOW-S committee. DOD Manual 5000.3-M-6 of April 1989 (NOTAL) was issued to establish policy, procedures and guidelines for simulator validation. CROSSBOW-S has established validation criteria and the capability to technically review service written validation reports for adequacy and to make approval recommendation to the EXCOM. 5000.3-M-6 has been canceled with the publication of the new DOD Directive 5000.1 of 23 February 1991 (NOTAL), DOD Instruction 5000.2 of 23 February 1991 (NOTAL), and 5000.2-M of February 1991 (NOTAL). It is the intent of this instruction to avoid conflict with the new DOD instructions and implement OSD guidelines that were presented in 5000.3-M-6.

b. Major acquisition and procurement decisions for airborne electronic warfare systems are based on the results of the testing of developmental hardware against air defense threat simulators. For those decisions to be informed and correct, the simulators must be valid representations of the threat. A "validated" simulator does not mean that every parameter of the simulator is identical to the equivalent parameter of the actual threat system, but that parametric measurement of the simulator fall within tolerances required to support the test/training requirements for which the simulator is to be used. The parameters of the actual threat system are those stated by the current DIA threat assessment.

3. Scope. This instruction is applicable to all Navy air defense threat simulators located at ranges and laboratories listed in enclosures (1) and (2) which are used for:

a. developmental testing,

b. operational testing and tactics development, or

c. Training of Navy aircrew personnel.

4. Approach

a. All Navy air defense simulators fielded after 1985, which are used for development and testing of weapon systems or training of Navy aircrew personnel, must be validated. Earlier simulators will be validated after undergoing a major update or modification, or as funding permits in conjunction with routine operations and maintenance.

b. The validity of air defense threat simulators will be evaluated throughout the lifetime of each simulator. This includes design validation, initial deployment validation, and in-service validation. Design validation will be conducted at the specification review, and reviewed/updated at the critical design review. Initial deployment validation will be conducted on-site at the applicable range or laboratory using measured data and factory acceptance test data as appropriate. In-service validation will be conducted whenever major modification or updates are made to the simulator.

c. Each threat simulator will be subjected to validation procedures or processes (following Commander Naval Air Systems Command Validation Procedures Manual) to establish and document a baseline comparison with its associated threat and to ascertain the extent of the operational and technical performance difference between the two. This validation process will be continued throughout the simulator's life cycle.

d. Validation reports will note and explain the differences between the simulators and the current Defense Intelligence Agency-approved threat data and will describe the impact of the differences on testing or training.

e. Every effect will be made within resource constraints to manage the threat simulator validation program so that there is an organizational difference between the functions of design and validation. If the validation management team is not different from that of the simulator developers, care must be taken to ensure that the results of validation are purely objective and free from bias.

5. Responsibilities

a. Chief of Naval Operations (CNO) will fund the air defense threat simulator validation process. OP-05 will fund the costs involved in validation of those simulators used for combat aircrew training purposes. OP-091 will fund the costs involved in the validation of simulators used for testing purposes. The Navy representative to the EXCOM, CNO (OP-913), will review and forward simulator validation reports to CROSSBOW-S Management Office.

b. Command Naval Air Systems Command (COMNAVAIRSYSCOM) will serve as CNO's technical agent for Navy air defense simulator validation. Validation procedures under this instruction will be implemented. Joint-Service requirements for simulator validation will be coordinated through the CROSSBOW-S Committee. Technical review of simulator validation reports will be coordinated with applicable user program offices for comment. Recommendations to approve simulators as valid for developmental testing will be submitted to CNO (OP-913). Recommendations to approve simulators as valid for training of Navy combat personnel will be submitted via CNO (OP-553) to CNO (OP-913).

c. Commander Operational Test and Evaluation Force will coordinate with COMNAVAIRSYSCOM to review simulator validation reports, and recommend to CNO (OP-913) approval of simulators as valid for the purpose of operational test and evaluation.

6. Action. COMNAVAIRSYSCOM will establish a Navy air defense simulator validation capability for the purpose of validating all air defense threat simulators developed or used by the Navy.

7. Reports. The reporting requirements contained in this instruction are exempt from reports control by SECNAVINST 5214.2B.

Distribution:

SNDL A5 (Chief of Naval Personnel)
A6 (Headquarters, U.S. Marine Corps) (15)
21A (Fleet Commanders in Chief)
22A (Fleet Commanders)
23 (Force Commanders)
24 (Type Commanders)
26F (Operational Test & Evaluation Force and Detachments)
42S (Air Test Evaluation Squadron (VX), Antarctic Development Squadron (VXE) and Oceanographic Development Squadron (VXN)) (AIRTEVRON 1, AIRTEVRON 4, AIRTEVRON 5, only)
C25A (OPNAV Support Activity Detachment) (Ft. Ritchie, only)
E3 (Activities under the command of the Chief of Naval Research)
E7 (Activities under the command of the Auditor General of the Navy)
FA30 (Weapons Training Facility)
FB7 (Air Station PAC) (NAS Fallon, only)
FF8 (Inspection and Survey Board)
FKA1 (Systems Commands)
FKP (Shore activities under the command of COMNAVSEASYSCOM as delegated by CNO)
FKQ (Shore activities under the command of the COMSPAWARSYSCOM as delegated by CNO)
FKR (Shore activities under the command of the COMNAVAIRSYSCOM as delegated by CNO)
FKR4B (Missile Range Facility)
FT1 (Chief of Naval Education and Training)
FT2 (Chief of Naval Air Training)
SNDL V5 (Marine Corps Air Stations) (Beaufort, Yuma, Cherry Point, only)
OPS 02, 03, 351, 04, 05, 501, 554, 07, 76, 08 092, 095, 091, 941
SECNAV/OPNAV Directives Control Office, Washington Navy Yard, Building 200, 1st Floor, Washington D.C. 20374-5074 (20 copies)

Copy to:
SNDL A1 (Immediate Office of the Secretary)
OP-913 (25 copies)

Stocked:
Naval Publications and Forms Directorate
ASO Code 10
5801 Tabor Ave.
Philadelphia, PA 19120-5099 (100 copies)

Pacific Missile Range Facility, Barking Sands, HI

Pinecastle Electronic Warfare Range, Jacksonville, FL

Southern California Offshore Range (SCORE)

Crow Valley, Republic of the Philippines (RP)

Yuma TACTS Range, MCAS Yuma, AZ

Fallon TACTS Range, NAS Fallon, NV

Beaufort TACTS Range, MCAS Beaufort, SC

Mid-Atlantic Electronic Warfare Range, Cherry Point, NC

Atlantic Fleet Weapons Training Facility, Roosevelt Roads, PR

Naval Air Warfare Center-Aircraft Division, Patuxent River, MD

Naval Air Warfare Center, Weapons Division, Point Mugu, CA

Naval Air Warfare Center, Weapons Division, China Lake, CA

Naval Research Laboratory, Washington, DC

Naval Air Development Center, Warminster, PA

Appendix C

NAVAIRINST 3960.00
PMA-253
4 MAY 92

NAVAIR INSTRUCTION 3960

From: Commander, Naval Air Systems Command

Subj: VALIDATION OF NAVY AIR DEFENSE THREAT SIMULATORS

Ref: (a) OPNAVINST 3960.15 (NOTAL)

Encl:
(1) U.S. Navy Electronic Warfare Training Ranges

(2) U.S. Navy Electronic Warfare Test and Evaluation Ranges/Laboratories

1. Purpose. To implement policy and assign responsibilities for the validation of air defense threat simulators being developed under the auspices of the Naval Air Systems Command (NAVAIR), or used by the United States (U.S.) Navy for development and testing of airborne weapon systems, or training of Navy aircrew personnel.

2. Background

a. Reference (a) assigns NAVAIR as the Chief of Naval Operations (CNO) technical agent for Navy Air Defense Simulator validation and tasks NAVAIR to implement validation procedures. Reference (a) requires that all Navy air defense threat simulations fielded during or after 1986, which are used for development and testing of weapon systems, or training of Navy combat personnel, will be validated simulators. Earlier simulations will be validated as required by major update or modification, or as funding permits in conjunction with routine operations and maintenance.

b. Major acquisition and procurement decisions are based on the results of testing developmental hardware against air defense threat simulators. In order for these decisions to be correct, the simulators must be valid representations of the treat. To be a validated simulator does not indicate that every parameter of the simulator is identical to the equivalent parameter of the actual threat system, but that parametric measurements of the simulator fall within tolerances required to support the test/training requirements for which the simulator is to be used. The parameters of the actual threat system are those stated by the current Defense Intelligence Agency (DIA) threat assessment.

3. Scope. This instruction is applicable to all Navy air defense threat simulators located at ranges and laboratories listed in enclosures (1) and (2), which are used for:

a. Developmental phase testing.

b. Operational phase testing and tactics development.

c. Training of Navy aircrew and associated personnel.

4. Policy

a. Threat Simulator Critical Parameters (TSCPs), "are judged as those which interact critically with ECM equipment and are important in the resolution of test and training issues and satisfaction of test and training objectives."

b. The validation process can identify critical parameters which may require instrumentation, to the actual instrumentation requirements are determined by the range user working with the range during mission planning.

c. Deviations will be analyzed, with respect to these TSCPs and the simulator instrumentation requirements, and impacts of these deviations on the mission of the simulator will be determined by the validation process.

d. Based on this analysis, informed decisions will be made by the CNO regarding the validity of simulators to support testing or training.

5. Responsibilities

a. Naval Air Systems Command Headquarters (NAVAIRHQ)

(1) Reconnaissance and Electronic Warfare Systems Program Manager (PMA-253) will:

(a) Obtain funding for the air defense threat simulator validation process from CNO and/or the CROSSBOW-S Committee.

(b) Fund the Naval Air Warfare Center Weapons Division, China Lake (NAVAIRWARCENWPNSDIV China Lake) to establish and independent Simulator Validation Coordinator to coordinate the conduct of unbiased simulator validation at all Navy laboratories and ranges.

(c) Fund Navy Laboratories and Ranges to conduct validation of assigned simulators.

(d) Coordinate NAVAIR review of validation report submitted by the Simulator Validation Coordinator. Recommend to CNO (OP-913) approval of these simulators as valid for the purposes of developmental testing and the training of Navy personnel, and when appropriate recommend waivers to that those systems not formally approved may continue to support limited operations.

(e) Provide Commander, Operational Test and Evaluation Force, material for review of validation reports.

b. NAVAIRWARCENWPNSDIV China Lake

(1) Commanding Officer, NAVAIRWARCENWPNSDIV, China Lake will designate an independent Simulator Validation Coordinator, who has technical expertise in both air defense threat simulator development and electronic warfare. If this Simulator Validation Coordinator is not organizationally different from that of the simulator developers, care must be taken to ensure that the results of validation are entirely objective, and free from bias imposed by the simulator development program.

(2) Simulator Validation Coordinator will:

(a) Operate independent from offices which are responsible for management of simulator development programs.

(b) Manage U.S. Navy simulator validation programs for NAVAIR coordinating with responsible Navy Laboratories and Ranges, as required.

(c) Develop and Maintain a U.S. Navy Air Defense Threat Simulator Validation Procedures Manual, defining validation procedures to be followed by Navy Laboratories and Ranges.

(d) Develop and maintain a validation data base.

(e) Review air defense simulator validation reports prepared by Navy Laboratories and Ranges, and forward the results to NAVAIRHQ (PMA-253/AIR-546/PMA-248).

c. U.S. Navy Laboratories and Ranges will:

(1) Designate a single point of contact to coordinate with the Navy Simulator Validation Coordinator, regarding validation requirements for assigned air defense simulators and validation report format.

(2) Coordinate with the Navy Simulator Validation Coordinator to:

(a) Determine TSCPs, derived from the Navy testing or training requirements for which the simulator was designed.

(b) Collect and analyze simulator performance data using specification data in the case of a simulator in development, or measured data in the case of an existing simulator.

(c) Compare simulator performance data with the corresponding threat parameters from the current DIA threat assessment, and the specific TSCPs.

(d) Submit a validation report, which analyzes the impact on specific testing or training requirements, of any deviations from TSCPs, to the Navy Simulator Validation Coordinator.

6. Reports. The reporting requirements contained in paragraph 5c(2) (d) of this instruction are exempt from reports control by SECNAV Instruction 5214.2B.

Distribution: FKA1A (established quantity); others 2 copies

SNDL: 26F; FKA1A (Deputy Commander for Acquisition and Operations, Assistant Commanders, Comptrollers, Command Special Assistants, Designated Program Managers, Directorate Directors, and Office and Division Directors); FKR

Copy to: (2 copies each unless otherwise indicated)

SNDL: A3 (OP-913 (5 copies); C21 (1 copy); FKA1A (AIR-00D A/L (1 copy), AIR-71232 (10 Series), AIR-71233B(40 copies), PMA-25304, AIR-1002E (1 copy), AIR-546 (5 copies)); RM27 (NPPSO-NDW C/L)

Stocked: Naval Publications and Forms Directorate, Navy Aviation Supply Office, 5801 Tabor Avenue, Philadelphia, PA 19120

U.S. NAVY ELECTRONIC WARFARE TRAINING RANGES

Fallon Complex, Naval Air Station Fallon, NV - FB7 (Fallon)

Marine Corp Air Station Beaufort, SC - V5 (Beaufort)

Atlantic Fleet Weapons Training Facility, Roosevelt Roads, RP - FA30

Southern California Offshore Complex
Fleet Area Control Surveillance Facility
FACSFAC
Naval Air Station, North Island
Bldg. 6, 1479
San Diego, CA 92135-5116

Mid-Atlantic Electronic Warfare Range, Cherry Point, NC
Commanding General
Second Marine Aircraft Wing
Marine Corps Air Station
Cherry Point, NC 28533
Attn: Codes G3 EWO, PSC 4468

Yuma Complex, AZ
Marine Aviation Weapons Tactics Squadron ONE
Marine Corps Air Station
Yuma, AZ 85369-6073

Pacific Missile Range Facility, Barking Sands, Hawaii
Pacific Missile Range Facility (PMRF)
Barking Sands, HI 96752-0128
Attn: Code 7324-3

U.S. NAVY ELECTRONIC WARFARE TEST AND EVALUATION RANGES

 

Naval Air Warfare Center-Aircraft Division, Patuxent River, MD

Naval Air Warfare Center, Weapons Division, Point Mugu, CA

Naval Air Warfare Center, Weapons Division, China Lake, CA

Naval Research Laboratory, DC-E3A*, Washington, D.C.

Naval Air Warfare Center, Aircraft Division, Warminster, PA

*(others under SNDL code "FKR")

Appendix D

ACRONYMS

ACCREDITATION: The process of determining the extent to which a simulator will support the requirements of a specific test, evaluation or training exercise by comparing those requirements with simulator validation documentation.

DIGITAL MODEL: Computer model or mathematical representation of a threat system.

EMITTER: A threat simulator which transmits selected threat electromagnetic emissions. May be unmanned and nonreactive. Normally used for training and the creation of threat densities for specific scenarios.

EMITTER-RECEIVER-PROCESSOR: A threat simulator which duplicates the transmission, reception and processing of critical threat electromagnetic characteristics. Normally a manned, permanent installation (or site) primarily used to satisfy test and training needs where visual recognition and mobility are not required.

FIDELITY: An expression of the extent that threat simulator duplicates the physical and electromagnetic characteristics of a threat system.

HARDWARE SIMULATOR: A general class of threat simulators which includes replicas, emitters, emitter-receiver-processors, surrogates and actual systems.

HYBRID REPRESENTATION: A threat simulator which incorporates some, but not all of the hardware components of a threat system. Other components either are not incorporated at all or are incorporated in the form of a computer simulation.

REPLICA: A mobile threat simulator which has the same electronic and visual signatures as a threat system. Characteristics include visual look-alike at 400 meters with similar human factors design, acoustical and infrared signatures, crew size, and command and control.

SURROGATE: U.S., Allied, or Foreign equipment which exhibits a limited number of specific characteristics of a threat weapon system.

TARGET: Economical, expendable simulator used for tracking and/or live fire missions supporting test and evaluation as well as training missions.

ACTUAL THREAT SYSTEM: Foreign developed or manufactured weapon system including export models. Includes Foreign developed weapon system manufactured by a third country to its own specifications.

THREAT SIMULATOR: A generic term used to describe a family of equipment used to represent threat weapons systems in developmental testing, operational testing and training. A threat simulator has one or more characteristics which, when detected by human senses or man-made sensors, provide the appearance of an actual threat weapons system and behaves similarly to the threat system in an operational environment with a prescribed degree of fidelity.

VERIFICATION: Analysis and testing to confirm a simulator or simulation satisfies documented requirements and is built as designed.