ALBERT

Description: Regulations that establish operational and performance requirements for unmanned aircraft systems (UAS) are being developed by a consortium of government, industry and academic institutions. Those requirements will apply to detect-and-avoid (DAA) systems and other equipment necessary to integrate UAS with the National Airspace System (NAS) and are determined according to their contribution to the overall level of safety required to operate in the airspace. Several key gaps must be addressed in order to link equipment requirements to an airspace level of safety. Foremost among these is the calculation of the relative effectiveness of a particular system to mitigate violations of a separation standard with other aircraft, which is known as the systems risk ratio. The risk ratio is calculated as the probability of mid-air collision with a DAA system divided by the probability of mid-air collision without a DAA system. The risk ratio of a DAA system, in combination with the risk ratios of other collision avoidance mitigations, will determine the overall safety of the airspace measured in terms of the number of mid-air collisions per flight hour. Defining the required risk ratio that the DAA system needs to ensure the safety of the airspace requires an evaluation of the current airspace and a simulated evaluation that incorporates UAS aerodynamic performance and the mission characteristics of future UAS operations that are projected to be conducted in areas that interact with current operations. These evaluations will produce the frequency of encounters that currently exist in the airspace and those that could be generated with the introduction of UAS. Together, the frequency of encounters, an evaluation of unmitigated risk of collision, and a desired level of safety of the airspace will yield a required risk ratio of the DAA system. This study will focus on evaluating the encounter rates between aircraft based on historical radar data and encounter rates that could occur based on simulated UAS missions.

Description: A key challenge to the routine, safe operation of unmanned aircraft systems (UAS) is the development of detect-and-avoid (DAA) systems to aid the UAS pilot in remaining "well clear" of nearby aircraft. The goal of this study is to investigate the effect of alerting criteria and pilot response delay on the safety and performance of UAS DAA systems in the context of routine civil UAS operations in the National Airspace System (NAS). A NAS-wide fast-time simulation study was conducted to assess UAS DAA system performance with a large number of encounters and a broad set of DAA alerting and guidance system parameters. Three attributes of the DAA system were controlled as independent variables in the study to conduct trade-off analyses: UAS trajectory prediction method (dead-reckoning vs. intent-based), alerting time threshold (related to predicted time to LoWC), and alerting distance threshold (related to predicted Horizontal Miss Distance, or HMD). A set of metrics, such as the percentage of true positive, false positive, and missed alerts, based on signal detection theory and analysis methods utilizing the Receiver Operating Characteristic (ROC) curves were proposed to evaluate the safety and performance of DAA alerting and guidance systems and aid development of DAA system performance standards. The effect of pilot response delay on the performance of DAA systems was evaluated using a DAA alerting and guidance model and a pilot model developed to support this study. A total of 18 fast-time simulations were conducted with nine different DAA alerting threshold settings and two different trajectory prediction methods, using recorded radar traffic from current Visual Flight Rules (VFR) operations, and supplemented with DAA-equipped UAS traffic based on mission profiles modeling future UAS operations. Results indicate DAA alerting distance threshold has a greater effect on DAA system performance than DAA alerting time threshold or ownship trajectory prediction method. Further analysis on the alert lead time (time in advance of predicted loss of well clear at which a DAA alert is first issued) indicated a strong positive correlation between alert lead time and DAA system performance (i.e. the ability of the UAS pilot to maneuver the unmanned aircraft to remain well clear). While bigger distance thresholds had beneficial effects on alert lead time and missed alert rate, it also generated a higher rate of false alerts. In the design and development of DAA alerting and guidance systems, therefore, the positive and negative effects of false alerts and missed alerts should be carefully considered to achieve acceptable alerting system performance by balancing false and missed alerts. The results and methodology presented in this study are expected to help stakeholders, policymakers and standards committees define the appropriate setting of DAA system parameter thresholds for UAS that ensure safety while minimizing operational impacts to the NAS and equipage requirements for its users before DAA operational performance standards can be finalized.

Description: Unmanned aircraft systems (UAS) will be required to equip with a detect-and-avoid (DAA) system in order to satisfy the federal aviation regulations to maintain well clear of other aircraft, some of which may be equipped with a Traffic Collision Avoidance System (TCAS) to mitigate the possibility of mid-air collisions. As such, the minimum operational performance standards (MOPS) for UAS DAA systems are being designed with TCAS interoperability in mind by a group of industry, government, and academic institutions named RTCA Special Committee-228 (SC-228). This document will discuss the development of the spatial-temporal volume known as the collision avoidance region in which the DAA system is not allowed to provide vertical guidance to maintain or regain DAA well clear that could conflict with resolution advisories (RAs) issued by the intruder aircraft's TCAS system. Three collision avoidance region definition candidates were developed based on the existing TCAS RA and DAA alerting definitions. They were evaluated against each other in terms of their interoperability with TCAS RAs and DAA alerts in an unmitigated factorial encounter analysis of 1.3 million simulated pairs.

Description: Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements for DAA alerting system performance. A recent study conducted using NASA's ACES (Airspace Concept Evaluation System) simulation capability begins to address questions surrounding the development of draft MOPS for DAA alerting systems. ACES simulations were conducted to study the performance of alerting systems proposed by the SC-228 DAA Alerting sub-group. Analysis included but was not limited to: 1) correct alert (and timeliness), 2) false alert (and severity and duration), 3) missed alert, and 4) probability of an alert type at the time of loss of well clear. The performance of DAA alerting systems when using intent vs. dead-reckoning for UAS ownship trajectories was also compared. The results will be used by SC-228 to inform decisions about the surveillance standards of UAS DAA systems and future requirements development and validation efforts.

Description: Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 (SC-228) is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements as well as the safety substantiation and end-to-end assessment of DAA system performance. With regard to the safety aspect being studied by the SC-228 DAA Safety sub-group, NASA has conducted a study using the ACES (Airspace Concept Evaluation System) simulation capability to determine: 1) the rate at which IFR aircraft encounter other IFR and VFR aircraft, and 2) the rate at which UAS aircraft encounter VFR aircraft as well as the corresponding encounter geometries. Five different separation thresholds were used (two for encounter and one each for well-clear, near mid-air collision, and closest point of approach). The results will be used by the SC-228 DAA Safety sub-group to inform decisions about the safety aspect of UAS DAA systems and future requirements development and validation efforts.

Description: Most unmanned aircraft systems will be required to be equipped with a detect-and-avoid system that is capable of maintaining appropriate separation from other aircraft. One of the critical components of detect-and-avoid systems is a surveillance system that identifies potential threat aircraft in real time and tracks these aircraft so that their future trajectories may be used to predict conflicts. The performance of the detect-and-avoid system generally depends on technical parameters of the surveillance system, such as the surveillance range. The quantitative requirements for detect-and-avoid systems will be determined to meet safety metrics for the operation of unmanned aircraft systems in the National Airspace System. This study employs a sensor model comprised of the surveillance range, and horizontal and vertical fields of regard that mainly characterize the overall performance of a surveillance system. In this study, potential metrics for evaluating the performance of a surveillance system were investigated through fast-time simulation with a traffic scenario that included both proposed unmanned aircraft flights and historical visual flight rule aircraft tracks. Using the simulation results, an overall analysis of encounter geometry highlights the encounter characteristics that relate surveillance parameters to safety metrics and detect-and-avoid system performance. Then, given several candidate surveillance volumes, performance and safety metrics are derived; these metrics include the ratio of undetected and late-detected violations and the time to violation at first detection. These example metrics demonstrate the utility of the database of encounters created in this work, a database which will be useful in the derivation of required detect-and-avoid surveillance system requirements.

Description: Most unmanned aircraft systems will be required to be equipped with a Detect-and-Avoid (DAA) system with a surveillance component. The surveillance performance requirements of the DAA system to detect and track intruder aircraft will depend on the encounter geometries that unmanned aircraft are expected to have with other aircraft in the airspace. This presentation shows the analysis of the encounter geometries that were simulated using historical low-altitude traffic data and some proposed UAS missions. This analysis suggests how the overall safety and performance of a surveillance system may relate to surveillance parameters such as surveillance range, horizontal and vertical fields or regard. This study proposed and investigated potential safety and performance metrics for evaluating the performance of a surveillance system, such as the ratio of undetected and late-detected separation violations, and the time to violation at first detection for given sets of surveillance parameters.

Description: NASA is working on the project for Unmanned Aircraft System (UAS) integration into the National Airspace System (NAS) and helping the RTCA's Special Committee 228 (RTCA SC-228) to develop the minimum operational performance standards (MOPS) for detect-and avoid (DAA) systems. The Ames' Separation Assurance-Sense and Avoid Interoperability (SSI) team for the project is supporting this effort through fast-time simulation studies using the Airspace Concept Evaluation System (ACES) simulation platform. This presentation introduces the Ames SSI team's tasks and shows some results from accomplished fast-time simulation studies. It is a part of overall presentations for NASA's efforts for the UAS integration into the NAS project.

Description: Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228, is committed to supporting the development and validation of draft requirements as well as the safety substantiation and end-to-end assessment of DAA system performance. A recent study conducted using NASA's ACES (Airspace Concept Evaluation System) simulation capability begins to address questions surrounding the development of draft MOPS for DAA systems. ACES analyses were conducted to determine: 1) the rate at which IFR aircraft encounter other IFR and VFR aircraft, and 2) the rate at which UAS aircraft encounter VFR aircraft. Five different separation thresholds were used (two for encounter and one each for well-clear, near mid-air collision, and closest point of approach). The results will be used by SC228 to inform decisions about the safety aspect of UAS DAA systems and future requirements development and validation efforts.

Description: Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements for DAA alerting and guidance systems. This presentation contains the results of two combinatorial encounter analysis studies using NASA's SAA Control fast-time simulation capability for this purpose. In these studies, encounters between two aircraft were simulated one at a time for the full factorial combination of encounter geometries (e.g., encounter angle, CPA offset) and aircraft performance (e.g., ownership and intruder ground speeds and vertical rates). The first study analyzes the relationships (e.g., timeline) between the different alerting-safety regions in the SC-228 MOPS (in order of increasing severity): 1) DAA warning alert, 2) well clear recovery (WCR) guidance, 3) DAA-Collision Avoidance (CA), and 4) TCAS RA. This study will focus primarily on encounter situations in which TCAS RA occurs prior to any of the other alerting-safety boundaries. In particular, this study will investigate whether using vertical distance or vertical distance at closest point of approach (i.e., vertical miss distance or VMD) is more appropriate for the definition of the DAA-CA region. In addition, cases where transitions between different regions skip an intermediate region will be analyzed. The second study in this presentation explores a proposal to use an altitude rate error threshold to determine if vertical maneuvers are acceptable for DAA WCR guidance against non-cooperative intruders. This study incorporates the radar from the Honeywell sensor model and examines a series of pairwise encounters between a non-cooperative intruder and a UAS ownship, with different combinations of intruder states and ownship performance levels. The study uses SAA Control as a simulation platform and pilot model, and Omnibands to provide DWC recovery guidance. Two simulation sets, one that allows vertical DWC recovery guidance and one that does not, are compared to determine if encounters with altitude rate errors above 250 feet-per-minute are more likely to have more severe losses of well clear, as determined by the Loss of Well-Clear Severity metric.