ALBERT

Description: Functional Near Infrared Spectroscopy (fNIRS) is an emerging neuronal measurement technique with many advantages for application in operational and training contexts. Instrumentation and protocol improvements, however, are required to obtain useful signals and produce expeditiously self-applicable, comfortable and unobtrusive headgear. Approaches for improving the validity and reliability of fNIRS data for the purpose of sensing the mental state of commercial aircraft operators are identified, and an exemplary system design for attentional state monitoring is outlined. Intelligent flight decks of the future can be responsive to state changes to optimally support human performance. Thus, the identification of cognitive performance decrement, such as lapses in operator attention, may be used to predict and avoid error-prone states. We propose that attentional performance may be monitored with fNIRS through the quantification of hemodynamic activations in cortical regions which are part of functionally-connected attention and resting state networks. Activations in these regions have been shown to correlate with behavioral performance and task engagement. These regions lie beneath superficial tissue in head regions beyond the forehead. Headgear development is key to reliably and robustly accessing locations beyond the hair line to measure functionally-connected networks across the whole head. Human subject trials using both fNIRS and functional Magnetic Resonance Imaging (fMRI) will be used to test this system. Data processing employs Support Vector Machines for state classification based on the fNIRS signals. If accurate state classification is achieved based on sensed activation patterns, fNIRS will be shown to be useful for monitoring attentional performance.

Description: Attention-related human performance limiting states (AHPLS) can cause pilots to lose airplane state awareness (ASA), and their detection is important to improving commercial aviation safety. The Commercial Aviation Safety Team found that the majority of recent international commercial aviation accidents attributable to loss of control inflight involved flight crew loss of airplane state awareness, and that distraction of various forms was involved in all of them. Research on AHPLS, including channelized attention, diverted attention, startle / surprise, and confirmation bias, has been recommended in a Safety Enhancement (SE) entitled "Training for Attention Management." To accomplish the detection of such cognitive and psychophysiological states, a broad suite of sensors has been implemented to simultaneously measure their physiological markers during high fidelity flight simulation human subject studies. Pilot participants were asked to perform benchmark tasks and experimental flight scenarios designed to induce AHPLS. Pattern classification was employed to distinguish the AHPLS induced by the benchmark tasks. Unimodal classification using pre-processed electroencephalography (EEG) signals as input features to extreme gradient boosting, random forest and deep neural network multiclass classifiers was implemented. Multi-modal classification using galvanic skin response (GSR) in addition to the same EEG signals and using the same types of classifiers produced increased accuracy with respect to the unimodal case (90 percent vs. 86 percent), although only via the deep neural network classifier. These initial results are a first step toward the goal of demonstrating simultaneous real time classification of multiple states using multiple sensing modalities in high-fidelity flight simulators. This detection is intended to support and inform training methods under development to mitigate the loss of ASA and thus reduce accidents and incidents.

Description: This paper reviews the effects of varying gravitational levels on functional Near-Infrared Spectroscopy (fNIRS) headgear. The fNIRS systems quantify neural activations in the cortex by measuring hemoglobin concentration changes via optical intensity. Such activation measurement allows for the detection of cognitive state, which can be important for emotional stability, human performance and vigilance optimization, and the detection of hazardous operator state. The technique depends on coupling between the fNIRS probe and users skin. Such coupling may be highly susceptible to motion if probe-containing headgear designs are not adequately tested. The lack of reliable and self-applicable headgear robust to the influence of motion artifact currently inhibits its operational use in aerospace environments. Both NASAs Aviation Safety and Human Research Programs are interested in this technology as a method of monitoring cognitive state of pilots and crew.

Description: Brain activity can predict a person's level of engagement in an attentional task. However, estimates of brain activity are often confounded by measurement artifacts and systemic physiological noise. The optimal method for filtering this noise - thereby increasing such state prediction accuracy - remains unclear. To investigate this, we asked study participants to perform an attentional task while we monitored their brain activity with functional near infrared spectroscopy (fNIRS). We observed higher state prediction accuracy when noise in the fNIRS hemoglobin [Hb] signals was filtered with a non-stationary (adaptive) model as compared to static regression (84% +/- 6% versus 72% +/- 15%).

Description: This paper provides an overview of utilizing the NASA KC-135 Reduced Gravity Aircraft for the Foam Optics and Mechanics (FOAM) microgravity flight project. The FOAM science requirements are summarized, and the KC-135 test-rig used to test hardware concepts designed to meet the requirements are described. Preliminary results regarding foam dispensing, foam/surface slip tests, and dynamic light scattering data are discussed in support of the flight hardware development for the FOAM experiment.

Description: Method and systems are disclosed for training state-classifiers for classification of cognitive state. A set of multimodal signals indicating physiological responses of an operator are sampled over a time period. A depiction of operation by the operator during the time period is displayed. In response to user input selecting a cognitive state for a portion of the time period, the one or more state-classifiers are trained. In training the state-classifiers, the set of multimodal signals sampled in the portion of the time period are used as input to the one or more state-classifiers and the selected one of the set of cognitive states is used as a target result to be indicated by the one or more state-classifiers.

Description: The Commercial Aviation Safety Team found the majority of recent international commercial aviation accidents attributable to loss of control inflight involved flight crew loss of airplane state awareness (ASA), and distraction was involved in all of them. Research on attention-related human performance limiting states (AHPLS) such as channelized attention, diverted attention, startle/surprise, and confirmation bias, has been recommended in a Safety Enhancement (SE) entitled "Training for Attention Management." To accomplish the detection of such cognitive and psychophysiological states, a broad suite of sensors was implemented to simultaneously measure their physiological markers during a high fidelity flight simulation human subject study. Twenty-four pilot participants were asked to wear the sensors while they performed benchmark tasks and motion-based flight scenarios designed to induce AHPLS. Pattern classification was employed to predict the occurrence of AHPLS during flight simulation also designed to induce those states. Classifier training data were collected during performance of the benchmark tasks. Multimodal classification was performed, using pre-processed electroencephalography, galvanic skin response, electrocardiogram, and respiration signals as input features. A combination of one, some or all modalities were used. Extreme gradient boosting, random forest and two support vector machine classifiers were implemented. The best accuracy for each modality-classifier combination is reported. Results using a select set of features and using the full set of available features are presented. Further, results are presented for training one classifier with the combined features and for training multiple classifiers with features from each modality separately. Using the select set of features and combined training, multistate prediction accuracy averaged 0.64 +/- 0.14 across thirteen participants and was significantly higher than that for the separate training case. These results support the goal of demonstrating simultaneous real-time classification of multiple states using multiple sensing modalities in high fidelity flight simulators. This detection is intended to support and inform training methods under development to mitigate the loss of ASA and thus reduce accidents and incidents.

Description: Heart rate complexity (HRC) is a proven metric for gaining insight into human stress and physiological deterioration. To calculate HRC, the detection of the exact instance of when the heart beats, the R-peak, is necessary. Electrocardiogram (ECG) signals can often be corrupted by environmental noise (e.g., from electromagnetic interference, movement artifacts), which can potentially alter the HRC measurement, producing erroneous inputs which feed into decision support models. Current literature has only investigated how HRC is affected by noise when R-peak detection errors occur (false positives and false negatives). However, the numerical methods used to calculate HRC are also sensitive to the specific location of the fiducial point of the R-peak. This raises many questions regarding how this fiducial point is altered by noise, the resulting impact on the measured HRC, and how we can account for noisy HRC measures as inputs into our decision models. This work uses Monte Carlo simulations to systematically add white and pink noise at different permutations of signal-to-noise ratios (SNRs), time segments, sampling rates, and HRC measurements to characterize the influence of noise on the HRC measure by altering the fiducial point of the R-peak. Using the generated information from these simulations provides improved decision processes for system design which address key concerns such as permutation entropy being a more precise, reliable, less biased, and more sensitive measurement for HRC than sample and approximate entropy.