ALBERT

Description: An eye-tracking experiment was conducted to examine whether differences in forecasters’ eye movements provide further insight into how radar update speed impacts their warning decision process. In doing so, this study also demonstrates the applications of a new research method for observing how National Weather Service forecasters distribute their attention across a radar display and warning interface. In addition to observing forecasters’ eye movements during this experiment, video data and retrospective recalls were collected. These qualitative data were used to provide an explanation for differences observed in forecasters’ eye movements. Eye movement differences were analyzed with respect to fixation measures (i.e., count and duration) and scanpath dimensions (i.e., vector, direction, length, position, and duration). These analyses were completed for four stages of the warning decision process: the first 5 min of the case, 2 min prior to warning decisions, the warning issuance process, and warning updates. While radar update speed did not impact forecasters’ fixation measures during these four stages, comparisons of scanpath dimensions revealed differences in their eye movements. Video footage and retrospective recall data illustrated how forecasters’ interactions with the radar display and warning interface, encounters with technological challenges, and varying approaches to similar tasks resulted in statistically significantly (p value 〈 0.05) lower scanpath similarity scores. The findings of this study support the combined use of eye-tracking and qualitative research methods for detecting and understanding individual differences in forecasters’ eye movements. Future applications of these methods in operational meteorology research have potential to aid usability studies and improve human–computer interactions for forecasters.

Description: Impacts of radar update time on forecasters’ warning decision processes were analyzed in the 2015 Phased Array Radar Innovative Sensing Experiment. Thirty National Weather Service forecasters worked nine archived phased-array radar (PAR) cases in simulated real time. These cases presented nonsevere, severe hail and/or wind, and tornadic events. Forecasters worked each type of event with approximately 5-min (quarter speed), 2-min (half speed), and 1-min (full speed) PAR updates. Warning performance was analyzed with respect to lead time and verification. Combining all cases, forecasters’ median warning lead times when using full-, half-, and quarter-speed PAR updates were 17, 14.5, and 13.6 min, respectively. The use of faster PAR updates also resulted in higher probability of detection and lower false alarm ratio scores. Radar update speed did not impact warning duration or size. Analysis of forecaster performance on a case-by-case basis showed that the impact of PAR update speed varied depending on the situation. This impact was most noticeable during the tornadic cases, where radar update speed positively impacted tornado warning lead time during two supercell events, but not for a short-lived tornado occurring within a bowing line segment. Forecasters’ improved ability to correctly discriminate the severe weather threat during a nontornadic supercell event with faster PAR updates was also demonstrated. Forecasters provided subjective assessments of their cognitive workload in all nine cases. On average, forecasters were not cognitively overloaded, but some participants did experience higher levels of cognitive workload at times. A qualitative explanation of these particular instances is provided.

Description: Eye-tracking technology can observe where and how someone’s eye gaze is directed, and therefore provides information about one’s attention and related cognitive processes in real time. The use of eye-tracking methods is evident in a variety of research domains, and has been used on few occasions within the meteorology community. With the goals of Weather Ready Nation in mind, eye-tracking applications in meteorology have so far supported the need to address how people interpret meteorological information through televised forecasts and graphics. However, eye tracking has not yet been applied to learning about forecaster behavior and decision processes. In this article, we consider what current methods are being used to study forecasters and why we believe eye tracking is a method that should be incorporated into our efforts. We share our first data collection of an NWS forecaster’s eye gaze data, and explore the types of information that these data provide about the forecaster’s cognitive processes. We also discuss how eye-tracking methods could be applied to other aspects of operational meteorology research in the future, and provide motivation for further exploration on this topic.

Description: On 14 June 2011, an intense multicell thunderstorm produced one nonsevere and three severe downbursts within 35 km of the rapid-update, S-band phased array radar (PAR) at the National Weather Radar Testbed in Norman, Oklahoma, and the nearby polarimetric research Weather Surveillance Radar 1988-Doppler (KOUN). Data collected from these radars provided the opportunity to conduct a quantitative analysis of downburst precursor signature evolution depicted by 1-min PAR data and the associated evolution of differential reflectivity ZDR depicted by 5-min KOUN data. Precursors analyzed included descent of the reflectivity core, evolution of the magnitude and size of midlevel convergence (i.e., number of bins), and descending “troughs” of ZDR. The four downbursts exhibited midlevel convergence that rapidly increased to peak magnitude as the reflectivity core (65-dBZ isosurface) bottom and top descended. The ZDR troughs seen in the 5-min KOUN data appeared to descend along with the core bottom. Midlevel convergence size increased to a peak value and decreased as the reflectivity core descended in the three severe downbursts. In contrast, midlevel convergence size exhibited little change in the nonsevere downburst. The time scale of trends seen in the PAR data was 11 min or less and happened several minutes prior to each downburst’s maximum intensity. These results point to the importance of 1-min volumetric data in effectively resolving the evolution of downburst precursors, which could be beneficial to forecast operations.

Description: The 2013 Phased Array Radar Innovative Sensing Experiment (PARISE) investigated the impacts of higher-temporal-resolution radar data on National Weather Service forecasters’ warning decision processes during severe hail and wind events. In total, 12 forecasters participated in the 2013 PARISE over a 6-week period during the summer of 2013. Participants were assigned to either a control [5-min phased-array radar (PAR) updates] or experimental (1-min PAR updates) group, and worked two cases in simulated real time. This paper focuses on the qualitative retrospective reports of participants’ warning decision processes that were collected using the recent case walk-through method. Timelines of participants’ warning decision process were created for both cases, which were then thematically coded according to a situational awareness framework. Coded themes included perception, comprehension, and projection. It was found that the experimental group perceived significantly more information during both cases than the control group (case 1 p = 0.045 and case 2 p = 0.041), which may have improved the quality of their comprehensions and projections. Analysis of timelines reveals that 1-min PAR updates were important to the experimental group’s more timely and accurate warning decisions. Not only did the 1-min PAR updates enable experimental participants to perceive precursor signatures earlier than control participants, but through monitoring trends in radar data, the experimental group was able to better detect storm motion, more accurately identify expected weather threats from severe thunderstorms, more easily observe strengthening and diminishing trends in storms, and make more correct tornado-related warning decisions.

Description: On 31 May 2013, a polarimetric WSR-88D located in Norman, Oklahoma (KOUN), was used to collect sectorized volumetric observations in a tornadic supercell. Because only a fraction of the full azimuthal volume was observed, rapid volume update times of ~1–2 min were achieved. In addition, the number of pulses used in each radial was larger than is conventional, increasing the statistical robustness of the calculated polarimetric variables. These rapid observations serve as a proxy for those of a future dual-polarized phased-array radar. Through comparison with contemporaneous observations from two nearby dual-polarized WSR-88Ds [Twin Lakes, Oklahoma (KTLX), and near University of Oklahoma Westheimer Airport in Norman (KCRI)], a number of instances in which the rapidly scanned KOUN radar detected or better resolved (in a temporal sense) features of severe convective storms are highlighted. In particular, the polarimetric signatures of merging updrafts, a rapidly descending giant hail core, an anticyclonic tornado, and a dissipating storm cell are examined. These observations provided insights into the rapid evolution of severe convective storms that could not be made (or would have been made with much lower confidence) with current, operational WSR-88D scanning strategies. Possible implications of these rapid updates for the warning decision process are discussed.

Description: Thirty National Weather Service forecasters worked with 1-, 2-, and 5-min phased-array radar (PAR) volumetric updates for a variety of weather events during the 2015 Phased Array Radar Innovative Sensing Experiment. Exposure to each of these temporal resolutions during simulated warning operations meant that these forecasters could provide valuable feedback on how rapidly updating PAR data impacted their warning decision processes. To capture this feedback, forecasters participated in one of six focus groups. A series of open-ended questions guided focus group discussions, and forecasters were encouraged to share their experiences and opinions from the experiment. Transcriptions of focus group discussions were thematically analyzed, and themes belonging to one of two groups were identified: 1) forecasters’ use of rapidly updating PAR data during the experiment and 2) how forecasters envision rapidly updating PAR data being integrated into warning operations. Findings from this thematic analysis are presented in this paper, and to illustrate these findings from the forecasters’ perspectives, dialogue that captures the essence of their discussions is shared. The identified themes provide motivation to integrate rapidly updating radar data into warning operations and highlight important factors that need to be addressed for the successful integration of these data.

Description: Many public safety officials (e.g., emergency managers and first responders) use weather-radar data to inform many life-saving decisions, such as sounding outdoor warning sirens and directing storm spotters. Therefore, to include this important user group in ongoing radar applications research, a knowledge coproduction framework is used to interact with, learn from, and provide information to public safety officials. From these interactions, it became clear that radar-based products that estimate a tornado’s location, intensity, or both could be valuable to public safety officials. Therefore, a survey was conducted and a focus group formed to 1) collect feedback on several of these products currently under development, 2) identify potential decisions that could be made with these products, and 3) examine the impact of radar update time on product usefulness. An analysis of the survey and focus group responses revealed that public safety officials preferred simple interactive products provided to them using multiple communication methods. Once received, any product that could clearly communicate where a tornado may have occurred would likely help public safety officials focus search and rescue efforts in the immediate aftermath of a tornado. Additionally, public safety officials preferred products created using rapid-update data (1–2-min volumetric updates) over conventional-update data (4–5-min volumetric updates) because it provided them with more complete information.

Description: An object-based verification methodology for the NSSL Experimental Warn-on-Forecast System for ensembles (NEWS-e) has been developed and applied to 32 cases between December 2015 and June 2017. NEWS-e forecast objects of composite reflectivity and 30-min updraft helicity swaths are matched to corresponding reflectivity and rotation track objects in Multi-Radar Multi-Sensor system data on space and time scales typical of a National Weather Service warning. Object matching allows contingency-table-based verification statistics to be used to establish baseline performance metrics for NEWS-e thunderstorm and mesocyclone forecasts. NEWS-e critical success index (CSI) scores of reflectivity (updraft helicity) forecasts decrease from approximately 0.7 (0.4) to 0.4 (0.2) over 3 h of forecast time. CSI scores decrease through the forecast period, indicating that errors do not saturate during the 3-h forecast. Lower verification scores for rotation track forecasts are primarily a result of a high-frequency bias. Comparison of different system configurations used in 2016 and 2017 shows an increase in skill for 2017 reflectivity forecasts, attributable mainly to improvements in the forecast initial conditions. A small decrease in skill in 2017 rotation track forecasts is likely a result of sample differences between 2016 and 2017. Although large case-to-case variation is present, evidence is found that NEWS-e forecast skill improves with increasing object size and intensity.