Publication Date:
2022-03-17
Description:
A new model validation and performance assessment tool is introduced, the sliding thresholdof observation for numeric evaluation (STONE) curve. It is based on the relative operating characteristic(ROC) curve technique, but instead of sorting all observations in a categorical classification, the STONE tooluses the continuous nature of the observations. Rather than defining events in the observations and thensliding the threshold only in the classifier/model data set, the threshold is changed simultaneously for boththe observational and model values, with the same threshold value for both data and model. This is onlypossible if the observations are continuous and the model output is in the same units and scale as theobservations, that is, the model is trying to exactly reproduce the data. The STONE curve has severalsimilarities with the ROC curve—plotting probability of detection against probability of false detection,ranging from the (1,1) corner for low thresholds to the (0,0) corner for high thresholds, and values above thezero‐intercept unity‐slope line indicating better than random predictive ability. The main difference isthat the STONE curve can be nonmonotonic, doubling back in both thexandydirections. These ripplesreveal asymmetries in the data‐model value pairs. This new technique is applied to modeling output of acommon geomagnetic activity index as well as energetic electronfluxes in the Earth's innermagnetosphere. It is not limited to space physics applications but can be used for any scientificorengineeringfield where numerical models are used to reproduce observations.
Language:
English
Type:
info:eu-repo/semantics/article
Format:
application/pdf
Permalink