ALBERT

Description: We describe the setup of testing regions for the China Earthquake Forecast Testing Center and provide preliminary forecast results in the scope of the Collaboratory for the Study of Earthquake Predictability (CSEP) project. We investigate the spatiotemporal variations of the completeness magnitude M c by using the frequency-magnitude distribution of the China Earthquake Networks Center (CENC) catalog. We find three periods of significantly different M c histories: (I) 1 January 1970–30 September 2001, (II) 1 October 2001–30 September 2008, and (III) 1 October 2008–31 August 2011. M c mapping provides median values , 2.2, and 1.6 for the three periods of time, respectively, showing the improvement in catalog completeness over time. We recommend using data from periods II and III to define a baseline long enough for retrospective forecast testing. Small magnitude events from period I should be used with caution due to important fluctuations in completeness. For period III, coordinates of all national and regional seismic stations are available, and we therefore apply the Bayesian magnitude of completeness (BMC) technique, mapping M c continuously by using prior information on the relationship between M c and the density of seismic stations. We define four potential testing/collection areas for CSEP-China: (A) All China, (B) North–South Seismic Belt (NSSB), (C) North and West Xinjiang Seismic Region, and (D) North China Seismic Region. In the current phase of CSEP-China, only the NSSB (region B) is considered. To demonstrate the type of earthquake predictability experiment that will be performed in the Chinese Testing Center, we present a series of retrospective forecast experiments with TripleS, a smoothed seismicity model. Online Material: The CENC earthquake catalog (1 January 1970–31 August 2011, restricted to magnitudes M ≥3.0) as well as completeness magnitude M c ( x , y ) spatial grids.

Description: The Regional Earthquake Likelihood Models experiment in California tested the performance of earthquake likelihood models over a five-year period. First-order analysis showed a smoothed-seismicity model by Helmstetter et al. (2007) to be the best model. We construct optimal multiplicative hybrids involving the best individual model as a baseline and one or more conjugate models. Conjugate models are transformed using an order-preserving function. Two parameters for each conjugate model and an overall normalizing constant are fitted to optimize the hybrid model. Many two-model hybrids have an appreciable information gain (log probability gain) per earthquake relative to the best individual model. For the whole of California, the Bird and Liu (2007) Neokinema and Holliday et al. (2007) pattern informatics (PI) models both give gains close to 0.25. For southern California, the Shen et al. (2007) geodetic model gives a gain of more than 0.5, and several others give gains of about 0.2. The best three-model hybrid for the whole region has the Neokinema and PI models as conjugates. The best three-model hybrid for southern California has the Shen et al. (2007) and PI models as conjugates. The information gains of the best multiplicative hybrids are greater than those of additive hybrids constructed from the same set of models. The gains tend to be larger when the contributing models involve markedly different concepts or data. These results need to be confirmed by further prospective tests. Multiplicative hybrids will be useful for assimilating other earthquake-related observations into forecasting models and for combining forecasting models at all timescales.

Description: The assessment of earthquake forecast models for practical purposes requires more than simply checking model consistency in a statistical framework. One also needs to understand how to construct the best model for specific forecasting applications. We describe a Bayesian approach to evaluating earthquake forecasting models, and we consider related procedures for constructing ensemble forecasts. We show how evaluations based on Bayes factors, which measure the relative skill among forecasts, can be complementary to common goodness-of-fit tests used to measure the absolute consistency of forecasts with data. To construct ensemble forecasts, we consider averages across a forecast set, weighted by either posterior probabilities or inverse log-likelihoods derived during prospective earthquake forecasting experiments. We account for model correlations by conditioning weights using the Garthwaite–Mubwandarikwa capped eigenvalue scheme. We apply these methods to the Regional Earthquake Likelihood Models (RELM) five-year earthquake forecast experiment in California, and we discuss how this approach can be generalized to other ensemble forecasting applications. Specific applications of seismological importance include experiments being conducted within the Collaboratory for the Study of Earthquake Predictability (CSEP) and ensemble methods for operational earthquake forecasting. Online Material: Tables of likelihoods for each testing phase and code to analyze the RELM experiment.

Description: Using analogies to gaming, we consider the problem of comparing multiple probabilistic seismicity forecasts. To measure relative model performance, we suggest a parimutuel gambling perspective which addresses shortcomings of other methods such as likelihood ratio, information gain and Molchan diagrams. We describe two variants of the parimutuel approach for a set of forecasts: head-to-head, in which forecasts are compared in pairs, and round table, in which all forecasts are compared simultaneously. For illustration, we compare the 5-yr forecasts of the Regional Earthquake Likelihood Models experiment for M 4.95+ seismicity in California.

Description: We present a testable stochastic earthquake source model for intermediate- to long-term forecasts. The model is based on fundamental observations: the frequency-magnitude distribution, slip rates on major faults, long-term strain rates, and source parameter values of instrumentally recorded and historic earthquakes. The basic building blocks of the model are two pairs of probability density maps. The first pair consists of smoothed seismicity and weighted focal mechanisms based on observed earthquakes. The second pair corresponds to mapped faults and their slip rates and consists of smoothed moment-rate and weighted focal mechanisms based on fault geometry. We construct from the model a "stochastic event set," that is to say, a large set of simulated earthquakes that are relevant for seismic hazard calculations and earthquake forecast development. Their complete descriptions are determined in the following order: magnitude, epicenter, moment tensor, length, displacement, and down-dip width. Our approach assures by construction that the simulated magnitudes are consistent with the observed frequency-magnitude distribution. We employ a magnitude-dependent weighting procedure that tends to place the largest simulated earthquakes near major faults with consistent focal mechanisms. Nevertheless, our stochastic model allows for surprises such as large off-fault earthquakes, events that comply with the observation that several recent destructive earthquakes occurred on previously unknown fault structures. We apply our model to California to illustrate its features.

Description: The Regional Earthquake Likelihood Models (RELM) working group designed a 5-year experiment to forecast the number, spatial distribution, and magnitude distribution of subsequent target earthquakes, defined to be those with magnitude ≥4.95 ( M 4.95+) in a well-defined California testing region. Included in the experiment specification were the description of the data source, the methods for data processing, and the proposed evaluation metrics. The RELM experiment began on 1 January 2006 and involved 17 time-invariant forecasts constructed by seismicity modelers; by the end of the experiment on 1 January 2011, 31 target earthquakes had occurred. We analyze the experiment outcome by applying the proposed consistency tests based on likelihood measures and additional comparison tests based on a measure of information gain. We find that the smoothed seismicity forecast by Helmstetter et al. , 2007 based on M 2+ earthquakes since 1981, is the best forecast, regardless of whether aftershocks are included in the analysis. The RELM experiment has helped to clarify ideas about testing that can be applied to more wide-ranging earthquake forecasting experiments conducted by the Collaboratory for the Study of Earthquake Predictability (CSEP). Online Material: Figures and tables showing the RELM testing region and collection region definitions, numerical results associated with the RELM experiment, and the uncorrected forecast by Ebel et al. (2007) .

Description: We consider a seismicity forecast experiment conducted during the last 4 yr. At the beginning of each year, three models make a 1-yr forecast of the distribution of large earthquakes everywhere on the Earth. The forecasts are generated and the observations are collected in the Collaboratory for the Study of Earthquake Predictability (CSEP). We apply CSEP likelihood measures of consistency and comparison to see how well the forecasts match the observations, and we compare results from some intuitive reference models. These results illustrate some undesirable properties of the consistency tests: the tests can be extremely sensitive to only a few earthquakes, and yet insensitive to seemingly obvious flaws—a naïve hypothesis that large earthquakes are equally likely everywhere is not always rejected. The results also suggest that one should check the assumptions of the so-called T and W comparison tests, and we illustrate some methods to do so. As an extension of model assessment, we explore strategies to combine forecasts, and we discuss the implications for operational earthquake forecasting. Finally, we make suggestions for the next generation of global seismicity forecast experiments.

Description: The Regional Earthquake Likelihood Models experiment in California tested the performance of earthquake likelihood models over a five-year period. First-order analysis showed a smoothed-seismicity model by Helmstetter et al. (2007) to be the best model. We construct optimal multiplicative hybrids involving the best individual model as a baseline and one or more conjugate models. Conjugate models are transformed using an order-preserving function. Two parameters for each conjugate model and an overall normalizing constant are fitted to optimize the hybrid model. Many two-model hybrids have an appreciable information gain (log probability gain) per earthquake relative to the best individual model. For the whole of California, the Bird and Liu (2007) Neokinema and Holliday et al. (2007) pattern informatics (PI) models both give gains close to 0.25. For southern California, the Shen et al. (2007) geodetic model gives a gain of more than 0.5, and several others give gains of about 0.2. The best three-model hybrid for the whole region has the Neokinema and PI models as conjugates. The best three-model hybrid for southern California has the Shen et al. (2007) and PI models as conjugates. The information gains of the best multiplicative hybrids are greater than those of additive hybrids constructed from the same set of models. The gains tend to be larger when the contributing models involve markedly different concepts or data. These results need to be confirmed by further prospective tests. Multiplicative hybrids will be useful for assimilating other earthquake-related observations into forecasting models and for combining forecasting models at all timescales.