ALBERT

Beschreibung: This paper compares several commonly used state-of-the-art ensemble-based data assimilation methods in a coherent mathematical notation. The study encompasses different methods that are applicable to high-dimensional geophysical systems, like ocean and atmosphere and provide an uncertainty estimate. Most variants of Ensemble Kalman Filters, Particle Filters and second-order exact methods are discussed, including Gaussian Mixture Filters, while methods that require an adjoint model or a tangent linear formulation of the model are excluded. The detailed description of all the methods in a mathematically coherent way provides both novices and experienced researchers with a unique overview and new insight in the workings and relative advantages of each method, theoretically and algorithmically, even leading to new filters. Furthermore, the practical implementation details of all ensemble and particle filter methods are discussed to show similarities and differences in the filters aiding the users in what to use when. Finally, pseudo-codes are provided for all of the methods presented in this paper.

Beschreibung: Different strategies for implementing ensemble-based data assimilation systems are discussed. Ensemble filters like ensemble Kalman filters and particle filters can be implemented so that they are nearly independent from the model into which they assimilate observations. This allows to develop implementations that clearly separate the data assimilation algorithm from the numerical model. For coupling the model with a data assimilation software one possibility is to use disk files to exchange the model state information between model and ensemble data assimilation methods. This offline coupling does not require changes in the model code, except for a possible component to simulate model error during the ensemble integration. However, using disk files can be inefficient, in particular when the time for the model integrations is not significantly larger than the time to restart the model for each ensemble member and to read and write the ensemble state information with the data assimilation program. In contrast, an online coupling strategy can be computational much more efficient. In this coupling strategy, subroutine calls for the data assimilation are directly inserted into the source code of an existing numerical model and augment the numerical model to become a data assimilative model. This strategy avoids model restarts as well as excessive writing of ensemble information into disk files. To allow for ensemble integrations, one of the subroutines modifies the parallelization of the model or adds one, if a model is not already parallelized. Then, the data assimilation can be performed efficiently using parallel computers. As the required modifications to the model code are very limited, this strategy allows one to quickly extent a model to a data assimilation system. In particular, the numerics of a model do not need to be changed and the model itself does not need to be a subroutine. The online coupling shows an excellent computational scalability on supercomputers and is well suited for high-dimensional numerical models. Further, a clear separation of the model and data assimilation components allows to continue the development of both components separately. Thus, new data assimilation methods can be easily added to the data assimilation system. Using the example of the parallel data assimilation framework [PDAF, http://pdaf.awi.de] and the ocean model NEMO, it is demonstrated how the online coupling can be achieved with minimal changes to the numerical model.

Beschreibung: This work assesses the large-scale applicability of the recently proposed nonlinear ensemble transform filter (NETF) in data assimilation experiments with the NEMO ocean general circulation model. The new filter constitutes a second-order exact approximation to fully nonlinear particle filtering. Thus, it relaxes the Gaussian assumption contained in ensemble Kalman filters. The NETF applies an update step similar to the local ensemble transform Kalman filter (LETKF), which allows for efficient and simple implementation. Here, simulated observations are assimilated into a simplified ocean configuration that exhibits globally highdimensional dynamics with a chaotic mesoscale flow. The model climatology is used to initialize an ensemble of 120 members. The number of observations in each local filter update is of the same order resulting from the use of a realistic oceanic observation scenario. Here, an importance sampling particle filter (PF) would require at least 106 members. Despite the relatively small ensemble size, the NETF remains stable and converges to the truth. In this setup, the NETF achieves at least the performance of the LETKF. However, it requires a longer spinup period because the algorithm only relies on the particle weights at the analysis time. These findings show that the NETF can successfully deal with a large-scale assimilation problem in which the local observation dimension is of the same order as the ensemble size. Thus, the second-order exact NETF does not suffer from the PF’s curse of dimensionality, even in a deterministic system.

Beschreibung: The EU-funded project SANGOMA - Stochastic Assimilation of the Next Generation Ocean Model Applications provides new developments in data assimilation to ensure that future operational systems can make use of state-of-the-art data-assimilation methods and related analysis tools. One task of SANGOMA is to develop a collection of common tools for data assimilation with a uniform interface so that the tools are usable from different data assimilation systems. The tool developments mainly aim at tools that support ensemble-based data assimilation applications like for the generation of perturbations, to perform transformations, to compute diagnostics, as well as further utilities. The tools are implemented in Fortran and as scripts for Matlab or Octave. The tools and their documentation are provided as free open-source programs via the project web site [http://www.data-assimilation.net]. This contribution provides an overview of the tools that are available in the latest release V1 of the SANGOMA tools as well as the plans for the next release.

Beschreibung: The NEMO model is a state-of-the-art ocean circulation model. For data assimilation applications with ensemble Kalman filters like the SEEK filter, e.g. for operational ocean forecasting, NEMO is typically run separately from the assimilation algorithm. Thus, NEMO is used to generate a set of restart files on disks that hold the ensemble of model forecasts providing the error covariance matrix information for the ensemble Kalman filter. These files need to be read by a separate assimilation program that computes the analysis step of the filter algorithm and generates new restart files for NEMO. This scheme requires a large amount of disk storage as well as time to read and write restart files and to perform the model restarts. To improve the efficiency, a data assimilation system for NEMO is introduced that is built using the parallel data assimilation framework PDAF [http://pdaf.awi.de]. Inserting a few subroutine calls to the source code of NEMO, the model is extended to a data assimilation system that consists of a single program. Utilizing the parallelization capacity of today's supercomputers, the system performs both the ensemble forecasts and the analysis step of the filter algorithm in a single execution of the program. The features of the resulting assimilation system are discussed as well as the parallel performance of the assimilation program when it is applied with a benchmark configuration of NEMO from the SANGOMA project [http://www.data-assimilation.net].

Beschreibung: In data assimilation applications using ensemble Kalman filter methods, localization is necessary to make the method work with high-dimensional geophysical models. For ensemble square-root Kalman filters, domain localization (DL) and observation localization (OL) are commonly used. Depending on the localization method, one has to choose appropriate values for the localization parameters, such as the localization length and the weight function. Although frequently used, the properties of the localization techniques are not fully investigated. Thus, up to now an optimal choice for these parameters is a priori unknown and they are generally found by expensive numerical experiments. In this study, the relationship between the localization length and the ensemble size in DL and OL is studied using twin experiments with the Lorenz-96 model and a 2-dimensional shallow water model. For both models, it is found that the optimal localization length for DL and OL depends linearly on an effective local observation dimension that is given by the sum of the observation weights. In the experiments no influence of the model dynamics on the optimal localization length was observed. The effective observation dimension defines the degrees of freedom that are required for assimilating observations, while the ensemble size defines the available degrees of freedom. Setting the localization radius such that the effective local observation dimension equals the ensemble size yields an adaptive localization radius. Its performance is tested using a global ocean model. The experiments show that the analysis quality using the adaptive localization is similar to the analysis quality of an optimally tuned constant localization radius.

Beschreibung: The recently-proposed nonlinear ensemble transform filter (NETF) is extended to a fixed lag smoother. The NETF approximates Bayes' theorem by applying a square root update. The smoother (NETS) is derived and formulated in a joint framework with the filter. The new smoother method is evaluated using the low-dimensional, highly nonlinear Lorenz-96 model and a square-box configuration of the NEMO ocean model, which is nonlinear and has a higher dimensionality. The new smoother is evaluated within the same assimilation framework against the local error subspace transform Kalman filter (LESTKF) and its smoother extension (LESTKS), which are state of the art ensemble square-root Kalman techniques. In the case of the Lorenz-96 model, both the filter NETF and its smoother extension NETS provide lower errors than the LESTKF and LESTKS for sufficiently large ensembles. In addition, the NETS shows a distinct dependence on the smoother lag, which results in a stronger error increase beyond the optimal lag of minimum error. For the experiment using NEMO, the smoothing in the NETS effectively reduces the errors in the state estimates, compared to the filter. For different state variables very similar optimal smoothing lags are found, which allows for a simultaneous tuning of the lag. In comparison to the LESTKS, the smoothing with the NETS yields a smaller relative error reduction with respect to the filter result, and the optimal lag of the NETS is shorter in both experiments. This is explained by the distinct update mechanisms of both filters. The comparison of both experiments shows that the NETS can provide better state estimates with similar smoother lags if the model exhibits a sufficiently high degree of nonlinearity or if the observations are not restricted to be Gaussian with a linear observation operator.

Beschreibung: Different strategies for implementing ensemble-based data assimilation systems are discussed. Ensemble filters like ensemble Kalman filters and particle filters can be implemented so that they are nearly independent from the model into which they assimilate observations. In particular, an online coupling strategy is computational efficient. In this coupling strategy, subroutine calls for the data assimilation are directly inserted into the source code of an existing numerical model and augment the numerical model to become a data assimilative model. The online coupling shows an excellent computational scalability on supercomputers and is hence well suited for high-dimensional numerical models, including coupled earth system models. Further a clear separation of the model and data assimilation components allows to continue the development of both components separately. Using the example of the parallel data assimilation framework (PDAF, http://pdaf.awi.de) and the ocean model NEMO, it is demonstrated how the online coupling can be achieved with minimal changes to the numerical model.

Beschreibung: In data assimilation using ensemble Kalman filter methods, localization is an important technique to get good assimilation results. For the LETKF, the domain localization (DL) and observation localization (OL) are typically used. Depending on the localization method, one has to choose appropriate values for the localization parameters, such as the localization length, the inflation factor or the weight function. Although being frequently used, the properties of the localization techniques are not fully investigated. Thus, up to now an optimal choice for these parameters is a priori unknown and they are generally found by doing expensive numerical experiments. The relationship between the localization length and the ensemble size in DL and OL is studied using twin experiments with the Lorenz-96 model. It is found that for DL the optimal localization length depends linearly on the local observation dimension. This also holds for the localization length at which the filter diverges. A similar behavior was observed for OL by considering an effective local observation dimension.