An ensemble multiscale filter for large nonlinear data assimilation problems

Operational data assimilation problems tend to be very large, both in terms of the number of unknowns to be estimated and the number of measurements to be processed. This poses significant computational challenges, especially for ensemble methods, which are critically dependent on the number of replicates used to derive sample covariances and other statistics. Most efforts to deal with the related problems of computational effort and sampling error in ensemble estimation have focused on spatial localization. The ensemble multiscale Kalman filter described here offers an alternative approach that effectively replaces, at each update time, the prior (or background) sample covariance with a multiscale tree. The tree is composed of nodes distributed over a relatively small number of discrete scales. Global correlations between variables at different locations are described in terms of local relationships between nodes at adjacent scales (parents and children). The Kalman updating process can be carried out very efficiently on such a tree, especially if the update calculations exploit the tree's parallel structure. In fact, the resulting savings in effort far exceeds the additional work required to construct the tree. The tree-identification process offers possibilities for introducing localization in scale, which can be used instead of or in addition to localization in space. The multiscale filter is able to continually adapt to changing problem scales through associated changes in the tree structure. This is illustrated with a large (10⁶) unknown turbulent fluid flow example that generates dynamic features that span a wide range of time and space scales. This filter is able to track changing features over long distances without any spatial localization, using a moderate ensemble size of 54. The computational savings provided by the multiscale approach, combined with opportunities for hybrid localization over both space and scale, offer significant practical benefits for large data assimilation applications.