TY - JOUR

T1 - Mathematical test criteria for filtering complex systems

T2 - Plentiful observations

AU - Castronovo, E.

AU - Harlim, J.

AU - Majda, A. J.

N1 - Funding Information:
The research of Andrew J. Majda is partially supported by NSF Grant DMS-0456713, ONR Grant N0014-05-1-0164, and the Defense Advanced Research Projects Agency Grant N00014-07-1-0750, while Emilio Castronovo is supported as a postdoctoral fellow through the first two grants and John Harlim is supported as a postdoctoral fellow through the last two grants.

PY - 2008/3/20

Y1 - 2008/3/20

N2 - An important emerging scientific issue is the real time filtering through observations of noisy turbulent signals for complex systems as well as the statistical accuracy of spatio-temporal discretizations for such systems. These issues are addressed here in detail for the setting with plentiful observations for a scalar field through explicit mathematical test criteria utilizing a recent theory [A.J. Majda, M.J. Grote, Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems, Proceedings of the National Academy of Sciences 104 (4) (2007) 1124-1129]. For plentiful observations, the number of observations equals the number of mesh points. These test criteria involve much simpler decoupled complex scalar filtering test problems with explicit formulas and elementary numerical experiments which are developed here as guidelines for filter performance. The theory includes information criteria to avoid filter divergence with large model errors, asymptotic Kalman gain, filter stability, and accurate filtering with small ensemble size as well as rigorous results delineating the role of various turbulent spectra for filtering under mesh refinement. These guidelines are also applied to discrete approximations for filtering the stochastically forced dissipative advection equation with very turbulent and noisy signals with either an equipartition of energy or -5/3 turbulent spectrum with infrequent observations as severe test problems. The theory and companion simulations demonstrate accurate statistical filtering in this context with implicit schemes with large time step with very small ensemble sizes and even with unstable explicit schemes under appropriate circumstances provided the filtering strategies are guided by the off-line theoretical criteria. The surprising failure of other strongly stable filtering strategies is also explained through these off-line criteria.

AB - An important emerging scientific issue is the real time filtering through observations of noisy turbulent signals for complex systems as well as the statistical accuracy of spatio-temporal discretizations for such systems. These issues are addressed here in detail for the setting with plentiful observations for a scalar field through explicit mathematical test criteria utilizing a recent theory [A.J. Majda, M.J. Grote, Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems, Proceedings of the National Academy of Sciences 104 (4) (2007) 1124-1129]. For plentiful observations, the number of observations equals the number of mesh points. These test criteria involve much simpler decoupled complex scalar filtering test problems with explicit formulas and elementary numerical experiments which are developed here as guidelines for filter performance. The theory includes information criteria to avoid filter divergence with large model errors, asymptotic Kalman gain, filter stability, and accurate filtering with small ensemble size as well as rigorous results delineating the role of various turbulent spectra for filtering under mesh refinement. These guidelines are also applied to discrete approximations for filtering the stochastically forced dissipative advection equation with very turbulent and noisy signals with either an equipartition of energy or -5/3 turbulent spectrum with infrequent observations as severe test problems. The theory and companion simulations demonstrate accurate statistical filtering in this context with implicit schemes with large time step with very small ensemble sizes and even with unstable explicit schemes under appropriate circumstances provided the filtering strategies are guided by the off-line theoretical criteria. The surprising failure of other strongly stable filtering strategies is also explained through these off-line criteria.

UR - http://www.scopus.com/inward/record.url?scp=39649123724&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=39649123724&partnerID=8YFLogxK

U2 - 10.1016/j.jcp.2007.12.016

DO - 10.1016/j.jcp.2007.12.016

M3 - Article

AN - SCOPUS:39649123724

VL - 227

SP - 3678

EP - 3714

JO - Journal of Computational Physics

JF - Journal of Computational Physics

SN - 0021-9991

IS - 7

ER -