Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean
Uncertainty prediction for ocean and climate predictions is essential for multiple applications today. Many-Task Computing can play a significant role in making such predictions feasible. In this manuscript, we focus on ocean uncertainty prediction using the Error Subspace Statistical Estimation (ES...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2018
|
| Online Access: | http://hdl.handle.net/1721.1/119827 https://orcid.org/0000-0002-1869-3883 https://orcid.org/0000-0003-3417-9056 |
| _version_ | 1826207644957802496 |
|---|---|
| author | Evangelinos, Constantinos Lermusiaux, Pierre Xu, Jinshan Haley, Patrick Hill, Christopher N. |
| author2 | Massachusetts Institute of Technology. Department of Earth, Atmospheric, and Planetary Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Earth, Atmospheric, and Planetary Sciences Evangelinos, Constantinos Lermusiaux, Pierre Xu, Jinshan Haley, Patrick Hill, Christopher N. |
| author_sort | Evangelinos, Constantinos |
| collection | MIT |
| description | Uncertainty prediction for ocean and climate predictions is essential for multiple applications today. Many-Task Computing can play a significant role in making such predictions feasible. In this manuscript, we focus on ocean uncertainty prediction using the Error Subspace Statistical Estimation (ESSE) approach. In ESSE, uncertainties are represented by an error subspace of variable size. To predict these uncertainties, we perturb an initial state based on the initial error subspace and integrate the corresponding ensemble of initial conditions forward in time, including stochastic forcing during each simulation. The dominant error covariance (generated via SVD of the ensemble) is used for data assimilation. The resulting ocean fields are used as inputs for predictions of underwater sound propagation. ESSE is a classic case of Many Task Computing: It uses dynamic heterogeneous workflows and ESSE ensembles are data intensive applications. We first study the execution characteristics of a distributed ESSE workflow on a medium size dedicated cluster, examine in more detail the I/O patterns exhibited and throughputs achieved by its components as well as the overall ensemble performance seen in practice. We then study the performance/usability challenges of employing Amazon EC2 and the Teragrid to augment our ESSE ensembles and provide better solutions faster. Keywords: MTC; assimilation; data-intensive; ensemble |
| first_indexed | 2024-09-23T13:52:44Z |
| format | Article |
| id | mit-1721.1/119827 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T13:52:44Z |
| publishDate | 2018 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1198272024-05-15T09:01:12Z Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean Evangelinos, Constantinos Lermusiaux, Pierre Xu, Jinshan Haley, Patrick Hill, Christopher N. Massachusetts Institute of Technology. Department of Earth, Atmospheric, and Planetary Sciences Massachusetts Institute of Technology. Department of Mechanical Engineering Evangelinos, Constantinos Lermusiaux, Pierre Xu, Jinshan Haley, Patrick Hill, Christopher N Uncertainty prediction for ocean and climate predictions is essential for multiple applications today. Many-Task Computing can play a significant role in making such predictions feasible. In this manuscript, we focus on ocean uncertainty prediction using the Error Subspace Statistical Estimation (ESSE) approach. In ESSE, uncertainties are represented by an error subspace of variable size. To predict these uncertainties, we perturb an initial state based on the initial error subspace and integrate the corresponding ensemble of initial conditions forward in time, including stochastic forcing during each simulation. The dominant error covariance (generated via SVD of the ensemble) is used for data assimilation. The resulting ocean fields are used as inputs for predictions of underwater sound propagation. ESSE is a classic case of Many Task Computing: It uses dynamic heterogeneous workflows and ESSE ensembles are data intensive applications. We first study the execution characteristics of a distributed ESSE workflow on a medium size dedicated cluster, examine in more detail the I/O patterns exhibited and throughputs achieved by its components as well as the overall ensemble performance seen in practice. We then study the performance/usability challenges of employing Amazon EC2 and the Teragrid to augment our ESSE ensembles and provide better solutions faster. Keywords: MTC; assimilation; data-intensive; ensemble United States. Office of Naval Research (Grant N00014-08-1-1097) United States. Office of Naval Research (Grant N00014-07-1-0501) United States. Office of Naval Research (Grant N00014-08-1-0586) 2018-12-21T20:02:50Z 2018-12-21T20:02:50Z 2011-02 2018-12-12T15:58:35Z Article http://purl.org/eprint/type/JournalArticle 1045-9219 http://hdl.handle.net/1721.1/119827 Evangelinos, C et al. “Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean.” IEEE Transactions on Parallel and Distributed Systems 22, 6 (June 2011): 1012–1024 © 2011 IEEE https://orcid.org/0000-0002-1869-3883 https://orcid.org/0000-0003-3417-9056 http://dx.doi.org/10.1109/TPDS.2011.64 IEEE Transactions on Parallel and Distributed Systems Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) Other repository |
| spellingShingle | Evangelinos, Constantinos Lermusiaux, Pierre Xu, Jinshan Haley, Patrick Hill, Christopher N. Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean |
| title | Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean |
| title_full | Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean |
| title_fullStr | Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean |
| title_full_unstemmed | Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean |
| title_short | Many Task Computing for Real-Time Uncertainty Prediction and Data Assimilation in the Ocean |
| title_sort | many task computing for real time uncertainty prediction and data assimilation in the ocean |
| url | http://hdl.handle.net/1721.1/119827 https://orcid.org/0000-0002-1869-3883 https://orcid.org/0000-0003-3417-9056 |
| work_keys_str_mv | AT evangelinosconstantinos manytaskcomputingforrealtimeuncertaintypredictionanddataassimilationintheocean AT lermusiauxpierre manytaskcomputingforrealtimeuncertaintypredictionanddataassimilationintheocean AT xujinshan manytaskcomputingforrealtimeuncertaintypredictionanddataassimilationintheocean AT haleypatrick manytaskcomputingforrealtimeuncertaintypredictionanddataassimilationintheocean AT hillchristophern manytaskcomputingforrealtimeuncertaintypredictionanddataassimilationintheocean |