Data access and analysis with distributed federated data servers in climateprediction.net
climateprediction.net is a large public resource distributed scientific computing project. Members of the public download and run a full-scale climate model, donate their computing time to a large perturbed physics ensemble experiment to forecast the climate in the 21st century and submit their resu...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
2006
|
| _version_ | 1797064446557814784 |
|---|---|
| author | Massey, N Aina, T Allen, M Christensen, C Frame, D Goodman, D Kettleborough, J Martin, A Pascoe, S Stainforth, D |
| author_facet | Massey, N Aina, T Allen, M Christensen, C Frame, D Goodman, D Kettleborough, J Martin, A Pascoe, S Stainforth, D |
| author_sort | Massey, N |
| collection | OXFORD |
| description | climateprediction.net is a large public resource distributed scientific computing project. Members of the public download and run a full-scale climate model, donate their computing time to a large perturbed physics ensemble experiment to forecast the climate in the 21st century and submit their results back to the project. The amount of data generated is large, consisting of tens of thousands of individual runs each in the order of tens of megabytes. The overall dataset is, therefore, in the order of terabytes. Access and analysis of the data is further complicated by the reliance on donated, distributed, federated data servers. This paper will discuss the problems encountered when the data required for even a simple analysis is spread across several servers and how webservice technology can be used; how different user interfaces with varying levels of complexity and flexibility can be presented to the application scientists, how using existing web technologies such as HTTP, SOAP, XML, HTML and CGI can engender the reuse of code across interfaces; and how application scientists can be notified of their analysis' progress and results in an asynchronous architecture. |
| first_indexed | 2024-03-06T21:14:25Z |
| format | Journal article |
| id | oxford-uuid:3f46d1fc-f6d0-4390-9dda-816536dd7d57 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T21:14:25Z |
| publishDate | 2006 |
| record_format | dspace |
| spelling | oxford-uuid:3f46d1fc-f6d0-4390-9dda-816536dd7d572022-03-26T14:31:05ZData access and analysis with distributed federated data servers in climateprediction.netJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:3f46d1fc-f6d0-4390-9dda-816536dd7d57EnglishSymplectic Elements at Oxford2006Massey, NAina, TAllen, MChristensen, CFrame, DGoodman, DKettleborough, JMartin, APascoe, SStainforth, Dclimateprediction.net is a large public resource distributed scientific computing project. Members of the public download and run a full-scale climate model, donate their computing time to a large perturbed physics ensemble experiment to forecast the climate in the 21st century and submit their results back to the project. The amount of data generated is large, consisting of tens of thousands of individual runs each in the order of tens of megabytes. The overall dataset is, therefore, in the order of terabytes. Access and analysis of the data is further complicated by the reliance on donated, distributed, federated data servers. This paper will discuss the problems encountered when the data required for even a simple analysis is spread across several servers and how webservice technology can be used; how different user interfaces with varying levels of complexity and flexibility can be presented to the application scientists, how using existing web technologies such as HTTP, SOAP, XML, HTML and CGI can engender the reuse of code across interfaces; and how application scientists can be notified of their analysis' progress and results in an asynchronous architecture. |
| spellingShingle | Massey, N Aina, T Allen, M Christensen, C Frame, D Goodman, D Kettleborough, J Martin, A Pascoe, S Stainforth, D Data access and analysis with distributed federated data servers in climateprediction.net |
| title | Data access and analysis with distributed federated data servers in climateprediction.net |
| title_full | Data access and analysis with distributed federated data servers in climateprediction.net |
| title_fullStr | Data access and analysis with distributed federated data servers in climateprediction.net |
| title_full_unstemmed | Data access and analysis with distributed federated data servers in climateprediction.net |
| title_short | Data access and analysis with distributed federated data servers in climateprediction.net |
| title_sort | data access and analysis with distributed federated data servers in climateprediction net |
| work_keys_str_mv | AT masseyn dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT ainat dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT allenm dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT christensenc dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT framed dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT goodmand dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT kettleboroughj dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT martina dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT pascoes dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet AT stainforthd dataaccessandanalysiswithdistributedfederateddataserversinclimatepredictionnet |