Total Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHC
The ATLAS Google Project was established as part of an ongoing evaluation of the use of commercial clouds by the ATLAS Collaboration, in anticipation of the potential future adoption of such resources by WLCG grid sites to fulfil or complement their computing pledges. Seamless integration of Google...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Springer
2025
|
| _version_ | 1824459295907905536 |
|---|---|
| author | Aad, G Aakvaag, E Abbott, B Abdelhameed, S Abeling, K Abicht, NJ Abidi, SH Aboelela, M Aboulhorma, A Abramowicz, H Abreu, H Abulaiti, Y Acharya, BS Ackermann, A Bourdarios, CA Adamczyk, L Addepalli, SV Addison, MJ Adelman, J Adiguzel, A Adye, T Affolder, AA Afik, Y Agaras, MN Barr, AJ Bortoletto, D Bruckler, TL Celli, F Chen, M Conroy, EI Cooper-Sarkar, AM Dingley, T Draguet, MA Eberwein, GH Frost, JA Gallas, EJ Grabarczyk, RP Grundy, JC Gwenlan, C Harris, YT Haslbeck, F Hays, CP Hodkinson, BH Huffman, TB Iizawa, T Karava, K Koch, SF Li, Z Moser, B Nagai, K Nedic, L Nickerson, RB Pacey, HA Rossi, E Ruggiero, A Schopf, E Shipsey, IPJ Smith, HA Sultan, DMS Viehhauser, GHA Weidberg, AR Yildirim, TP |
| author_facet | Aad, G Aakvaag, E Abbott, B Abdelhameed, S Abeling, K Abicht, NJ Abidi, SH Aboelela, M Aboulhorma, A Abramowicz, H Abreu, H Abulaiti, Y Acharya, BS Ackermann, A Bourdarios, CA Adamczyk, L Addepalli, SV Addison, MJ Adelman, J Adiguzel, A Adye, T Affolder, AA Afik, Y Agaras, MN Barr, AJ Bortoletto, D Bruckler, TL Celli, F Chen, M Conroy, EI Cooper-Sarkar, AM Dingley, T Draguet, MA Eberwein, GH Frost, JA Gallas, EJ Grabarczyk, RP Grundy, JC Gwenlan, C Harris, YT Haslbeck, F Hays, CP Hodkinson, BH Huffman, TB Iizawa, T Karava, K Koch, SF Li, Z Moser, B Nagai, K Nedic, L Nickerson, RB Pacey, HA Rossi, E Ruggiero, A Schopf, E Shipsey, IPJ Smith, HA Sultan, DMS Viehhauser, GHA Weidberg, AR Yildirim, TP |
| author_sort | Aad, G |
| collection | OXFORD |
| description | The ATLAS Google Project was established as part of an ongoing evaluation of the use of commercial clouds by the ATLAS Collaboration, in anticipation of the potential future adoption of such resources by WLCG grid sites to fulfil or complement their computing pledges. Seamless integration of Google cloud resources into the worldwide ATLAS distributed computing infrastructure was achieved at large scale and for an extended period of time, and hence cloud resources are shown to be an effective mechanism to provide additional, flexible computing capacity to ATLAS. For the first time a total cost of ownership analysis has been performed, to identify the dominant cost drivers and explore effective mechanisms for cost control. Network usage significantly impacts the costs of certain ATLAS workflows, underscoring the importance of implementing such mechanisms. Resource bursting has been successfully demonstrated, whilst exposing the true cost of this type of activity. A follow-up to the project is underway to investigate methods for improving the integration of cloud resources in data-intensive distributed computing environments and reducing costs related to network connectivity, which represents the primary expense when extensively utilising cloud resources. |
| first_indexed | 2025-02-19T04:39:31Z |
| format | Journal article |
| id | oxford-uuid:67820e8b-589e-4c03-8b88-6dcff68a2d5e |
| institution | University of Oxford |
| language | English |
| last_indexed | 2025-02-19T04:39:31Z |
| publishDate | 2025 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:67820e8b-589e-4c03-8b88-6dcff68a2d5e2025-02-15T20:12:35ZTotal Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHCJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:67820e8b-589e-4c03-8b88-6dcff68a2d5eEnglishJisc Publications RouterSpringer2025Aad, GAakvaag, EAbbott, BAbdelhameed, SAbeling, KAbicht, NJAbidi, SHAboelela, MAboulhorma, AAbramowicz, HAbreu, HAbulaiti, YAcharya, BSAckermann, ABourdarios, CAAdamczyk, LAddepalli, SVAddison, MJAdelman, JAdiguzel, AAdye, TAffolder, AAAfik, YAgaras, MNBarr, AJBortoletto, DBruckler, TLCelli, FChen, MConroy, EICooper-Sarkar, AMDingley, TDraguet, MAEberwein, GHFrost, JAGallas, EJGrabarczyk, RPGrundy, JCGwenlan, CHarris, YTHaslbeck, FHays, CPHodkinson, BHHuffman, TBIizawa, TKarava, KKoch, SFLi, ZMoser, BNagai, KNedic, LNickerson, RBPacey, HARossi, ERuggiero, ASchopf, EShipsey, IPJSmith, HASultan, DMSViehhauser, GHAWeidberg, ARYildirim, TPThe ATLAS Google Project was established as part of an ongoing evaluation of the use of commercial clouds by the ATLAS Collaboration, in anticipation of the potential future adoption of such resources by WLCG grid sites to fulfil or complement their computing pledges. Seamless integration of Google cloud resources into the worldwide ATLAS distributed computing infrastructure was achieved at large scale and for an extended period of time, and hence cloud resources are shown to be an effective mechanism to provide additional, flexible computing capacity to ATLAS. For the first time a total cost of ownership analysis has been performed, to identify the dominant cost drivers and explore effective mechanisms for cost control. Network usage significantly impacts the costs of certain ATLAS workflows, underscoring the importance of implementing such mechanisms. Resource bursting has been successfully demonstrated, whilst exposing the true cost of this type of activity. A follow-up to the project is underway to investigate methods for improving the integration of cloud resources in data-intensive distributed computing environments and reducing costs related to network connectivity, which represents the primary expense when extensively utilising cloud resources. |
| spellingShingle | Aad, G Aakvaag, E Abbott, B Abdelhameed, S Abeling, K Abicht, NJ Abidi, SH Aboelela, M Aboulhorma, A Abramowicz, H Abreu, H Abulaiti, Y Acharya, BS Ackermann, A Bourdarios, CA Adamczyk, L Addepalli, SV Addison, MJ Adelman, J Adiguzel, A Adye, T Affolder, AA Afik, Y Agaras, MN Barr, AJ Bortoletto, D Bruckler, TL Celli, F Chen, M Conroy, EI Cooper-Sarkar, AM Dingley, T Draguet, MA Eberwein, GH Frost, JA Gallas, EJ Grabarczyk, RP Grundy, JC Gwenlan, C Harris, YT Haslbeck, F Hays, CP Hodkinson, BH Huffman, TB Iizawa, T Karava, K Koch, SF Li, Z Moser, B Nagai, K Nedic, L Nickerson, RB Pacey, HA Rossi, E Ruggiero, A Schopf, E Shipsey, IPJ Smith, HA Sultan, DMS Viehhauser, GHA Weidberg, AR Yildirim, TP Total Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHC |
| title | Total Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHC |
| title_full | Total Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHC |
| title_fullStr | Total Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHC |
| title_full_unstemmed | Total Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHC |
| title_short | Total Cost of Ownership and Evaluation of Google Cloud Resources for the ATLAS Experiment at the LHC |
| title_sort | total cost of ownership and evaluation of google cloud resources for the atlas experiment at the lhc |
| work_keys_str_mv | AT aadg totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT aakvaage totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abbottb totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abdelhameeds totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abelingk totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abichtnj totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abidish totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT aboelelam totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT aboulhormaa totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abramowiczh totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abreuh totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT abulaitiy totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT acharyabs totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT ackermanna totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT bourdariosca totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT adamczykl totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT addepallisv totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT addisonmj totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT adelmanj totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT adiguzela totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT adyet totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT affolderaa totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT afiky totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT agarasmn totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT barraj totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT bortolettod totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT brucklertl totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT cellif totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT chenm totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT conroyei totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT coopersarkaram totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT dingleyt totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT draguetma totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT eberweingh totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT frostja totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT gallasej totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT grabarczykrp totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT grundyjc totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT gwenlanc totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT harrisyt totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT haslbeckf totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT hayscp totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT hodkinsonbh totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT huffmantb totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT iizawat totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT karavak totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT kochsf totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT liz totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT moserb totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT nagaik totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT nedicl totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT nickersonrb totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT paceyha totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT rossie totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT ruggieroa totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT schopfe totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT shipseyipj totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT smithha totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT sultandms totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT viehhausergha totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT weidbergar totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc AT yildirimtp totalcostofownershipandevaluationofgooglecloudresourcesfortheatlasexperimentatthelhc |