Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD
The High Luminosity Large Hadron Collider provides a data challenge. The amount of data recorded from the experiments and transported to hundreds of sites will see a thirty fold increase in annual data volume. A systematic approach to contrast the performance of different Third Party Copy (TPC) tran...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2021-01-01
|
| Series: | EPJ Web of Conferences |
| Online Access: | https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02001.pdf |
| _version_ | 1818717077686452224 |
|---|---|
| author | Fajardo Edgar Arora Aashay Davila Diego Gao Richard Würthwein Frank Bockelman Brian |
| author_facet | Fajardo Edgar Arora Aashay Davila Diego Gao Richard Würthwein Frank Bockelman Brian |
| author_sort | Fajardo Edgar |
| collection | DOAJ |
| description | The High Luminosity Large Hadron Collider provides a data challenge. The amount of data recorded from the experiments and transported to hundreds of sites will see a thirty fold increase in annual data volume. A systematic approach to contrast the performance of different Third Party Copy (TPC) transfer protocols arises. Two contenders, XRootD-HTTPS and the GridFTP are evaluated in their performance for transferring files from one server to another over 100Gbps interfaces. The benchmarking is done by scheduling pods on the Pacific Research Platform Kubernetes cluster to ensure reproducible and repeatable results. This opens a future pathway for network testing of any TPC transfer protocol. |
| first_indexed | 2024-12-17T19:29:25Z |
| format | Article |
| id | doaj.art-95f9941768b14d319b41193bfbfa40cc |
| institution | Directory Open Access Journal |
| issn | 2100-014X |
| language | English |
| last_indexed | 2024-12-17T19:29:25Z |
| publishDate | 2021-01-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | EPJ Web of Conferences |
| spelling | doaj.art-95f9941768b14d319b41193bfbfa40cc2022-12-21T21:35:19ZengEDP SciencesEPJ Web of Conferences2100-014X2021-01-012510200110.1051/epjconf/202125102001epjconf_chep2021_02001Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootDFajardo Edgar0Arora Aashay1Davila Diego2Gao Richard3Würthwein Frank4Bockelman Brian5University of California San DiegoUniversity of California San DiegoUniversity of California San DiegoUniversity of California San DiegoUniversity of California San DiegoMorgridge InstituteThe High Luminosity Large Hadron Collider provides a data challenge. The amount of data recorded from the experiments and transported to hundreds of sites will see a thirty fold increase in annual data volume. A systematic approach to contrast the performance of different Third Party Copy (TPC) transfer protocols arises. Two contenders, XRootD-HTTPS and the GridFTP are evaluated in their performance for transferring files from one server to another over 100Gbps interfaces. The benchmarking is done by scheduling pods on the Pacific Research Platform Kubernetes cluster to ensure reproducible and repeatable results. This opens a future pathway for network testing of any TPC transfer protocol.https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02001.pdf |
| spellingShingle | Fajardo Edgar Arora Aashay Davila Diego Gao Richard Würthwein Frank Bockelman Brian Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD EPJ Web of Conferences |
| title | Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD |
| title_full | Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD |
| title_fullStr | Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD |
| title_full_unstemmed | Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD |
| title_short | Systematic benchmarking of HTTPS third party copy on 100Gbps links using XRootD |
| title_sort | systematic benchmarking of https third party copy on 100gbps links using xrootd |
| url | https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02001.pdf |
| work_keys_str_mv | AT fajardoedgar systematicbenchmarkingofhttpsthirdpartycopyon100gbpslinksusingxrootd AT aroraaashay systematicbenchmarkingofhttpsthirdpartycopyon100gbpslinksusingxrootd AT daviladiego systematicbenchmarkingofhttpsthirdpartycopyon100gbpslinksusingxrootd AT gaorichard systematicbenchmarkingofhttpsthirdpartycopyon100gbpslinksusingxrootd AT wurthweinfrank systematicbenchmarkingofhttpsthirdpartycopyon100gbpslinksusingxrootd AT bockelmanbrian systematicbenchmarkingofhttpsthirdpartycopyon100gbpslinksusingxrootd |