CernVM-FS powered container hub
Containers became the de-facto standard to package and distribute modern applications and their dependencies. The HEP community demonstrates an increasing interest in such technology, with scientists encapsulating their analysis workflow and code inside a container image. The analysis is first valid...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2021-01-01
|
| Series: | EPJ Web of Conferences |
| Online Access: | https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02033.pdf |
| _version_ | 1819069014174859264 |
|---|---|
| author | Bocchi Enrico Blomer Jakob Mosciatti Simone Valenzuela Andrea |
| author_facet | Bocchi Enrico Blomer Jakob Mosciatti Simone Valenzuela Andrea |
| author_sort | Bocchi Enrico |
| collection | DOAJ |
| description | Containers became the de-facto standard to package and distribute modern applications and their dependencies. The HEP community demonstrates an increasing interest in such technology, with scientists encapsulating their analysis workflow and code inside a container image. The analysis is first validated on a small dataset and minimal hardware resources to then run at scale on the massive computing capacity provided by the grid. The typical approach for distributing containers consists of pulling their image from a remote registry and extracting it on the node where the container runtime (e.g., Docker, Singularity) runs. This approach, however, does not easily scale to large images and thousands of nodes. CVMFS has long been used for the efficient distribution of software directory trees at a global scale. In order to extend its optimized caching and network utilization to the distribution of containers, CVMFS recently implemented a dedicated container image ingestion service together with container runtime integrations. CVMFS ingestion is based on per-file deduplication, instead of the per-layer deduplication adopted by traditional container registries. On the client-side, CVMFS implements on-demand fetching of the chunks required for the execution of the container instead of the whole image. |
| first_indexed | 2024-12-21T16:43:18Z |
| format | Article |
| id | doaj.art-5aff2a2c72d74d5182c4358c5d1a9876 |
| institution | Directory Open Access Journal |
| issn | 2100-014X |
| language | English |
| last_indexed | 2024-12-21T16:43:18Z |
| publishDate | 2021-01-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | EPJ Web of Conferences |
| spelling | doaj.art-5aff2a2c72d74d5182c4358c5d1a98762022-12-21T18:57:03ZengEDP SciencesEPJ Web of Conferences2100-014X2021-01-012510203310.1051/epjconf/202125102033epjconf_chep2021_02033CernVM-FS powered container hubBocchi Enrico0Blomer Jakob1Mosciatti Simone2Valenzuela Andrea3CERNCERNCERNCERNContainers became the de-facto standard to package and distribute modern applications and their dependencies. The HEP community demonstrates an increasing interest in such technology, with scientists encapsulating their analysis workflow and code inside a container image. The analysis is first validated on a small dataset and minimal hardware resources to then run at scale on the massive computing capacity provided by the grid. The typical approach for distributing containers consists of pulling their image from a remote registry and extracting it on the node where the container runtime (e.g., Docker, Singularity) runs. This approach, however, does not easily scale to large images and thousands of nodes. CVMFS has long been used for the efficient distribution of software directory trees at a global scale. In order to extend its optimized caching and network utilization to the distribution of containers, CVMFS recently implemented a dedicated container image ingestion service together with container runtime integrations. CVMFS ingestion is based on per-file deduplication, instead of the per-layer deduplication adopted by traditional container registries. On the client-side, CVMFS implements on-demand fetching of the chunks required for the execution of the container instead of the whole image.https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02033.pdf |
| spellingShingle | Bocchi Enrico Blomer Jakob Mosciatti Simone Valenzuela Andrea CernVM-FS powered container hub EPJ Web of Conferences |
| title | CernVM-FS powered container hub |
| title_full | CernVM-FS powered container hub |
| title_fullStr | CernVM-FS powered container hub |
| title_full_unstemmed | CernVM-FS powered container hub |
| title_short | CernVM-FS powered container hub |
| title_sort | cernvm fs powered container hub |
| url | https://www.epj-conferences.org/articles/epjconf/pdf/2021/05/epjconf_chep2021_02033.pdf |
| work_keys_str_mv | AT bocchienrico cernvmfspoweredcontainerhub AT blomerjakob cernvmfspoweredcontainerhub AT mosciattisimone cernvmfspoweredcontainerhub AT valenzuelaandrea cernvmfspoweredcontainerhub |