LHCb Continuous Integration and Deployment system: a message based approach
A continuous integration system is crucial to maintain the quality of the 6 millions lines of C++ and Python source code of the LHCb software in order to ensure consistent builds of the software as well as to run the unit and integration tests. Jenkins automation server is used for this purpose. It...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2019-01-01
|
| Series: | EPJ Web of Conferences |
| Online Access: | https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_05001.pdf |
| _version_ | 1818622059079532544 |
|---|---|
| author | Chitic Stefan-Gabriel Couturier Ben Clemencic Marco Closier Joel |
| author_facet | Chitic Stefan-Gabriel Couturier Ben Clemencic Marco Closier Joel |
| author_sort | Chitic Stefan-Gabriel |
| collection | DOAJ |
| description | A continuous integration system is crucial to maintain the quality of the 6 millions lines of C++ and Python source code of the LHCb software in order to ensure consistent builds of the software as well as to run the unit and integration tests. Jenkins automation server is used for this purpose. It builds and tests around 100 configurations and produces in the order of 1500 built artifacts per day which are installed on the CVMFS file system or potentially on the developers’ machines.
Faced with a large and growing number of configurations built every day, and in order to ease inter-operation between the continuous integration system and the developers, we decided to put in place a flexible messaging system. As soon as the built artifacts have been produced, the distributed system allows their deployment based on the priority of the configurations.
We will describe the architecture of the new system, which is based on RabbitMQ messaging system (and the pika Python client library), and uses priority queues to start the LHCb software integration tests and to drive the installation of the nightly builds on the CVMFS file system. We will also show how the introduction of an event based system can help with the communication of results to developers. |
| first_indexed | 2024-12-16T18:19:08Z |
| format | Article |
| id | doaj.art-0f42e9cc5251403ba1d68212e9c578aa |
| institution | Directory Open Access Journal |
| issn | 2100-014X |
| language | English |
| last_indexed | 2024-12-16T18:19:08Z |
| publishDate | 2019-01-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | EPJ Web of Conferences |
| spelling | doaj.art-0f42e9cc5251403ba1d68212e9c578aa2022-12-21T22:21:36ZengEDP SciencesEPJ Web of Conferences2100-014X2019-01-012140500110.1051/epjconf/201921405001epjconf_chep2018_05001LHCb Continuous Integration and Deployment system: a message based approachChitic Stefan-GabrielCouturier BenClemencic MarcoClosier JoelA continuous integration system is crucial to maintain the quality of the 6 millions lines of C++ and Python source code of the LHCb software in order to ensure consistent builds of the software as well as to run the unit and integration tests. Jenkins automation server is used for this purpose. It builds and tests around 100 configurations and produces in the order of 1500 built artifacts per day which are installed on the CVMFS file system or potentially on the developers’ machines. Faced with a large and growing number of configurations built every day, and in order to ease inter-operation between the continuous integration system and the developers, we decided to put in place a flexible messaging system. As soon as the built artifacts have been produced, the distributed system allows their deployment based on the priority of the configurations. We will describe the architecture of the new system, which is based on RabbitMQ messaging system (and the pika Python client library), and uses priority queues to start the LHCb software integration tests and to drive the installation of the nightly builds on the CVMFS file system. We will also show how the introduction of an event based system can help with the communication of results to developers.https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_05001.pdf |
| spellingShingle | Chitic Stefan-Gabriel Couturier Ben Clemencic Marco Closier Joel LHCb Continuous Integration and Deployment system: a message based approach EPJ Web of Conferences |
| title | LHCb Continuous Integration and Deployment system: a message based approach |
| title_full | LHCb Continuous Integration and Deployment system: a message based approach |
| title_fullStr | LHCb Continuous Integration and Deployment system: a message based approach |
| title_full_unstemmed | LHCb Continuous Integration and Deployment system: a message based approach |
| title_short | LHCb Continuous Integration and Deployment system: a message based approach |
| title_sort | lhcb continuous integration and deployment system a message based approach |
| url | https://www.epj-conferences.org/articles/epjconf/pdf/2019/19/epjconf_chep2018_05001.pdf |
| work_keys_str_mv | AT chiticstefangabriel lhcbcontinuousintegrationanddeploymentsystemamessagebasedapproach AT couturierben lhcbcontinuousintegrationanddeploymentsystemamessagebasedapproach AT clemencicmarco lhcbcontinuousintegrationanddeploymentsystemamessagebasedapproach AT closierjoel lhcbcontinuousintegrationanddeploymentsystemamessagebasedapproach |