Technology readiness levels for machine learning systems
The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequ...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Internet publication |
| Language: | English |
| Published: |
Research Square
2021
|
| _version_ | 1826311431477264384 |
|---|---|
| author | Lavin, A Gilligan-Lee, CM Visnjic, A Ganju, S Newman, D Baydin, AG Ganguly, S Lange, D Sharma, A Zheng, S Xing, EP Gibson, A Parr, J Mattmann, C Gal, Y |
| author_facet | Lavin, A Gilligan-Lee, CM Visnjic, A Ganju, S Newman, D Baydin, AG Ganguly, S Lange, D Sharma, A Zheng, S Xing, EP Gibson, A Parr, J Mattmann, C Gal, Y |
| author_sort | Lavin, A |
| collection | OXFORD |
| description | The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards to streamline development for high-quality, reliable results. The extreme is spacecraft systems, where mission critical measures and robustness are ingrained in the development process. Drawing on experience in both spacecraft engineering and ML (from research through product across domain areas), we have developed a proven systems engineering approach for machine learning development and deployment. Our Machine Learning Technology Readiness Levels (MLTRL) framework defines a principled process to ensure robust, reliable, and responsible systems while being streamlined for ML workflows, including key distinctions from traditional software engineering. Even more, MLTRL defines a lingua franca for people across teams and organizations to work collaboratively on artificial intelligence and machine learning technologies. Here we describe the framework and elucidate it with several real world use-cases of developing ML methods from basic research through productization and deployment, in areas such as medical diagnostics, consumer computer vision, satellite imagery, and particle physics. |
| first_indexed | 2024-03-07T08:08:19Z |
| format | Internet publication |
| id | oxford-uuid:8c5d06b1-b030-4033-907d-f9140da67317 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:08:19Z |
| publishDate | 2021 |
| publisher | Research Square |
| record_format | dspace |
| spelling | oxford-uuid:8c5d06b1-b030-4033-907d-f9140da673172023-11-14T08:20:15ZTechnology readiness levels for machine learning systemsInternet publicationhttp://purl.org/coar/resource_type/c_7ad9uuid:8c5d06b1-b030-4033-907d-f9140da67317EnglishSymplectic ElementsResearch Square2021Lavin, AGilligan-Lee, CMVisnjic, AGanju, SNewman, DBaydin, AGGanguly, SLange, DSharma, AZheng, SXing, EPGibson, AParr, JMattmann, CGal, YThe development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards to streamline development for high-quality, reliable results. The extreme is spacecraft systems, where mission critical measures and robustness are ingrained in the development process. Drawing on experience in both spacecraft engineering and ML (from research through product across domain areas), we have developed a proven systems engineering approach for machine learning development and deployment. Our Machine Learning Technology Readiness Levels (MLTRL) framework defines a principled process to ensure robust, reliable, and responsible systems while being streamlined for ML workflows, including key distinctions from traditional software engineering. Even more, MLTRL defines a lingua franca for people across teams and organizations to work collaboratively on artificial intelligence and machine learning technologies. Here we describe the framework and elucidate it with several real world use-cases of developing ML methods from basic research through productization and deployment, in areas such as medical diagnostics, consumer computer vision, satellite imagery, and particle physics. |
| spellingShingle | Lavin, A Gilligan-Lee, CM Visnjic, A Ganju, S Newman, D Baydin, AG Ganguly, S Lange, D Sharma, A Zheng, S Xing, EP Gibson, A Parr, J Mattmann, C Gal, Y Technology readiness levels for machine learning systems |
| title | Technology readiness levels for machine learning systems |
| title_full | Technology readiness levels for machine learning systems |
| title_fullStr | Technology readiness levels for machine learning systems |
| title_full_unstemmed | Technology readiness levels for machine learning systems |
| title_short | Technology readiness levels for machine learning systems |
| title_sort | technology readiness levels for machine learning systems |
| work_keys_str_mv | AT lavina technologyreadinesslevelsformachinelearningsystems AT gilliganleecm technologyreadinesslevelsformachinelearningsystems AT visnjica technologyreadinesslevelsformachinelearningsystems AT ganjus technologyreadinesslevelsformachinelearningsystems AT newmand technologyreadinesslevelsformachinelearningsystems AT baydinag technologyreadinesslevelsformachinelearningsystems AT gangulys technologyreadinesslevelsformachinelearningsystems AT langed technologyreadinesslevelsformachinelearningsystems AT sharmaa technologyreadinesslevelsformachinelearningsystems AT zhengs technologyreadinesslevelsformachinelearningsystems AT xingep technologyreadinesslevelsformachinelearningsystems AT gibsona technologyreadinesslevelsformachinelearningsystems AT parrj technologyreadinesslevelsformachinelearningsystems AT mattmannc technologyreadinesslevelsformachinelearningsystems AT galy technologyreadinesslevelsformachinelearningsystems |