A fair evaluation of the potential of machine learning in maritime transportation
Machine learning (ML) techniques are extensively applied to practical maritime transportation issues. Due to the difficulty and high cost of collecting large volumes of data in the maritime industry, in many maritime studies, ML models are trained with small training datasets. The relative predictiv...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/173568 |
| _version_ | 1811687388753166336 |
|---|---|
| author | Luo, Xi Yan, Ran Wang, Shuaian Zhen, Lu |
| author2 | School of Civil and Environmental Engineering |
| author_facet | School of Civil and Environmental Engineering Luo, Xi Yan, Ran Wang, Shuaian Zhen, Lu |
| author_sort | Luo, Xi |
| collection | NTU |
| description | Machine learning (ML) techniques are extensively applied to practical maritime transportation issues. Due to the difficulty and high cost of collecting large volumes of data in the maritime industry, in many maritime studies, ML models are trained with small training datasets. The relative predictive performances of these trained ML models are then compared with each other and with the conventional model using the same test set. The ML model that performs the best out of the ML models and better than the conventional model on the test set is regarded as the most effective in terms of this prediction task. However, in scenarios with small datasets, this common process may lead to an unfair comparison between the ML and the conventional model. Therefore, we propose a novel process to fairly compare multiple ML models and the conventional model. We first select the best ML model in terms of predictive performance for the validation set. Then, we combine the training and the validation sets to retrain the best ML model and compare it with the conventional model on the same test set. Based on historical port state control (PSC) inspection data, we examine both the common process and the novel process in terms of their ability to fairly compare ML models and the conventional model. The results show that the novel process is more effective at fairly comparing the ML models with the conventional model on different test sets. Therefore, the novel process enables a fair assessment of ML models’ ability to predict key performance indicators in the context of limited data availability in the maritime industry, such as predicting the ship fuel consumption and port traffic volume, thereby enhancing their reliability for real-world applications. |
| first_indexed | 2024-10-01T05:15:31Z |
| format | Journal Article |
| id | ntu-10356/173568 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:15:31Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1735682024-02-16T15:33:05Z A fair evaluation of the potential of machine learning in maritime transportation Luo, Xi Yan, Ran Wang, Shuaian Zhen, Lu School of Civil and Environmental Engineering Engineering Fair evaluation of prediction models Machine learning Machine learning (ML) techniques are extensively applied to practical maritime transportation issues. Due to the difficulty and high cost of collecting large volumes of data in the maritime industry, in many maritime studies, ML models are trained with small training datasets. The relative predictive performances of these trained ML models are then compared with each other and with the conventional model using the same test set. The ML model that performs the best out of the ML models and better than the conventional model on the test set is regarded as the most effective in terms of this prediction task. However, in scenarios with small datasets, this common process may lead to an unfair comparison between the ML and the conventional model. Therefore, we propose a novel process to fairly compare multiple ML models and the conventional model. We first select the best ML model in terms of predictive performance for the validation set. Then, we combine the training and the validation sets to retrain the best ML model and compare it with the conventional model on the same test set. Based on historical port state control (PSC) inspection data, we examine both the common process and the novel process in terms of their ability to fairly compare ML models and the conventional model. The results show that the novel process is more effective at fairly comparing the ML models with the conventional model on different test sets. Therefore, the novel process enables a fair assessment of ML models’ ability to predict key performance indicators in the context of limited data availability in the maritime industry, such as predicting the ship fuel consumption and port traffic volume, thereby enhancing their reliability for real-world applications. Published version 2024-02-14T06:04:20Z 2024-02-14T06:04:20Z 2023 Journal Article Luo, X., Yan, R., Wang, S. & Zhen, L. (2023). A fair evaluation of the potential of machine learning in maritime transportation. Electronic Research Archive, 31(8), 4753-4772. https://dx.doi.org/10.3934/era.2023243 2688-1594 https://hdl.handle.net/10356/173568 10.3934/era.2023243 2-s2.0-85168591413 8 31 4753 4772 en Electronic Research Archive © 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0). application/pdf |
| spellingShingle | Engineering Fair evaluation of prediction models Machine learning Luo, Xi Yan, Ran Wang, Shuaian Zhen, Lu A fair evaluation of the potential of machine learning in maritime transportation |
| title | A fair evaluation of the potential of machine learning in maritime transportation |
| title_full | A fair evaluation of the potential of machine learning in maritime transportation |
| title_fullStr | A fair evaluation of the potential of machine learning in maritime transportation |
| title_full_unstemmed | A fair evaluation of the potential of machine learning in maritime transportation |
| title_short | A fair evaluation of the potential of machine learning in maritime transportation |
| title_sort | fair evaluation of the potential of machine learning in maritime transportation |
| topic | Engineering Fair evaluation of prediction models Machine learning |
| url | https://hdl.handle.net/10356/173568 |
| work_keys_str_mv | AT luoxi afairevaluationofthepotentialofmachinelearninginmaritimetransportation AT yanran afairevaluationofthepotentialofmachinelearninginmaritimetransportation AT wangshuaian afairevaluationofthepotentialofmachinelearninginmaritimetransportation AT zhenlu afairevaluationofthepotentialofmachinelearninginmaritimetransportation AT luoxi fairevaluationofthepotentialofmachinelearninginmaritimetransportation AT yanran fairevaluationofthepotentialofmachinelearninginmaritimetransportation AT wangshuaian fairevaluationofthepotentialofmachinelearninginmaritimetransportation AT zhenlu fairevaluationofthepotentialofmachinelearninginmaritimetransportation |