Processing moldable tasks on the grid : late job binding with lightweight user-level overlay
Independent observations and everyday user experience indicate that performance and reliability of large grid infrastructures may suffer from large and unpredictable variations. In this paper we study the impact of the job queuing time on processing of moldable tasks which are commonly found in larg...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2013
|
| Online Access: | https://hdl.handle.net/10356/96035 http://hdl.handle.net/10220/10124 |
| _version_ | 1826118223827828736 |
|---|---|
| author | Mościcki, J. T. Lamanna, M. Sloot, Peter M. A. Bubak, Marian |
| author2 | School of Computer Engineering |
| author_facet | School of Computer Engineering Mościcki, J. T. Lamanna, M. Sloot, Peter M. A. Bubak, Marian |
| author_sort | Mościcki, J. T. |
| collection | NTU |
| description | Independent observations and everyday user experience indicate that performance and reliability of large grid infrastructures may suffer from large and unpredictable variations. In this paper we study the impact of the job queuing time on processing of moldable tasks which are commonly found in large-scale production grids. We use the mean value and variance of makespan as the quality of service indicators. We develop a general task processing model to provide a quantitative comparison between two models: early and late job binding in a user-level overlay applied to the EGEE Grid infrastructure. We find that the late-binding model effectively defines a transformation of the distribution of makespan according to the Central Limit Theorem. As demonstrated by Monte Carlo simulations using real job traces, this transformation allows to substantially reduce the mean value and variance of makespan. For certain classes of applications task granularity may be adjusted such that a speedup of an order of magnitude or more may be achieved. We use this result to propose a general strategy for managing access to resources and optimization of workload based on Ganga and DIANE user-level overlay tools. Key features of this approach include: a late-binding scheduler, an ability to interface to a wide range of distributed systems, an ability to extend and customize the system to cover application-specific scheduling and processing patterns and finally, ease of use and lightweight deployment in the user space. We discuss the impact of this approach for some practical applications where efficient processing of many tasks is required to solve scientific problems. |
| first_indexed | 2024-10-01T04:39:52Z |
| format | Journal Article |
| id | ntu-10356/96035 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:39:52Z |
| publishDate | 2013 |
| record_format | dspace |
| spelling | ntu-10356/960352020-05-28T07:18:33Z Processing moldable tasks on the grid : late job binding with lightweight user-level overlay Mościcki, J. T. Lamanna, M. Sloot, Peter M. A. Bubak, Marian School of Computer Engineering Independent observations and everyday user experience indicate that performance and reliability of large grid infrastructures may suffer from large and unpredictable variations. In this paper we study the impact of the job queuing time on processing of moldable tasks which are commonly found in large-scale production grids. We use the mean value and variance of makespan as the quality of service indicators. We develop a general task processing model to provide a quantitative comparison between two models: early and late job binding in a user-level overlay applied to the EGEE Grid infrastructure. We find that the late-binding model effectively defines a transformation of the distribution of makespan according to the Central Limit Theorem. As demonstrated by Monte Carlo simulations using real job traces, this transformation allows to substantially reduce the mean value and variance of makespan. For certain classes of applications task granularity may be adjusted such that a speedup of an order of magnitude or more may be achieved. We use this result to propose a general strategy for managing access to resources and optimization of workload based on Ganga and DIANE user-level overlay tools. Key features of this approach include: a late-binding scheduler, an ability to interface to a wide range of distributed systems, an ability to extend and customize the system to cover application-specific scheduling and processing patterns and finally, ease of use and lightweight deployment in the user space. We discuss the impact of this approach for some practical applications where efficient processing of many tasks is required to solve scientific problems. 2013-06-10T07:09:23Z 2019-12-06T19:24:45Z 2013-06-10T07:09:23Z 2019-12-06T19:24:45Z 2011 2011 Journal Article Mościcki, J. T., Lamanna, M., Bubak, M., & Sloot, Peter M. A. (2011). Processing moldable tasks on the grid : late job binding with lightweight user-level overlay. Future generation computer systems, 27(6), 725-736. https://hdl.handle.net/10356/96035 http://hdl.handle.net/10220/10124 10.1016/j.future.2011.02.002 en Future generation computer systems © 2011 Elsevier B.V. |
| spellingShingle | Mościcki, J. T. Lamanna, M. Sloot, Peter M. A. Bubak, Marian Processing moldable tasks on the grid : late job binding with lightweight user-level overlay |
| title | Processing moldable tasks on the grid : late job binding with lightweight user-level overlay |
| title_full | Processing moldable tasks on the grid : late job binding with lightweight user-level overlay |
| title_fullStr | Processing moldable tasks on the grid : late job binding with lightweight user-level overlay |
| title_full_unstemmed | Processing moldable tasks on the grid : late job binding with lightweight user-level overlay |
| title_short | Processing moldable tasks on the grid : late job binding with lightweight user-level overlay |
| title_sort | processing moldable tasks on the grid late job binding with lightweight user level overlay |
| url | https://hdl.handle.net/10356/96035 http://hdl.handle.net/10220/10124 |
| work_keys_str_mv | AT moscickijt processingmoldabletasksonthegridlatejobbindingwithlightweightuserleveloverlay AT lamannam processingmoldabletasksonthegridlatejobbindingwithlightweightuserleveloverlay AT slootpeterma processingmoldabletasksonthegridlatejobbindingwithlightweightuserleveloverlay AT bubakmarian processingmoldabletasksonthegridlatejobbindingwithlightweightuserleveloverlay |