The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems
© 1963-2012 IEEE. We study a generalization of the setting of regenerating codes, motivated by applications to storage systems consisting of clusters of storage nodes. There are $n$ clusters in total, with $m$ nodes per cluster. A data file is coded and stored across the $mn$ nodes, with each node s...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/134886 |
| _version_ | 1826200564252278784 |
|---|---|
| author | Prakash, N Abdrashitov, Vitaly Medard, Muriel |
| author_facet | Prakash, N Abdrashitov, Vitaly Medard, Muriel |
| author_sort | Prakash, N |
| collection | MIT |
| description | © 1963-2012 IEEE. We study a generalization of the setting of regenerating codes, motivated by applications to storage systems consisting of clusters of storage nodes. There are $n$ clusters in total, with $m$ nodes per cluster. A data file is coded and stored across the $mn$ nodes, with each node storing α symbols. For availability of data, we require that the file be retrievable by downloading the entire content from any subset of $k$ clusters. Nodes represent entities that can fail. We distinguish between intra-cluster and inter-cluster bandwidth (BW) costs during node repair. Node-repair in a cluster is accomplished by downloading β symbols each from any set of $d$ other clusters, dubbed remote helper clusters, and also up to α symbols each from any set of $\ell $ surviving nodes, dubbed local helper nodes, in the host cluster. We first identify the optimal trade-off between storage-overhead and inter-cluster repair-bandwidth under functional repair, and also present optimal exact-repair code constructions for a class of parameters. The new trade-off is strictly better than what is achievable via space-sharing existing coding solutions, whenever ℓ > 0$. We then obtain sharp lower bounds on the necessary intra-cluster repair BW to achieve optimal trade-off. Under functional repair, random linear network codes (RLNCs) simultaneously optimize usage of both inter- and intra-cluster repair BW; simulation results based on RLNCs suggest optimality of the bounds on intra-cluster repair-bandwidth. Our bounds reveal the interesting fact that, while it is beneficial to increase the number of local helper nodes ℓ $ in order to improve the storage-vs-inter-cluster-repair-BW trade-off, increasing ℓ $ not only increases intra-cluster BW in the host-cluster, but also increases the intra-cluster BW in the remote helper clusters. We also analyze resilience of the clustered storage system against passive eavesdropping by providing file-size bounds and optimal code constructions. |
| first_indexed | 2024-09-23T11:38:20Z |
| format | Article |
| id | mit-1721.1/134886 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T11:38:20Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1348862022-04-01T16:18:07Z The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems Prakash, N Abdrashitov, Vitaly Medard, Muriel © 1963-2012 IEEE. We study a generalization of the setting of regenerating codes, motivated by applications to storage systems consisting of clusters of storage nodes. There are $n$ clusters in total, with $m$ nodes per cluster. A data file is coded and stored across the $mn$ nodes, with each node storing α symbols. For availability of data, we require that the file be retrievable by downloading the entire content from any subset of $k$ clusters. Nodes represent entities that can fail. We distinguish between intra-cluster and inter-cluster bandwidth (BW) costs during node repair. Node-repair in a cluster is accomplished by downloading β symbols each from any set of $d$ other clusters, dubbed remote helper clusters, and also up to α symbols each from any set of $\ell $ surviving nodes, dubbed local helper nodes, in the host cluster. We first identify the optimal trade-off between storage-overhead and inter-cluster repair-bandwidth under functional repair, and also present optimal exact-repair code constructions for a class of parameters. The new trade-off is strictly better than what is achievable via space-sharing existing coding solutions, whenever ℓ > 0$. We then obtain sharp lower bounds on the necessary intra-cluster repair BW to achieve optimal trade-off. Under functional repair, random linear network codes (RLNCs) simultaneously optimize usage of both inter- and intra-cluster repair BW; simulation results based on RLNCs suggest optimality of the bounds on intra-cluster repair-bandwidth. Our bounds reveal the interesting fact that, while it is beneficial to increase the number of local helper nodes ℓ $ in order to improve the storage-vs-inter-cluster-repair-BW trade-off, increasing ℓ $ not only increases intra-cluster BW in the host-cluster, but also increases the intra-cluster BW in the remote helper clusters. We also analyze resilience of the clustered storage system against passive eavesdropping by providing file-size bounds and optimal code constructions. 2021-10-27T20:09:40Z 2021-10-27T20:09:40Z 2018 2019-06-21T12:57:50Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/134886 en 10.1109/TIT.2018.2806342 IEEE Transactions on Information Theory Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Prakash, N Abdrashitov, Vitaly Medard, Muriel The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems |
| title | The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems |
| title_full | The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems |
| title_fullStr | The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems |
| title_full_unstemmed | The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems |
| title_short | The Storage Versus Repair-Bandwidth Trade-off for Clustered Storage Systems |
| title_sort | storage versus repair bandwidth trade off for clustered storage systems |
| url | https://hdl.handle.net/1721.1/134886 |
| work_keys_str_mv | AT prakashn thestorageversusrepairbandwidthtradeoffforclusteredstoragesystems AT abdrashitovvitaly thestorageversusrepairbandwidthtradeoffforclusteredstoragesystems AT medardmuriel thestorageversusrepairbandwidthtradeoffforclusteredstoragesystems AT prakashn storageversusrepairbandwidthtradeoffforclusteredstoragesystems AT abdrashitovvitaly storageversusrepairbandwidthtradeoffforclusteredstoragesystems AT medardmuriel storageversusrepairbandwidthtradeoffforclusteredstoragesystems |