Minimum repair bandwidth for exact regeneration in distributed storage

Viveck R. Cadambe, Syed A. Jafar, Hamed Maleki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Citations (Scopus)

Abstract

We consider a set up where a file of size M is stored in n distributed storage nodes, using an (n, k) minimum storage regenerating (MSR) code, i.e., a maximum distance separable (MDS) code that also allows efficient exact-repair of any failed node. The MDS property ensures that the originalfile can be reconstructed even if any n - k storage nodes fail. When a node fails, a new node collects data from the remaining n - 1 healthy nodes and repairs the failed node. The problem of interest in this paper is to minimize the repair bandwidth B for exact regeneration of the failed node, i.e., the minimum data to be downloaded by the new node to replace the failed node by its exact replica. Previous work has shown that with random network coding, a bandwidth of B = M(n-1)/k(n-k) is necessary and sufficient for functional (not exact) regeneration, i.e., if the repaired new node need not be exactly identical to the failed node, but only information equivalent to it. It has also been shown using interference alignment based techniques that if k ≤ max(n/2, 3) then, surprisingly, there is no extra cost of exact regeneration over functional regeneration and the same repair bandwidth of M(n-1)/k(n-k) suffices for exact regeneration. The practically relevant setting of low-redundancy, i.e., k/n > 1/2 remained open for k > 3 and it has been shown that there is an extra bandwidth cost for exact repair over functional repair in this case. In this work, we adopt into the distributed storage context an asymptotically optimal interference alignment scheme previously proposed by Cadambe and Jafar for large wireless interference networks. With this scheme we solve the problem of repair bandwidth minimization for (n, k) exact-MSR codes for all (n, k) values including the previously open case of k > max(n/2, 3). Our main result is that, for any (n, k), and sufficiently large file sizes, there is no extra cost of exact regeneration over functional regeneration in terms of the repair bandwidth per bit of regenerated data. More precisely, we show that lim M→∞ B/M = n-1/k(n-k) . The result is analogous to the wireless interference channel setting where exact interference alignment through linear beamforming is seen to be infeasible for more than 3 users, but almost perfect alignmentis achieved asymptotically by the Cadambe-Jafar scheme over a large number of signaling dimensions for any number of users.

Original languageEnglish (US)
Title of host publication2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010
Pages65-70
Number of pages6
DOIs
StatePublished - Aug 5 2010
Event2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010 - Boston, MA, United States
Duration: Jun 21 2010Jun 21 2010

Publication series

Name2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010

Other

Other2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010
CountryUnited States
CityBoston, MA
Period6/21/106/21/10

Fingerprint

Repair
Bandwidth
Costs
Network coding
Beamforming
Redundancy

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Computer Networks and Communications

Cite this

Cadambe, V. R., Jafar, S. A., & Maleki, H. (2010). Minimum repair bandwidth for exact regeneration in distributed storage. In 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010 (pp. 65-70). [5507931] (2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010). https://doi.org/10.1109/WINC.2010.5507931
Cadambe, Viveck R. ; Jafar, Syed A. ; Maleki, Hamed. / Minimum repair bandwidth for exact regeneration in distributed storage. 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010. 2010. pp. 65-70 (2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010).
@inproceedings{01a20ab9147943068802ef112aa7633d,
title = "Minimum repair bandwidth for exact regeneration in distributed storage",
abstract = "We consider a set up where a file of size M is stored in n distributed storage nodes, using an (n, k) minimum storage regenerating (MSR) code, i.e., a maximum distance separable (MDS) code that also allows efficient exact-repair of any failed node. The MDS property ensures that the originalfile can be reconstructed even if any n - k storage nodes fail. When a node fails, a new node collects data from the remaining n - 1 healthy nodes and repairs the failed node. The problem of interest in this paper is to minimize the repair bandwidth B for exact regeneration of the failed node, i.e., the minimum data to be downloaded by the new node to replace the failed node by its exact replica. Previous work has shown that with random network coding, a bandwidth of B = M(n-1)/k(n-k) is necessary and sufficient for functional (not exact) regeneration, i.e., if the repaired new node need not be exactly identical to the failed node, but only information equivalent to it. It has also been shown using interference alignment based techniques that if k ≤ max(n/2, 3) then, surprisingly, there is no extra cost of exact regeneration over functional regeneration and the same repair bandwidth of M(n-1)/k(n-k) suffices for exact regeneration. The practically relevant setting of low-redundancy, i.e., k/n > 1/2 remained open for k > 3 and it has been shown that there is an extra bandwidth cost for exact repair over functional repair in this case. In this work, we adopt into the distributed storage context an asymptotically optimal interference alignment scheme previously proposed by Cadambe and Jafar for large wireless interference networks. With this scheme we solve the problem of repair bandwidth minimization for (n, k) exact-MSR codes for all (n, k) values including the previously open case of k > max(n/2, 3). Our main result is that, for any (n, k), and sufficiently large file sizes, there is no extra cost of exact regeneration over functional regeneration in terms of the repair bandwidth per bit of regenerated data. More precisely, we show that lim M→∞ B/M = n-1/k(n-k) . The result is analogous to the wireless interference channel setting where exact interference alignment through linear beamforming is seen to be infeasible for more than 3 users, but almost perfect alignmentis achieved asymptotically by the Cadambe-Jafar scheme over a large number of signaling dimensions for any number of users.",
author = "Cadambe, {Viveck R.} and Jafar, {Syed A.} and Hamed Maleki",
year = "2010",
month = "8",
day = "5",
doi = "10.1109/WINC.2010.5507931",
language = "English (US)",
isbn = "9781424479801",
series = "2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010",
pages = "65--70",
booktitle = "2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010",

}

Cadambe, VR, Jafar, SA & Maleki, H 2010, Minimum repair bandwidth for exact regeneration in distributed storage. in 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010., 5507931, 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010, pp. 65-70, 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010, Boston, MA, United States, 6/21/10. https://doi.org/10.1109/WINC.2010.5507931

Minimum repair bandwidth for exact regeneration in distributed storage. / Cadambe, Viveck R.; Jafar, Syed A.; Maleki, Hamed.

2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010. 2010. p. 65-70 5507931 (2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Minimum repair bandwidth for exact regeneration in distributed storage

AU - Cadambe, Viveck R.

AU - Jafar, Syed A.

AU - Maleki, Hamed

PY - 2010/8/5

Y1 - 2010/8/5

N2 - We consider a set up where a file of size M is stored in n distributed storage nodes, using an (n, k) minimum storage regenerating (MSR) code, i.e., a maximum distance separable (MDS) code that also allows efficient exact-repair of any failed node. The MDS property ensures that the originalfile can be reconstructed even if any n - k storage nodes fail. When a node fails, a new node collects data from the remaining n - 1 healthy nodes and repairs the failed node. The problem of interest in this paper is to minimize the repair bandwidth B for exact regeneration of the failed node, i.e., the minimum data to be downloaded by the new node to replace the failed node by its exact replica. Previous work has shown that with random network coding, a bandwidth of B = M(n-1)/k(n-k) is necessary and sufficient for functional (not exact) regeneration, i.e., if the repaired new node need not be exactly identical to the failed node, but only information equivalent to it. It has also been shown using interference alignment based techniques that if k ≤ max(n/2, 3) then, surprisingly, there is no extra cost of exact regeneration over functional regeneration and the same repair bandwidth of M(n-1)/k(n-k) suffices for exact regeneration. The practically relevant setting of low-redundancy, i.e., k/n > 1/2 remained open for k > 3 and it has been shown that there is an extra bandwidth cost for exact repair over functional repair in this case. In this work, we adopt into the distributed storage context an asymptotically optimal interference alignment scheme previously proposed by Cadambe and Jafar for large wireless interference networks. With this scheme we solve the problem of repair bandwidth minimization for (n, k) exact-MSR codes for all (n, k) values including the previously open case of k > max(n/2, 3). Our main result is that, for any (n, k), and sufficiently large file sizes, there is no extra cost of exact regeneration over functional regeneration in terms of the repair bandwidth per bit of regenerated data. More precisely, we show that lim M→∞ B/M = n-1/k(n-k) . The result is analogous to the wireless interference channel setting where exact interference alignment through linear beamforming is seen to be infeasible for more than 3 users, but almost perfect alignmentis achieved asymptotically by the Cadambe-Jafar scheme over a large number of signaling dimensions for any number of users.

AB - We consider a set up where a file of size M is stored in n distributed storage nodes, using an (n, k) minimum storage regenerating (MSR) code, i.e., a maximum distance separable (MDS) code that also allows efficient exact-repair of any failed node. The MDS property ensures that the originalfile can be reconstructed even if any n - k storage nodes fail. When a node fails, a new node collects data from the remaining n - 1 healthy nodes and repairs the failed node. The problem of interest in this paper is to minimize the repair bandwidth B for exact regeneration of the failed node, i.e., the minimum data to be downloaded by the new node to replace the failed node by its exact replica. Previous work has shown that with random network coding, a bandwidth of B = M(n-1)/k(n-k) is necessary and sufficient for functional (not exact) regeneration, i.e., if the repaired new node need not be exactly identical to the failed node, but only information equivalent to it. It has also been shown using interference alignment based techniques that if k ≤ max(n/2, 3) then, surprisingly, there is no extra cost of exact regeneration over functional regeneration and the same repair bandwidth of M(n-1)/k(n-k) suffices for exact regeneration. The practically relevant setting of low-redundancy, i.e., k/n > 1/2 remained open for k > 3 and it has been shown that there is an extra bandwidth cost for exact repair over functional repair in this case. In this work, we adopt into the distributed storage context an asymptotically optimal interference alignment scheme previously proposed by Cadambe and Jafar for large wireless interference networks. With this scheme we solve the problem of repair bandwidth minimization for (n, k) exact-MSR codes for all (n, k) values including the previously open case of k > max(n/2, 3). Our main result is that, for any (n, k), and sufficiently large file sizes, there is no extra cost of exact regeneration over functional regeneration in terms of the repair bandwidth per bit of regenerated data. More precisely, we show that lim M→∞ B/M = n-1/k(n-k) . The result is analogous to the wireless interference channel setting where exact interference alignment through linear beamforming is seen to be infeasible for more than 3 users, but almost perfect alignmentis achieved asymptotically by the Cadambe-Jafar scheme over a large number of signaling dimensions for any number of users.

UR - http://www.scopus.com/inward/record.url?scp=77955113465&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77955113465&partnerID=8YFLogxK

U2 - 10.1109/WINC.2010.5507931

DO - 10.1109/WINC.2010.5507931

M3 - Conference contribution

AN - SCOPUS:77955113465

SN - 9781424479801

T3 - 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010

SP - 65

EP - 70

BT - 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010

ER -

Cadambe VR, Jafar SA, Maleki H. Minimum repair bandwidth for exact regeneration in distributed storage. In 2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010. 2010. p. 65-70. 5507931. (2010 3rd IEEE International Workshop on Wireless Network Coding, WiNC 2010). https://doi.org/10.1109/WINC.2010.5507931