Census: Location-Aware Membership Management for Large-Scale Distributed Systems
We present Census, a platform for building large-scale distributed applications. Census provides a membership service and a multicast mechanism. The membership service provides every node with a consistent view of the system membership, which may be global or partitioned into location-based regions....
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
USENIX Association
2011
|
| Online Access: | http://hdl.handle.net/1721.1/61401 https://orcid.org/0000-0002-5914-1866 |
| _version_ | 1826213814381576192 |
|---|---|
| author | Cowling, James Alexander Ports, Dan R. K. Liskov, Barbara H. Popa, Raluca Ada Gaikwad, Abhijeet |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Cowling, James Alexander Ports, Dan R. K. Liskov, Barbara H. Popa, Raluca Ada Gaikwad, Abhijeet |
| author_sort | Cowling, James Alexander |
| collection | MIT |
| description | We present Census, a platform for building large-scale distributed applications. Census provides a membership service and a multicast mechanism. The membership service provides every node with a consistent view of the system membership, which may be global or partitioned into location-based regions. Census distributes membership updates with low overhead, propagates changes promptly, and is resilient to both crashes and Byzantine failures. We believe that Census is the first system to provide a consistent membership abstraction at very large scale, greatly simplifying the design of applications built atop large deployments such as multi-site data centers.
Census builds on a novel multicast mechanism that is closely integrated with the membership service. It organizes nodes into a reliable overlay composed of multiple distribution trees, using network coordinates to minimize latency. Unlike other multicast systems, it avoids the cost of using distributed algorithms to construct and maintain trees. Instead, each node independently produces the same trees from the consistent membership view. Census uses this multicast mechanism to distribute membership updates, along with application-provided messages.
We evaluate the platform under simulation and on a real-world deployment on PlanetLab. We find that it imposes minimal bandwidth overhead, is able to react quickly to node failures and changes in the system membership, and can scale to substantial size. |
| first_indexed | 2024-09-23T15:55:15Z |
| format | Article |
| id | mit-1721.1/61401 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T15:55:15Z |
| publishDate | 2011 |
| publisher | USENIX Association |
| record_format | dspace |
| spelling | mit-1721.1/614012022-10-02T05:02:28Z Census: Location-Aware Membership Management for Large-Scale Distributed Systems Cowling, James Alexander Ports, Dan R. K. Liskov, Barbara H. Popa, Raluca Ada Gaikwad, Abhijeet Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Liskov, Barbara H. Cowling, James Alexander Ports, Dan R. K. Liskov, Barbara H. Popa, Raluca Ada We present Census, a platform for building large-scale distributed applications. Census provides a membership service and a multicast mechanism. The membership service provides every node with a consistent view of the system membership, which may be global or partitioned into location-based regions. Census distributes membership updates with low overhead, propagates changes promptly, and is resilient to both crashes and Byzantine failures. We believe that Census is the first system to provide a consistent membership abstraction at very large scale, greatly simplifying the design of applications built atop large deployments such as multi-site data centers. Census builds on a novel multicast mechanism that is closely integrated with the membership service. It organizes nodes into a reliable overlay composed of multiple distribution trees, using network coordinates to minimize latency. Unlike other multicast systems, it avoids the cost of using distributed algorithms to construct and maintain trees. Instead, each node independently produces the same trees from the consistent membership view. Census uses this multicast mechanism to distribute membership updates, along with application-provided messages. We evaluate the platform under simulation and on a real-world deployment on PlanetLab. We find that it imposes minimal bandwidth overhead, is able to react quickly to node failures and changes in the system membership, and can scale to substantial size. National Science Foundation (U.S.) (ITR grant CNS-0428107) 2011-03-04T14:33:41Z 2011-03-04T14:33:41Z 2009-06 2009-06 Article http://purl.org/eprint/type/ConferencePaper 978-1-931971-68-3 http://hdl.handle.net/1721.1/61401 Cowling, J., et al. "Census: Location-Aware Membership Management for Large-Scale Distributed Systems." Proceedings of the 2009 USENIX Annual Technical Conference (San Diego: June 14-19, 2009). https://orcid.org/0000-0002-5914-1866 en_US http://www.usenix.org/event/usenix09/tech/ Proceedings of the 2009 USENIX Annual Technical Conference Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf USENIX Association MIT web domain |
| spellingShingle | Cowling, James Alexander Ports, Dan R. K. Liskov, Barbara H. Popa, Raluca Ada Gaikwad, Abhijeet Census: Location-Aware Membership Management for Large-Scale Distributed Systems |
| title | Census: Location-Aware Membership Management for Large-Scale Distributed Systems |
| title_full | Census: Location-Aware Membership Management for Large-Scale Distributed Systems |
| title_fullStr | Census: Location-Aware Membership Management for Large-Scale Distributed Systems |
| title_full_unstemmed | Census: Location-Aware Membership Management for Large-Scale Distributed Systems |
| title_short | Census: Location-Aware Membership Management for Large-Scale Distributed Systems |
| title_sort | census location aware membership management for large scale distributed systems |
| url | http://hdl.handle.net/1721.1/61401 https://orcid.org/0000-0002-5914-1866 |
| work_keys_str_mv | AT cowlingjamesalexander censuslocationawaremembershipmanagementforlargescaledistributedsystems AT portsdanrk censuslocationawaremembershipmanagementforlargescaledistributedsystems AT liskovbarbarah censuslocationawaremembershipmanagementforlargescaledistributedsystems AT poparalucaada censuslocationawaremembershipmanagementforlargescaledistributedsystems AT gaikwadabhijeet censuslocationawaremembershipmanagementforlargescaledistributedsystems |