We have recently proposed a semi-decentralized collaborative hosting system Wikipedia. In this system, individuals willing to help host Wikipedia join their computer to an overlay network of nodes, which handles a significant part of Wikipedia's workload, resulting in reduced operational cost for Wikipedia. We achieve this by distributing replicas of Wikipedia pages and files over nodes in the collaborative overlay. A key mechanism of this architecture is a service that allows clients to discover which nodes host replicas of any given object.
This objective of this project is to explore the trade-off between centralized and decentralized strategies for locating objects. A fully decentralized approach provides maximum load reduction for the centrally managed infrastructure at the cost of higher client-perceived latency, whereas a centrally managed approach can provide lower latency, but no load reduction.
We have tested our architecture using a location service based on a fully decentralized DHT model, but we also want to evaluate centralized and semi-decentralized strategies. We propose the development and evaluation of two simple centralized and semi-decentralized solutions.
The semi-decentralized approach uses the DHT for location-data distribution, but all nodes send hearbeats to a central site that consequently knows about all members in the system. Under this semi-decentralized scheme, all lookups can start at the central site and be resolved in only two hops, as opposed to the logarithmic number of hops of a typical (secure) DHT.
The centralized approach involves all nodes sending not only heartbeats to the central site, but also notifications about the addition and removal of replicated objects. This allows all lookups to be resolved by the central site in a singe step.
In both cases the simple proposed heartbeat-based failure-detection system may be replaced by a more efficient failure detection system.
Prerequisites
The project will require development under a simulation environment written in C. Therefore, proficiency in this programming language is required.
References