Internet-scale volunteer desktop grids allow multiple applications to execute concurrently on heterogeneous and dynamic platforms. In this paper we explore the problem of discovering resources and scheduling multiple Bag-of-Task (BoT) applications in unstructured peer-to-peer (P2P) volunteer desktop grids using a super-based approach. This approach tackles the single-point failure and scalability problems faced by centralized and distributed approaches. A light-weight change driven gossip protocol is used to exchange resource availability information aimed at (1) providing a good degree of speed-up in bag turnaround time, and (2) minimizing the control message overhead to maintain scalability. We present a comprehensive study of the properties of fixed threshold and adaptive threshold...[ Read more ]

Internet-scale volunteer desktop grids allow multiple applications to execute concurrently on heterogeneous and dynamic platforms. In this paper we explore the problem of discovering resources and scheduling multiple Bag-of-Task (BoT) applications in unstructured peer-to-peer (P2P) volunteer desktop grids using a super-based approach. This approach tackles the single-point failure and scalability problems faced by centralized and distributed approaches. A light-weight change driven gossip protocol is used to exchange resource availability information aimed at (1) providing a good degree of speed-up in bag turnaround time, and (2) minimizing the control message overhead to maintain scalability. We present a comprehensive study of the properties of fixed threshold and adaptive threshold based gossiping mechanisms using detailed simulation experiments. Our results indicate that while gossiping with a fixed threshold reflects resource usage effectively in a scalable network, an adaptive gossip threshold mechanism based on a simple feedback mechanism considering system state adapts much better to a volatile environment with small latency, reduced overhead and yields better system performance compared to fixed threshold protocols.