Smartphone is now a necessity of people’s daily life, and we are enjoying various services on it with numerous mobile applications. However, the resource and communication limitations of a single mobile device make it insufficient in satisfying the real-time and interactive constraints of some computation intensive applications, such as mobile Augmented Reality (AR) and mobile Virtual Reality (VR). To bridge the gap, we utilize the processing power of edge servers via task offloading and build practical mobile systems which significantly outperforms state-of-the-art mobile systems in terms of latency, scalability, quality of experience (QoE), and many other aspects.

In the case of mobile Augmented Reality, large-scale object recognition is an essential but time consuming task....[ Read more ]

Smartphone is now a necessity of people’s daily life, and we are enjoying various services on it with numerous mobile applications. However, the resource and communication limitations of a single mobile device make it insufficient in satisfying the real-time and interactive constraints of some computation intensive applications, such as mobile Augmented Reality (AR) and mobile Virtual Reality (VR). To bridge the gap, we utilize the processing power of edge servers via task offloading and build practical mobile systems which significantly outperforms state-of-the-art mobile systems in terms of latency, scalability, quality of experience (QoE), and many other aspects.

In the case of mobile Augmented Reality, large-scale object recognition is an essential but time consuming task. To offload the object recognition task and enhance the system performance, we explore how the GPU and the multi-core architecture on the edge servers would accelerate the large-scale object recognition process. With the carefully designed offloading pipeline and edge acceleration, we are able to finish the whole AR pipeline within one camera frame interval while maintaining high recognition accuracy.

Despite the performance concerns on mobile devices, edge servers are also faced with scalability issue, as too many concurrent offloading requests would exhaust the processing capacity of the edge and result in delayed execution with tasks queuing on the edge. To address this issue, we propose a two-tier hybrid architecture for mobile AR applications: 1) when there is enough processing capacity remaining on the edge, the mobile AR client would offload the recognition task to the edge to achieve the best performance, and 2) when the edge is heavily occupied, the mobile AR client would derive its own result based on the previous recognition result shared from nearby device’s cache, which is made possible by the fact that users in vicinity have a high chance of querying the same physical objects. This two-tier hybrid architecture not only guarantees the system performance when serving massive concurrent users, but also enables innovative features such as multi-player AR.

In the case of mobile Virtual Reality, existing 360° video streaming systems are suffering from insufficient pixel density, and the video resolution falling within the user’s field of view (FoV) is relatively low. We utilize the edge server for ultra-high resolution video transcoding and implement a system which streams tile-based viewport adaptive 360° videos onto the mobile client. With this edge proxy, we successfully achieve 16K 360° video streaming onto off-the-shelf smartphones, achieving high frame quality and fluent playback without overwhelming the processing capacity of smartphones.