The advancement of artificial intelligence and wearable computing triggers radical innovation of cognitive applications. In this work, we propose VIMES, an augmented reality based memory assistance system that helps recall declarative memory, such as who you met and what you chatted about. VIMES runs on smart glasses like Google Glass, and takes first-person video as input for compiling personal profiles and event information. The results are stored in a private database and are used later for recognizing people and reminding the user with previous conversations. Concerning the resource constraints of smart glasses, the compute-intensive tasks involved in the data processing, including face recognition, speech recognition and natural language processing, are offloaded to a Bluetooth-con...[ Read more ]

The advancement of artificial intelligence and wearable computing triggers radical innovation of cognitive applications. In this work, we propose VIMES, an augmented reality based memory assistance system that helps recall declarative memory, such as who you met and what you chatted about. VIMES runs on smart glasses like Google Glass, and takes first-person video as input for compiling personal profiles and event information. The results are stored in a private database and are used later for recognizing people and reminding the user with previous conversations. Concerning the resource constraints of smart glasses, the compute-intensive tasks involved in the data processing, including face recognition, speech recognition and natural language processing, are offloaded to a Bluetooth-connected companion smartphone. According to our measurement, with computation offloading the system lowers the response delay to less than one second, and saves over 40% energy for Google Glass. Meanwhile, the system can generate personal information with over 90% accuracy in most cases. Furthermore, a user study including 50 participants has provided evidence of effectiveness for memory recall by using VIMES.