The architecture, engineering, construction and operation (AECO) industry has been widely regarded as a highly resource consuming industry. Among different stages of the AECO industry, the operations and maintenance (O&M) lasts the longest in the lifecycle of a building and incurs more than 85% of the total costs, indicating the importance of optimizing management and improving efficiency during O&M. However, it was indicated that two-thirds of the estimated cost of facility management is lost due to inefficiencies during the O&M stage. With current approaches for O&M activities, it is difficult for people to directly visualize and update information of building facilities and many facilities are hidden (e.g. ventilation ducts above ceilings and water pipes under floors). Theref...[ Read more ]

The architecture, engineering, construction and operation (AECO) industry has been widely regarded as a highly resource consuming industry. Among different stages of the AECO industry, the operations and maintenance (O&M) lasts the longest in the lifecycle of a building and incurs more than 85% of the total costs, indicating the importance of optimizing management and improving efficiency during O&M. However, it was indicated that two-thirds of the estimated cost of facility management is lost due to inefficiencies during the O&M stage. With current approaches for O&M activities, it is difficult for people to directly visualize and update information of building facilities and many facilities are hidden (e.g. ventilation ducts above ceilings and water pipes under floors). Therefore, this research aims to apply innovations to improve efficiency during the O&M stage. In recent years, professionals begin to realize the practical value of mixed reality (MR) technology, which can aid in various tasks during O&M. Through integrating virtual information with the real world, MR makes the information of users surrounding facilities readable and manipulable. However, there are two major limitations while implementing MR in O&M: (1) All existing methods for MR spatial registration have their own limitations in either accuracy or practicality. (2) There is a lack of efficient methods for data transfer from BIM to MR, which limits the functionality and complexity of MR applications. To tackle these limitations, this research develops an MR engine that can achieve accurate and robust MR spatial registration and efficient data transfer from BIM to MR.

For the development of the MR engine, an indoor localization approach is proposed for MR spatial registration. A transfer learning technique named transferable CNN-LSTM is proposed for improving the accuracy of localization and reducing Wi-Fi fingerprinting’s vulnerability to environmental dynamics. A deep learning approach that combines convolutional neural network (CNN) with long short term memory (LSTM) networks is first proposed to predict the locations of unlabeled fingerprints based on labeled fingerprints. Then the transferable CNN-LSTM model is derived from the CNN-LSTM networks based on transfer learning to improve the robustness against time and devices. The proposed transferable CNN-LSTM model is tested and compared with some conventional approaches and even some transfer learning approaches. Another part of the engine focuses on efficient mechanisms for BIM-to-MR data transfer. An ontology-based approach is proposed for transfer of semantic data. For geometric models, building components are classified into four types according to their different features and different model simplification algorithms are proposed accordingly. The algorithms were first tested with single components, and then a whole building was used to evaluate the overall performance of the developed mechanisms. As illustrated in the tests, the developed mechanisms can efficiently transfer both semantic information and geometric information of BIM models into MR applications, thus reducing the time for model transfer and improving the fluency of corresponding MR applications.

The developed MR engine is then applied to facility maintenance management (FMM) and emergency evacuation. To improve the efficiency of FMM, a BIM-based location aware MR collaborative framework is developed, with BIM as the data source, MR for interaction between users and facilities, and Wi-Fi fingerprinting for providing real-time location information. An experiment is designed to evaluate the effectiveness of the developed system framework. For emergency evacuation, a graph-based network is formed by integrating medial axis transform (MAT) with visibility graph (VG), with the addition of buffer zones. Closed-circuit television (CCTV) processing techniques are also developed to monitor the flow of people so that evacuees and can avoid congested areas. An Internet of things (IoT) sensor network is established as well to detect the presence of hazardous areas. With the constructed graph-based network, congestion analysis and environment index of each area, an optimal evacuation path can be obtained and augmented with MR devices.

This research develops an MR engine that can improve the accuracy and robustness of conventional Wi-Fi fingerprinting based MR spatial registration and efficiency of BIM-to-MR data transfer. The developed MR engine has been implemented in FMM and emergency evacuation, illustrating the potential of the proposed approaches in improving the efficiency of O&M activities.