HoloLens Integration into a Multi-Kinect Tracking Environment

The Microsoft HoloLens is one of the latest headsets that facilitates mixed and augmented reality (AR) applications. It has a high potential to leverage AR applications in many domains. However; the first version also comes with several limitations such as the limited battery lifetime and the small field of view. An important one is the lack of high-fidelity depth data which would facilitate object detection and tracking research; a capability imperative for many applications. To mitigate this limitation; we integrated the HoloLens into a point cloud-based tracking system. Our system uses several Kinect range cameras to obtain a point cloud and to detect and track real assets of interest in this point cloud. The pose data for all objects are forwarded to the HoloLens; which can then render 3D models from the right perspective. However; this system is not free of tolerances and tracking errors; which mandates calibrations. This poster explains how the system was set-up and verifies the feasibility of a system such as this for 3D model registration using the HoloLens as a display device.