ParaPara: Synthesizing Pseudo-2.5D Content from Monocular Videos for Mixed Reality

In this work, we propose a simple yet effective method for synthesizing a pseudo-2.5D scene from a monocular video for mixed reality (MR) content. We also propose the ParaPara system, which applies this method. Most previously proposed systems convert real-world objects into 3D graphic models using expensive equipment; this is a barrier for individuals or small groups to create MR content. ParaPara uses four points in an image and their manually estimated distances to synthesize MR content by applying deep neural networks and simple image processing techniques to monocular videos. The synthesized content can be observed through an MR head-mounted display, and spatial mapping and spatial sound are applied to support the interaction between the real-world and MR content. The proposed system is expected to reduce the entry barriers to create MR content because it can create such content from a large number of previously captured videos.

Papers:

Dong-Hyun Hwang and Hideki Koike. 2018. ParaPara: Synthesizing Pseudo-2.5D Content from Monocular Videos for Mixed Reality. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, New York, NY, USA, Paper LBW608, 6 pages. (PDF)