MonoEye: Monocular Fisheye Camera-based 3D Human Pose Estimation

Wearable cameras have the potential to be used in various ways in combination with egocentric views such as action recognition, gesture input method for augmented/virtual reality (AR/VR) as well as lifelogger. Particularly, the pose of the camera wearer is one of the interesting factors of the egocentric view and various eccentric view-based pose estimation systems have been proposed; however, there is no balance between recognizable poses and enough egocentric views. In this work, we propose MonoEye, a system to provide wearer's estimated 3D pose and wide egocentric view. Our system's chest-mounted camera, equipped with the ultra-wide fisheye lens, covers the wearer's limbs and wide egocentric view; our pose estimation network estimates 3D body pose of the wearer from the camera's egocentric view. The proposed system not only can be used as an input interface of AR and VR through estimation of a various pose of the wearer but also has a potential to be used for action recognition by providing a wide egocentric view.

Papers:

• Dong-Hyun Hwang, Kohei Aso, Hideki Koike: MonoEye: Monocular Fisheye Camera based 3D Human Pose Estimation, IEEE VR 2019, pp.988-989, 2019.
• Dong-Hyun Hwang, Kohei Aso, Hideki Koike: Toward human motion capturing with an ultra-wide fisheye camera on the chest, The First IEEE VR Workshop on Human Augmentation and Its Applications (HAA2019), co-located with IEEE VR 2019, pp.1524-1526, 2019.
• Kohei Aso, Dong-Hyun Hwang, Hideki Koike: Generating Synthetic Humans for Learning 3D Pose Estimation, The First IEEE VR Workshop on Human Augmentation and Its Applications (HAA2019), co-located with IEEE VR 2019, pp.1519-1520, 2019.