We developed a projector-camera system which visually tracks flying balls in real-time and projects images to each balls also in real-time. It is relatively easy to track balls in real-time. However, it is difficult to project images to the flying balls because of the latency of the projection system. We introduced a prediction model using a Kalman filter and achieved stable projection to the flying balls. The system composed of a motion capture system with 6 cameras (250 fps) to detect 3D position of the balls. We applied our system to juggling performance which is called "Digital Juggling."
Hiroaki Yamaguchi and Hideki Koike. LumoSpheres: Real-Time Tracking of Flying Objects and Image Projection for a Volumetric Display, 6th Augmented Human International Conference (AH'15), Singapore, ACM, pp.93-96, 2015. (Best Short Paper) (PDF)
Hiroaki Yamaguchi and Hideki Koike. LumoSpheres: Real-Time Tracking of Flying Objects and Image Projection, Proc. of the 21st Workshop on Interactive Systems and Software (WISS 2013), 2013 (PDF)