Identification of objects in a real world plays a key role for human-computer interaction in a computer-augmented environment using augmented reality techniques. For providing natural interaction in such environments, it is necessary for an interface system to know which objects a user is using. In the previously developed interface systems, real objects are identified by using specially designed tags attached to objects. In this work, we propose a new method for interactive object recognition and registration for more natural and intuitive interaction without using any tags. In particular, we introduce interactive object registration and recognition by combining direct manipulation with user's hands and a color-based object recognition algorithm.