3D Virtual Building Block System Works Without Markers or Sensors
A research group at Tsukuba University is developing a virtual building block system. It works by recognizing hand and finger movements with a camera, and outputting 3D shape data.
"This system recognizes the movements of a person's hand and fingers, or hand and arm, using a single camera. Based on those movements, it manipulates virtual building blocks. We've used two cameras here, to make sure that even more minute gestures are recognized."
A unique thing about this system is, it doesn't use markers or sensors on the hand.
"To implement this feature, we've uploaded a huge database in advance. The database contains information about 3D hand shapes. We've also provided 2D data on "how hands look." The system finds the 2D view that looks most like the rapidly changing picture from the camera, and outputs 3D data based on that."
As well as enabling the use of virtual building blocks, this system can also remotely control a robot, making it move like the user. There's no need to precisely determine the camera position. The system can be operated just by placing the camera approximately where the hand is most visible. A supplementary camera can also be used as long as it's placed perpendicular to the first one.
"What our group is really focusing on is operating PCs using gestures. In other words, now that 3D TV has come into the home, we don't want to settle for just watching 3D pictures. We also want to enable 3D operation of PCs. When you use a PC in 3D, the 3D icons will appear to float in midair. By manipulating them through gestures, you could open and delete files, or enlarge and reduce things. We're also thinking of a system that enables PCs to be used through movements such as turning pages."
The researchers plan to develop a practical system for reading 3D e-books using gestures within six months to a year.