Researchers from Cornell
University and the University of Wisconsin, Madison, have designed a
wrist-mounted device that continuously tracks the entire human hand in 3D. The
bracelet, called FingerTrak, can sense and translate into 3D the many positions
of the human hand, including 20 finger joint positions, using three or four
miniature, low-resolution thermal cameras that read contours on the wrist. The
device could be used in sign language translation, virtual reality, mobile
health, human-robot interaction and other areas. Past wrist-mounted cameras
have been considered too bulky and obtrusive for everyday use, and most could
reconstruct only a few discrete hand gestures.
FingerTrak's breakthrough is a
lightweight bracelet, allowing for free movement. Instead of using cameras to
directly capture the position of the fingers, the focus of most prior research,
FingerTrak uses a combination of thermal imaging and machine learning to
virtually reconstruct the hand. The bracelet's four miniature, thermal cameras
(each about the size of a pea) snap multiple silhouette images to form an
outline of the hand. A deep neural network then stitches these silhouette
images together and reconstructs the virtual hand in 3D. FingerTrak could also
have an impact on health care applications, specifically in monitoring
disorders that affect fine-motor skills.
More information: