Clay Kent sat down and rested his chin in a little brace that was at the end of a three-foot-long pole. At the other end was a computer monitor displaying a grid of sparkling green dots. Though he looked like he might have been undergoing some kind of Clockwork Orange-style brainwashing, Kent, an undergraduate in digital arts, was actually creating music.
The restraint was meant to keep his head in position for the sensitive eye-tracker technology, which used a pair of infrared cameras to monitor Kent’s eye movements. The grid he was viewing was actually a tone matrix, each dot representing a different variation of preprogrammed melodies.
As Kent moved his gaze from dot to dot, the matrix moved accordingly, and a pleasing, variegated harmony of chimes suffused the room.
“I designed this thinking a guitarist could at some point create accompanying music just by moving his eyes,” he said. “Of course, the eye tracking would have to be more sophisticated, but it could work.”
Photo: Undergraduate Clay Kent (seated) demonstrates his “eye music” program for digital arts instructor John Park (left) and computer science professor Anthony Hornof.
Kent designed his program as one of several students in associate professor Anthony Hornof’s eye tracking class, a computer and information science course that was opened to digital arts students for the first time last spring.
With eye tracking, Hornof sees the opportunity to explore the confluence of art and technology, a chance to dabble in the poetry of science. “Science and art can’t progress simply through rational thought,” he said. “There needs to be intuition, emotion and feeling that guides decisions.”
The revolutionary eye tracking technology has been around since the late 1990s but there’s recently been a surge in research and experimentation with it, Hornof said. Primarily, it’s been seen as a technology that could benefit the profoundly disabled, people whose movements and ability to communicate might be so restricted that their eyes represent one of the few active ways they can interact with their environment.
One of Hornof’s experiments was called EyeDraw, an innovative and intuitive program in which users can use their eyes to create art by placing “stamps” of animal shapes and lines on a blank computer canvas.
“Eye tracking could be used in schools for speech language pathologists or to help kids with motor impairments to better communicate,” he said. “We’re still finding new ways to use eye movements as we analyze them in real time.”
In collaboration with doctoral music student Troy Rogers, Hornof has created a more sophisticated version of Kent’s project, in which sound is also tied to a user’s eye movements. The wrinkle is that there is sometimes a conflict between a user’s desire to use his or her eyes to see something and the desire to create an appropriate sound.
And as anyone who has attempted to manipulate Kent’s tone matrix can tell you, it’s an unusual experience to consciously control your own eye movements when you’re so used to moving them instinctively.
“It’s a really interesting way to approach a work of art,” said Brendan Veale, another undergraduate digital arts student. “But it’s a strange feeling, manipulating things with your eyes.”
— Marc Dadigan
Photo: Marc Dadigan
lt's not all head-banger heavy metal on soldiers' play lists. Some prefer balladeers or even anti-war anthems from decades past.
Learn how experts across disciplines are together advancing green chemistry.