Purdue University Employs Virtual Reality to Teach Deaf Children Mathematics

December 25, 2006

Purdue University is using technology employed in the films "King Kong" and "Lord of the Rings" to create colorful characters in a virtual reality world who teach deaf children mathematics.

Computer graphics technology students are working in Purdue's Envision Center for Data Perceptualization with high-tech cameras and "cybergloves" that can translate body and hand motions into digital images. The resulting cartoon rabbit, robot, and pig use sign language in fun, interactive environments projected on the walls and floor of a "cave" of screens that surrounds the deaf students.

The students wear lightweight stereoscopic glasses so the virtual reality images appear three-dimensional. A device monitors the student's head position so that the environment is consistently redrawn to match the uesr's perspective. A wrist tracker and telemetric "pinch gloves" monitor the student's hand and finger movements allowing interaction with the virtual environment and prompting responses from the characters.

The project's supervising professor, Nicoletta Adamo-Villani, said virtual reality helps break down some of the barriers deaf children experience. "Environments are more stimulating when students are able to interact with the subject and travel through the scenes," said Adamo-Villani, assistant professor of computer graphics. "Hands-on experiences equate to a better understanding of mathematical concepts in real-world situations."

Adamo-Villani reported that research shows that people process visual information 60,000 times faster than textual information and that eight-week virtual reality program can improve students' mathematics scores by more than 15 percent. She said that learning enthusiasm remains even after the novelty of virtual reality fades.

The program is being designed to overcome the barriers deaf children experience in learning mathematical skills. Those obstacles include:

• Significant delay in reading comprehension
• Parents' inability to convey mathematical concepts through sign language
• Difficulty taking advantage of supplemental learning opportunities, such as television shows and dinner-table conversation

Adamo-Villani hopes that by reducing the impact of these barriers, the program can help increase the number of deaf students who go on to study science, technology, engineering, and mathematics - fields in which they are statistically underrepresented. Background research for the application revealed that deaf students historically have had difficulty gaining entry into higher education that leads to these careers.

The university's virtual reality program is designed to provide disabled, early elementary school age students with a number of active, individualized learning conditions:

• The ability to control their environment
• The ability to engage in learning activities at their own pace
• The ability to repeat activities as needed
• The ability to see or feel items or processes in concrete terms
• The ability to practice daily living tasks in a safe and barrier-free environment
• Motivation to succeed

Because research shows that children prefer bright colors, student programmers are working to ensure that the program is as colorful as possible. Also, certain colors increase alpha waves directly linked to awareness and improve students' attention span and elicit emotional responses. In addition, "Fluid, non-mechanical motion is fundamental to learning sign language effectively," said Edward Carpenter, the graduate research assistant working directly with undergraduate programmers, "That's why we have invested our efforts in developing natural gestures that are appealing to children.

Undergraduate David D. Jones never expected to be mastering complex skills such as environment modeling, character modeling, and rigging motion-capture data application and programming interaction in 3-D space so early in his academic career. He said he is excited that his work will be utilized for years to come to help others learn - a prospect that he said far outshines any class project grade he will receive.

"I truly hope that this program plants some of the seeds necessary to develop good math skills in those children," said Jones, who continues his computer graphics training as a graduate student at Purdue fall semester. "Who knows, maybe someday they'll be as fortunate as I was and get to apply those skills to something interesting and worthwhile."

Gifted children attending academic camps at Purdue came to the Envision Center last summer to test the system and provide feedback. In the fall, a portable version of the virtual reality application will be taken to Indianapolis and introduced to classrooms at the Indiana School for the Deaf.

Adamo-Villani credits the university's investment in information technology and its emphasis on internal and external collaboration for making the new instructional tool possible. She and her students from the Department of Computer Graphics Technology have been working with the Envision Center for Data Perceptualization; Information Technology at Purdue; the College of Liberal Arts' Department of Speech, Language, and Hearing Sciences; and the Indiana School for the Deaf to develop the virtual reality program.

Source: nadamov@purdue.edu

Related Web sites: Envision Center for Data Perceptualization: http://www.itap.purdue.edu/envision

College of Technology: http://www.tech.purdue.edu

Computer Graphics Technology: http://www.tech.purdue.edu/cgt/

Information Technology at Purdue: http://www.itap.purdue.edu

33rd International Conference on Computer Graphics and Interactive Techniques: http://www.siggraph.org/s2006/