Deeply immersive 3D experiences – not on screens but in special rooms called CAVES –have demonstrated very effective in teaching in science and mathematics for deaf students. The Smile project developing an immersive virtual learning environment in which deaf and hearing children (age 5-10) interact with fantasy 3D avatars and objects to learn maths and science concepts, and relative American Sign Language (ASL) terminology.The applications are displayed in stationary VR projection systems such as the FLEX from Mechadyne, and can be interacted with using a pair of sensor (pinch) gloves, or a 6 degrees-of-freedom (dof) wand, coupled with a wrist tracker.

One of the main applications has been mathsigner and the overall project is called SMILE

Nicoletta Adamo-Villani, assistant professor in Purdue’s Department of Computer Graphics Technology, who teaches 2D and 3D animation and specializes in character animation and design has worked for three years with Ronnie Wilbur, professor and chair of linguistics in Purdue’s Department of Speech, Language and Hearing Science.

Adamo-Villani reports “Research shows that immersive games are more effective than non-immersive games,” says Adamo-Villani. Studies have found that humans process visual information 60,000 times faster than text-based information, she reports, noting that students can raise their math scores 16% after eight weeks using an immersive VR program.

The game features four shops where students learn a different maths concept: a bakery where they learn about weight, a clock shop where they learn to tell time, a toy shop where they learn how to count money and a sweet shop where they learn addition and subtraction. The avatar and the child can communicate- so that the child and avatar shop keeper can count how many sweets have been bought in ASL and work out how many more the child can have.

Motion-capture is used to record the signs, so the avatars — characters such as rabbits, robots and pigs that serve as the storekeepers — can communicate with the students in real time.

The application is being refined to add more gaming elements. The researchers are looking at ways to link the shops so that a complete narrative can be developed so the child can move from one to another as he or she completes a task, making the action more like a videogame.

See also:- Adamo-Villani, N., Carpenter, E., Arns, L. (2006). An immersive virtual environment for learning sign language mathematics. ACM proceedings of Siggraph 2006- Educators Program, Boston.

Trackback

This entry was posted on Thursday, September 27th, 2007 at 9:07 am and is filed under Flux. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.