Emotion VR

Project Overview

Summary:

In collaboration with Hope Services, a local non-profit organization providing support and resources for adults with developmental disabilities, I designed, developed and evaluated a VR game to help clients with recognizing emotions through collective emotional cues.

Problem statement:

Individuals with developmental disabilities face challenges in identifying emotions through emotional cues, such as facial features, vocal intonations and body language.

Being able to recognize emotions is a cruicial part of developing relationships with others, often playing a key role in maintaining friendships, romantic relationships and professional relationships.

VR can provide a safe, controlled and realistic method of social skills training where individuals with developmental disabilities can practice recognizing and responding to emotional cues.

Team:

I led a 5 person team including a program manager, psychologist, and a 3 person research and development team.

Research Methods and Process

Role:

VR Researcher

Methods and Tools:

  • Usability testing

  • Contextual interviews

  • Prototyping

  • User testing

  • Unity

Usability Testing

Usability testing of the HTC Vive VR headset was conducted with adults with developmental disabilities to understand their comfort with using the headset and its controllers.

During testing, participants were asked to play a simple game involving gross motor movements and use of the trigger button on the right-hand controller, and after answered questions from the System Usability Scale (SUS) along with qualitative follow-up questions to determine how usable the VR headset was.

Contextual Interviews

To further understand existing practices of emotion recognition training for adults with developmental disabilities, I conducted a contextual interview with a psychologist working with Hope Services’s clients.

During the initial one-on-one interview, the psychologist gave an overview of her practice, the skill levels of her clients and the tools she used during sessions, which included paper worksheets and iPad games.

An observation of a session was then conducted, demonstrating how the tools were used during the session with a client.

After a discussion on how the VR game might be designed to be integrated into existing session structures ensued.

Prototyping

High-fidelity prototypes of the game were created using the Unity game engine for VR game development.

Prototypes were designed based on data collected from the contextual interview with the psychologist, and took into account the data collected from the usability test regarding designs for controls and UI.

User Testing

Subjects

  • Adults ages 18+ with a diagnosed developmental disability, including autism spectrum disorder, down syndrome, cerebral palsy, etc.

Procedure

Baseline emotion recognition data was collected by the psychologist using a modified Emotion Recognition Task (ERT) containing photos and videos to determine subjects’ current skill levels.

There were 2 iterations of user testing conducted.

During the 1st iteration, qualitative feedback and emotion recognition scores were collected after subjects played the game to determine what improvements could be made to the game to promote effective emotion recognition training.

The 2nd iteration of user testing was conducted after feedback and data from the 1st iteration of user testing was incorprated into the game through content improvements, and data was collected after subject gameplay to understand whether the game would be feasible as an emotion recognition training tool.

Measures

Results

The SUS found that the usability of the HTC Vive was above average for adults with developmental disabilities.

The baseline ERT data found that subjects were able to correctly identify emotion through emotional cues such as facial expression and body posture 62% of the time.

During iteration 1 of user testing, it was found that subjects were able to correctly identify emotions 49% of the time, which suggested that there was an issue with the game’s content or UI.

Data from the user game experience questionnaire suggested that participants had difficulty viewing the faces of characters in the game, with 67% of participants rating their ability to see character facial expressions low.

The new version of the game incorporated feedback and data from iteration 1 of user testing, which resulted in the development of an additional module providing reinforcement for concepts learned during the game.

Iteration 2 of user testing found that subjects were able to correctly identify emotions 62% of the time, matching baseline measurements.

Feedback from subjects participating in iteration 2 of user testing enjoyed using the game, with 81% stating they would want to play the game again.

Insights and Takeaways

Data from iteration 2 of user testing found that:

  • Subjects were able to correctly identify emotions in the VR game 62% of the time, matching baseline data collected with non-VR measures

  • The usability of the HTC Vive was above average for adults with developmental disabilities

  • 81% of subjects stated that they would want to play the game again during emotion recognition training sessions

These findings suggest that a VR game developed for adults with developmental disabilities may be feasible in providing emotion recognition training.

Issues identified with VR social skills training for this population included realism of characters and interactivity of game content.

I published and presented research findings for this study at UCSC Grad Slam 2018, ICDVRAT 2018 and SMARTCOMP 2018.