Student Research and Virtual Reality Education Module

The MOCORE Lab is proud to have a community educational program centered around the development of virtual reality (VR) platforms for improving physical and cognitive rehabilitation. The sections below present previous and current research projects from local New Jersey high school students. Additionally, the MOCORE Lab is developing an Education Module to teach the basics of creating environments with the Unity VR engine.


Program Coordinators: Sophie Dewil M.Eng., Yu Shi M.S., and Zachary Marvin M.Eng.


Download a PDF of the MOCORE Virtual Reality Education Module
Last Updated May 1st, 2021

Isha Bhatia – VR Daily Activities Game for TBI Rehabilitation


The Virtual Reality Daily Activities Game targets rehabilitation for patients with Traumatic Brain Injury. It is programmed to only sense one limb, motivating the user to use their affected limb. The game incorporates Activities of Daily Living, such as pouring drinks, organizing objects, and doing laundry. The first level is a simulation of pouring water (represented by pearls) from a water bottle into a cup. This level targets physical rehabilitation, requiring the user to tilt their hands and aim into the cup. It includes features that allow the user to reset the objects to their initial position if needed, since it gives feedback depending on if the user passed the level or not. The second level is a simulation of organizing different colored objects on different shelves. This level targets physical and cognitive rehabilitation, as it requires the user to remember the directions regarding which colored objects should be on which shelf. It also has features that allow the user to reset the objects into their initial positions. After three failed attempts, the user will be given the option to view the instructions for 10 seconds, since the game is also partially dependent on memory. The third level is a simulation of two laundry loads, one with light clothes and the other with dark clothes. This level targets physical and cognitive rehabilitation, acting as a guided tutorial to teach the user how to properly wait for each step to be complete. The user must wait in between each step, similar to how people have to wait for their laundry to finish. This level is the most complex, requiring the user to follow the steps accurately to mimic the action in real life.





Kevin Chen – VR Clock-Reading Game for Post-Stroke Rehabilitation


The Virtual Reality Clock-Reading application aims to facilitate motor and cognitive post-stroke rehabilitation by immersing stroke patients in an interactive environment centered around an analog clock design. Stroke survivors often grapple with both motor impairments, such as weakness and lack of coordination, and cognitive challenges, encompassing memory and thinking deficiencies (Lugtmeijer et al. 2020). This VR game is designed to address this dual-aspect rehabilitation need. Firstly, it assesses a patient's motor skills by requiring them to manipulate a clock hand to a specified position (Level 1) and a slider (Levels 2 and 3). On the cognitive front, it evaluates working memory, an area where deficits persist in stroke survivors, by asking patients to remember a digital time and then translate it into analog format and vice versa. Additionally, it tests visuospatial processing, essential for orienting in space and tracking the movement of objects. The VR application offers a comprehensive and engaging rehabilitation experience for stroke survivors, addressing the complex interplay of motor and cognitive functions crucial for their recovery.







Lab Presentation

Udhirna Krishnamurthy - Connect-the-Dots for Parkinson's rehabilitation


The Virtual Reality Connect-the-Dots game is designed to focus on Parkinson’s rehabilitation. Within the game, users are prompted on the screen to connect a specified number of dots in either ascending or descending order. With each progressive level, the game presents users with more challenging configurations, including a greater number of dots and varying order patterns. This game offers a holistic approach to Parkinson's therapy, as it targets both motor and cognitive rehabilitation. The act of drawing to connect the dots serves as an effective method for targeting motor rehabilitation, while the cognitive challenges involved in interpreting the dot's numerical sequences and connecting them in the precise order cater to crucial cognitive functions such as task sequencing, memory, and executive functions. Studies reveal that incorporating such cognitive rehabilitation significantly enhances brain connectivity in Parkinson's patients, presenting a promising avenue for comprehensive treatment (Díez-Cirarda et al., 2016).





Khush Parikh


Applications: This program will prospectively be used in classrooms by school districts to gain a better understanding of individual student needs, especially those with auditory learning impairments. With student reports at hand, educators can make informed decisions about how to limit distractions in their classroom or shift the student to a more conducive learning space.

After the pandemic, cognitive comprehension, especially at a primary to tertiary academic level, has been impacted by virtual learning in middle school students. In order to collect personalized data, I created a virtual reality(VR) diagnostic to gauge comprehension amongst the target population. Subjects will use virtual reality goggles to place themselves in a classroom environment where a lecture is taking place. Concurrently, students will be given a simple and understandable two minute lecture on historical events that they have previously learned about in school. While the lecture is playing, common audio distractors, such as papers rustling, will be played in the background at different volume levels. This volume level is pre-determined using field literature regarding signal-to-noise ratios in learning environments. At the end of the lecture, students will be asked questions which correspond with the periods where the signal-to-noise ratio changes. The questions will be modeled upon Zeamer & Tree (2013) which posed both critical thinking and direct recall ability questions and drew conclusions about comprehension thresholds through them.

Future research entails using an EEG to map electrical activity in the brain and draw a correlation to attention span and information processing.


Example of Direct Recall Question in the Immersive Virtual Reality Setting

Example of a Critical Thinking Question in the Immersive Virtual Reality Setting

Kwabena Boateng - Sense of Agency and Movement Control


The Sense of Agency and Movement Control Training Environment was designed to offer users a method to improve their sense of agency and movement control through several gamified training tasks. The sense of agency auditory and visual task were designed to function similarly to a "hit the target" game. It implements Haggard et al. (2002)'s intentional binding method which utilizes time perception and sensory feedback to measure agency in a user. The movement control task was designed to function similarly to a "dodge the blocks" game. It utilizes positional and speed delays to obseve a user's perception of their agency in relation to their actual performance. All training tasks were developed in the game engine Unity® and use several additional programs (Blender®, Gravit®, and Visual Studio®) for assets.

Download a PDF Tutorial Document



Sense of Agency Task: Beginner

Sense of Agency Task: Advanced

Movement Coordination Task



Advanced Visual Feedback Sense of Agency Task trial



Beginner Auditory Feedback Sense of Agency Task trial



MOCORE lab motion capture integration



MOCORE lab motion capture integration

Aqsa Owais - Fruit Ninja


The Fruit Ninja Upper-Extremity Rehabilitation application was designed to help post-stroke patients recover functionality in their arms. In the current version, patients use computer mouse input to slice the fruits that appear on the screen in half. In comparison to the original Fruit Ninja game, this application prompts the patient to slice the fruit in a predetermined direction. This was implemented to facilitate increased cognitive engagement. Systematic reviews have indicated that interventions that enable active participation lead to increased engagement and motivation which leads to increased effectiveness (Brett et al, 2017). The control group and experimental group will receive nearly identical games; however, the experimental group will have an added feature. This added feature will allow participants in the experimental group to choose certain prizes, such as different color blades and backgrounds, as they earn more points. Ultimately, this research aims to study the effects on performance and cognitive engagement if players are allowed to choose their own prizes.

Download a Poster Presentation

Annotated image of the participants display during the experiment

Home Screen: Play, How to Play, Quit



Task Trial: Participant playing the complex version of the game with designated directions to slice


Ashita Birla - VR Maze Game for Neurological Rehabilitation Therapy


The Virtual Reality Maze Game is designed to focus on working memory rehabilitation for patients with neurological disorders/injuries/deficits. I will concentrate towards the visuospatial sketchpad from the central executive function of the brain as the patients will react to a visual environment with different perspectives to test retention, and whether different perspectives will help or hinder the visual retention. The 10-level (increasing difficulty) game will be played by a control and experimental group. For each of their training periods, they would navigate through a 3D maze without (control) / with (experimental) a preview beforehand and attempt to reach the end point of the maze. In the collective final task trial, both groups would navigate without any preview, but with different fixed start and end points. By comparing the completion times of both groups in the final game, we could test the visuospatial retention of each to see if the preview helped or hindered as well as create newer levels to induce/test improvement in working memory capacity.

Download a Poster Presentation

Player navigational view

Bird's eye view of the maze



Control group: Same level, no preview



Experimental group: Same level, with preview (10-seconds)



Final task: Same level, same circumstances, changed location + destination

Matt Neissen - Language Skill Recovery


The Motor Control and Language Skill Recovery Application will serve to aid post-stroke rehabilitation of aphasia patients. Patients use a computer mouse to facilitate the transfer of an object across the computer interface. In doing so, patients exercise fine motor skills and cross the midline of their field of vision, addressing two common symptoms of aphasia patients. The application employs bandwidth feedback to visually alert the patient if the object strays from the intended path. Between rounds, patients will complete a language rehabilitation exercise designed to improve word association ability, vocabulary recall, and language-based memory. As reported by Anderlini et al. (2019) in a meta-analysis of aphasia rehabilitation studies, research is scarce on the promising correlation between language and motor recovery. If expanded to a clinical study, this application could assist in furthering this area of research.

Download a PDF Tutorial Document

Participant display of a simple task, with no obstructions

Participant display of a complex task, with added obstacles

Alumni



Raj Shah - Beat Saber

The Beat Saber Rehabilitation game (Beat Saber R), modeled after the popular VR game, Beat Saber, serves to aid in the rehabilitation of spinal cord injury (SCI) patients.

Mayur Srinivas - Bandwidth Tracing Task

The Motor Control Rehabilitation Environment is a virtual reality simulation that serves to allow patients with neuromuscular disorders to increase control of his/her arm.


Joseph Gisoldi, Micah Rubin, and Liam Sittig - Hand Tremor Rehabilitation

The Hand Tremor Rehabilitation Game is a virtual reality program in which the user completes common rehabilitation tasks that are used to treat hand tremors, such as weighted or resistance exercises, in a gamified environment.


Johanna Gomez - Memory and Tracing

The Memory and Tracing Task is an application designed to put the user through a series of related tests in a gamified environment tp screen for cognitive illness.

Jeeva Murugan - Cognitive Agency Rehabilitation

This application was built as a virtual gamified enviornment, which allows patients to play through various levels while training their sense of agency.