Leap Motion Depth Sensing Camera for Physical Therapy #leapmotion

Over the last couple of months I have been working with a development version of the Leap Motion camera, in collaboration with SilverFish Studios, to build VR prototypes for physical therapy focused on finger and hand exercises. We set out to create simulations of actual clinical exercises that we could demonstrate to physiotherapists and occupational therapists to obtain feedback on the potential use of the Leap camera in the home. Three prototypes were constructed and are shown in the video below. The virtual tasks comprised: Cotton Balls (lift virtual objects and place them in bowl), Stacking Blocks (stacking virtual blocks), and the Nine Hole Peg Test (lifting and moving of nine virtual pegs).

We decided to use a fairly high level (easy to use) game development tool that facilitated quick prototyping, had a research/indie friendly licence, and had plugins for both Leap and Kinect. Shiva3D was the engine chosen and we were able to create both Kinect and Leap interactive prototypes very quickly – its a shame that the Stonetrip, the company behind Shiva3D, are now in liquidation but Unity3D is another excellent engine for this type of work (despite the limitation of requiring the pro license for plugins). A good game engine provides many useful tools for building effective 3D interactions such as built in lighting and physics. The simulation of Jenga that we built provides a very good illustration of what can be constructed in a few days:

We were very happy with the trials and the response from clinicians at the Brain Injury Unit in Musgrave Hopspital, Belfast, and we hope to present the details and results of the trials at the ITAG conference this year. Physiotherapists and occupational therapists at Musgrave already use game based therapies quite a lot and were very positive about the potential use of Leap based virtual therapies for home use and particularly for younger patients. Clinicians provided a lot of very specific feedback on particular finger/hand/arm motion that they required including exercises that encourage patients to extend out their arm and open out their hand at the same time – like they were pushing someone away. We have already started on a gameplay mechanic that can include this motion and the video below shows a very early version:

The grab gesture is also very easy to implement with Leap and the following video demonstrates a simple shelf stacking task that could be the basis for a rehab game: