I had a couple of hours this afternoon to be able to make some progress learning how to use Oculus Rift and the Leap Motion controller together.
Its not hard when you know how and in about 30 mins I connected an Oculus camera to one of the flying Leap Motion samples. Orientation of palms used to steer ship.
I’m just getting a feel for what works and how to program these technologies. I’m not sure we can use the Oculus in our games for rehab work (certainly in games with movement of the camera) due to vertigo and similar issues, but its really interesting and fun to work with.
I love this game, I bought it for the PS2 around 2002 and again for the PS3 (for the better graphics) in 2012. I’ve referenced it in my teaching often, but embarrassingly I hadn’t actually got round to finishing it. Ico is still playable after all this time, it has a perfect balance between puzzle solving, exploration, story and action. It ended quite abruptly but I enjoyed the end, particularly the bit where you get to play a little longer after the credits! I think a proper sequel could be fantastic and I think it could make for a nice indie-style movie, so long as they kept it about the three main characters.
I was sparked into action by Edge magazine’s readers games of a lifetime (see below). I have all of these games in the house (except Golden Eye). I agree that they are all good games and I would like to try and finish a few more before the end of the year – I hate unfinished games! I’ve already completed Portal, Bioshock (except the last boss!), Ico, The last of us, Halo, and RE4. A few like Dark Souls I’ll never complete and I’ve had enough of World of Warcraft.
20 – World of Warcraft
19 – Marjoa’s Mask
18 – Vice City
17 – Chrono Trigger
16 – Portal
15 – Red Dead Redemption
14 – Bioshock
13 – Ico
12 – The Last of Us
11- Dark Souls
10 – Super Mario Galaxy
9 – Metal Gear Solid
8 – Shadow of the Colossus
7 – Goldeneye
6 – Halo
5 – RE4
4 – Final Fantasy VII
3 – Half Life 2
2 – Super Mario 64
1 – Ocarina of Time
Over the last couple of months I have been working with a development version of the Leap Motion camera, in collaboration with SilverFish Studios, to build VR prototypes for physical therapy focused on finger and hand exercises. We set out to create simulations of actual clinical exercises that we could demonstrate to physiotherapists and occupational therapists to obtain feedback on the potential use of the Leap camera in the home. Three prototypes were constructed and are shown in the video below. The virtual tasks comprised: Cotton Balls (lift virtual objects and place them in bowl), Stacking Blocks (stacking virtual blocks), and the Nine Hole Peg Test (lifting and moving of nine virtual pegs).
We decided to use a fairly high level (easy to use) game development tool that facilitated quick prototyping, had a research/indie friendly licence, and had plugins for both Leap and Kinect. Shiva3D was the engine chosen and we were able to create both Kinect and Leap interactive prototypes very quickly – its a shame that the Stonetrip, the company behind Shiva3D, are now in liquidation but Unity3D is another excellent engine for this type of work (despite the limitation of requiring the pro license for plugins). A good game engine provides many useful tools for building effective 3D interactions such as built in lighting and physics. The simulation of Jenga that we built provides a very good illustration of what can be constructed in a few days:
We were very happy with the trials and the response from clinicians at the Brain Injury Unit in Musgrave Hopspital, Belfast, and we hope to present the details and results of the trials at the ITAG conference this year. Physiotherapists and occupational therapists at Musgrave already use game based therapies quite a lot and were very positive about the potential use of Leap based virtual therapies for home use and particularly for younger patients. Clinicians provided a lot of very specific feedback on particular finger/hand/arm motion that they required including exercises that encourage patients to extend out their arm and open out their hand at the same time – like they were pushing someone away. We have already started on a gameplay mechanic that can include this motion and the video below shows a very early version:
The grab gesture is also very easy to implement with Leap and the following video demonstrates a simple shelf stacking task that could be the basis for a rehab game:
SilverFish Studios is an independent software company based within Northern Ireland. SilverFish will have been operating for 2 years come June – no mean feat in the current financial climate. They have had many ups and downs but SilverFish provides a fun working environment and the team are highly motivated, talented and supportive of each other. They have been involved in designing and making several games (one due for release soon), been teaching game development for local schools, and have been engaged in quite a lot of web development and consultancy – typically they enjoy taking on jobs that require aspects of problem solving and innovation.
They work hard and no more so in building Fly Cast Master which is an iOS app designed to help (beginner and experienced) anglers improve their technique by analysing their casting motion (while holding the phone). Fly Cast Master was a complex App. to make due to the design and construction of robust motion detection algorithms and the necessary collaboration with experts from the angling domain for data collection, testing, and in the provision of suitable training materials – trainers, equipment suppliers, anglers, and collaborators at Irish Angler magazine.
My wife, Therese Charles (or @bionic_rez), established SilverFish with the support of her father Brian McGinnis who was her greatest fan. He would be incredibly proud of what she has achieved with the company and particularly in achieving the release of Fly Cast Master. I’m involved with SilverFish in an advisory role – mainly over a working lunch or after hours – but it has been very educational and enjoyable for me to be even superficially involved in helping a software product reach the App. store. I’ve learned about the process and this will help me in my future teaching, research and consultancy.
With the support of the local and international angling community and industry there are a number of ways that Fly Cast Master can be taken forward, and SilverFish have a number of plans in place already. So I hope that you will download and have a go at Fly Cast Master – see if you can beat the top score! One of the great things about Fly Cast Master is that you can still practise your casting even if you can’t get out! If you are enjoying the App. or have any constructive feedback let SilverFish know – they are passionate about making the angling trainer as educational and fun as possible.
TITLE: Facilitating Player Interaction in a Dynamic Storytelling Environment
AUTHORS: Richard Paul, Darryl Charles, Michael McNeill and David McSherry
Abstract. Facilitating player interaction with stories generated using artificial intelligence planning techniques is an important challenge to be addressed in the development of interactive computer game worlds. The problem that we focus on within this paper is the loss of story context that occurs when plan steps are reduced to primitive actions which can be executed in the game world, making it difficult for players to understand their purpose. We propose a solution to this problem based on a mechanism for dual representation of story plans at the levels of abstraction required for meaningful player interaction and to enable plan steps to be executed in the game world.
Recently we have had three journal papers accepted for publication on the basis of collaborative research that has gone on over a several years. I think this sort of research is crucial, and very worthwhile, though the government’s REF research quality measurements do not really encourage this type of work (due to measurements being explicitly within discipline based units of assessment).
Collaborative journal papers are often years in the making – time to establish a group of researchers, months to conduct the research, more months to write the paper, and generally many months to have the paper reviewed, amended and then hopefully obtain final acceptance. There is then some time before the paper actual appears as a publication.
If you are not familiar with the process you will understand why we were delighted to have the following three journal papers accepted on the back of years of work:
