It’s been a busy period of research with the research exercise coming towards a conclusion and new grant opportunities appearing. Barry Herbert is in the last stages of writing up his PhD but we managed to get a paper out the door and accepted for ITAG. This paper presents results of an experiment which investigates user typology for the gamification of a virtual learning environment. We show that it is possible to distinguish between variation in the ways that learners can be motivated to engage within learning processes and content.
- Barry Herbert, Darryl Charles, Adrian Moore, Therese Charles, “An Investigation of Gamification Typologies for Enhancing Learner Motivation”, ITAG: Interactive Technologies and Games – Education, Health and Disability, Nottingham Trent University, Nottingham, 2014
Extending these ideas and applying them within a rehabilitation context we developed a new framework called PACT (people, aesthetics, context, technology) and we are looking forward to presenting a short paper on this at ICDVRAT. PACT has an implicit focus on participatory design and involvement with all of the relevant stakeholders from the beginning of a rehabilitation design process. The emphasis on gamification within the PACT framework has a number of significant advances. Firstly, the outcome of a gamification process may not be an obvious game but may simply result in the addition of fun feedback (e.g. points and badges) to a non-game context (e.g. physical movements round the home), could recommend the use of gaming hardware in a non-game context (e.g. digital painting), or the use of game worlds to immerse and inspire (e.g. walks with friends in virtual game worlds). Secondly, new advanced gamification approaches can help tailor system design to account for diversity in motivation between different people.
- Darryl Charles, Suzanne McDonough, “A Participatory Design Framework for the Gamification of Rehabilitation Systems”, ICDVRAT: The 10th International Conference on Disability, Virtual Reality and Associated Technologies 2014.
Craig Hull is just about to start his second year of his PhD and he recently presented his first paper (below). I’m particularly pleased that we received generally positive feedback on this approach and that we managed to get the word FRAGED into the title – Babylon 5 :-). We should see if we can get “smeg” and “frak” into future paper titles.
- Craig Hull, Darryl Charles, Philip Morrow, Gerard Parr, “FRAGED: A Framework for Adaptive Game Execution and Delivery to Improve the Quality of Experience in Network Aware Games”, PGNET 2014.
We also just received news that our research with the Leap Motion controller for use in stroke rehabilitation was accepted for publication in Jatech. This technology has a lot of potential and we need to try to find some funding to be able to take it forward.
- Charles, D.K., Pedlow, K., McDonough, S., Shek, K. & Charles, T. (2014). Close Range Depth Sensing Cameras for Virtual Reality based Hand Rehabilitation. Journal of Assistive Technologies
In September Dominic Holmes is starting a PhD with us at Ulster to investigate the use of modern game hardware and software to create tailored and motivating exercise programmes to help prevent falls. He will focus on adherence to exercise as a measurement of success and investigate variation in engagement mechanisms between users. The fun part will be a chance to work with Oculus Rift VR (or similar) , Leap/Kinect/Myo controllers and the Omni-Treadmill. Dominic has been shortlisted for the Creative Buzz Award for his excellent work on his final year project. He designed and made a feedback jacket for connecting to a computer game via an Arduino microcontroller – it helps provide a more immersive gaming experience by supplying haptic feedback around a player’s body, e.g. when shot a player will feel the motion of an actuator in the area of the body that correlates to where they have been shot in-game. We hope that he will be able to use some of these approaches in his PhD. Dominic will be at Culture Tech in Derry/Londonderry in September for a 4 day bootcamp – I will put more information up about this closer to the event.
Yesterday Suzanne McDonough, Katy Pedlow (Health and Rehabilitation Sciences) and I had the privilege of presenting our recent research on the Oculus Rift VR headset and Leap Motion Controller to our local politicians at Stormont. Some pictures below.
We have been progressing from our previous simulations of traditional rehabilitation tasks using the Leap Controller to investigating and uncovering the best practices in interaction design among recent commercial releases of software. We are keen to map clinical requirements for rehabilitation exercises to existing games that contain similar movements in their controller design. We are also learning from best practices for the design of our own rehabilitation games – the main issues issue with commercial games is (obviously) that they are not tailored to treatments and are not usually adaptive to individual requirements. During this phase of our investigations we are also looking at whether the use of the Oculus Rift VR headset improves the usability and function of our rehabilitation games. In particular can it help patients with depth perception as they reach out in a virtual 3d scene?
The demos generated a lot of interest as they have when I have presented the technology in our uni open days (see bottom picture) and also when I recently visited a local primary school.
I love this game, I bought it for the PS2 around 2002 and again for the PS3 (for the better graphics) in 2012. I’ve referenced it in my teaching often, but embarrassingly I hadn’t actually got round to finishing it. Ico is still playable after all this time, it has a perfect balance between puzzle solving, exploration, story and action. It ended quite abruptly but I enjoyed the end, particularly the bit where you get to play a little longer after the credits! I think a proper sequel could be fantastic and I think it could make for a nice indie-style movie, so long as they kept it about the three main characters.
I was sparked into action by Edge magazine’s readers games of a lifetime (see below). I have all of these games in the house (except Golden Eye). I agree that they are all good games and I would like to try and finish a few more before the end of the year – I hate unfinished games! I’ve already completed Portal, Bioshock (except the last boss!), Ico, The last of us, Halo, and RE4. A few like Dark Souls I’ll never complete and I’ve had enough of World of Warcraft.
20 – World of Warcraft
19 – Marjoa’s Mask
18 – Vice City
17 – Chrono Trigger
16 – Portal
15 – Red Dead Redemption
14 – Bioshock
13 – Ico
12 – The Last of Us
11- Dark Souls
10 – Super Mario Galaxy
9 – Metal Gear Solid
8 – Shadow of the Colossus
7 – Goldeneye
6 – Halo
5 – RE4
4 – Final Fantasy VII
3 – Half Life 2
2 – Super Mario 64
1 – Ocarina of Time
Over the last couple of months I have been working with a development version of the Leap Motion camera, in collaboration with SilverFish Studios, to build VR prototypes for physical therapy focused on finger and hand exercises. We set out to create simulations of actual clinical exercises that we could demonstrate to physiotherapists and occupational therapists to obtain feedback on the potential use of the Leap camera in the home. Three prototypes were constructed and are shown in the video below. The virtual tasks comprised: Cotton Balls (lift virtual objects and place them in bowl), Stacking Blocks (stacking virtual blocks), and the Nine Hole Peg Test (lifting and moving of nine virtual pegs).
We decided to use a fairly high level (easy to use) game development tool that facilitated quick prototyping, had a research/indie friendly licence, and had plugins for both Leap and Kinect. Shiva3D was the engine chosen and we were able to create both Kinect and Leap interactive prototypes very quickly – its a shame that the Stonetrip, the company behind Shiva3D, are now in liquidation but Unity3D is another excellent engine for this type of work (despite the limitation of requiring the pro license for plugins). A good game engine provides many useful tools for building effective 3D interactions such as built in lighting and physics. The simulation of Jenga that we built provides a very good illustration of what can be constructed in a few days:
We were very happy with the trials and the response from clinicians at the Brain Injury Unit in Musgrave Hopspital, Belfast, and we hope to present the details and results of the trials at the ITAG conference this year. Physiotherapists and occupational therapists at Musgrave already use game based therapies quite a lot and were very positive about the potential use of Leap based virtual therapies for home use and particularly for younger patients. Clinicians provided a lot of very specific feedback on particular finger/hand/arm motion that they required including exercises that encourage patients to extend out their arm and open out their hand at the same time – like they were pushing someone away. We have already started on a gameplay mechanic that can include this motion and the video below shows a very early version:
The grab gesture is also very easy to implement with Leap and the following video demonstrates a simple shelf stacking task that could be the basis for a rehab game:
SilverFish Studios is an independent software company based within Northern Ireland. SilverFish will have been operating for 2 years come June – no mean feat in the current financial climate. They have had many ups and downs but SilverFish provides a fun working environment and the team are highly motivated, talented and supportive of each other. They have been involved in designing and making several games (one due for release soon), been teaching game development for local schools, and have been engaged in quite a lot of web development and consultancy – typically they enjoy taking on jobs that require aspects of problem solving and innovation.
They work hard and no more so in building Fly Cast Master which is an iOS app designed to help (beginner and experienced) anglers improve their technique by analysing their casting motion (while holding the phone). Fly Cast Master was a complex App. to make due to the design and construction of robust motion detection algorithms and the necessary collaboration with experts from the angling domain for data collection, testing, and in the provision of suitable training materials – trainers, equipment suppliers, anglers, and collaborators at Irish Angler magazine.
My wife, Therese Charles (or @bionic_rez), established SilverFish with the support of her father Brian McGinnis who was her greatest fan. He would be incredibly proud of what she has achieved with the company and particularly in achieving the release of Fly Cast Master. I’m involved with SilverFish in an advisory role – mainly over a working lunch or after hours – but it has been very educational and enjoyable for me to be even superficially involved in helping a software product reach the App. store. I’ve learned about the process and this will help me in my future teaching, research and consultancy.
With the support of the local and international angling community and industry there are a number of ways that Fly Cast Master can be taken forward, and SilverFish have a number of plans in place already. So I hope that you will download and have a go at Fly Cast Master – see if you can beat the top score! One of the great things about Fly Cast Master is that you can still practise your casting even if you can’t get out! If you are enjoying the App. or have any constructive feedback let SilverFish know – they are passionate about making the angling trainer as educational and fun as possible.
We have had a paper accepted to The 26th International Conference On Industrial, Engineering & Other Applications Of Applied Intelligent Systems (IEA/AIE) June 17-21, 2013 Amsterdam (The Netherlands). The abstract is copied below – this is only a part of the unpublished work that we have on the back of Richard’s PhD, and hopefully we will find time to work on a journal paper that wraps this work up nicely. David will be attending and presenting the paper.
TITLE: Facilitating Player Interaction in a Dynamic Storytelling Environment
AUTHORS: Richard Paul, Darryl Charles, Michael McNeill and David McSherry
Abstract. Facilitating player interaction with stories generated using artificial intelligence planning techniques is an important challenge to be addressed in the development of interactive computer game worlds. The problem that we focus on within this paper is the loss of story context that occurs when plan steps are reduced to primitive actions which can be executed in the game world, making it difficult for players to understand their purpose. We propose a solution to this problem based on a mechanism for dual representation of story plans at the levels of abstraction required for meaningful player interaction and to enable plan steps to be executed in the game world.