Mobile Ground Truth in a VR Workspace

It’s funny how a twenty year old idea can be resurrected and blast into public consciousness once again. What am I talking about? Why it’s that rusty old Virtual Reality (VR) idea of course. For those old enough to remember, we had a plethora of movies (Lawnmower Man), game accessories (Nintendo Power Glove), and many attempts to make virtual reality an experience that matched the staggering highs of our wildest dreams.  We’ve wised up of course.  The crushing defeat of VR manifested itself in the failed Nintendo Virtual Boy.  It rolled up the worst of what VR had to offer.  Moving graphics, but your head stayed lightly anchored to your table top. The unit tipped over easily and it activated the worst motion sickness, eye strain, and neck strain. Virtual Boy simply wasn’t a VR experience of our dreams at all, but its wide attention and high cost killed everyone’s VR dreams.

Fast-forward to 2012, Palmer Lucky put together a prototype Oculus Rift VR Headset, and managed to pique the interest of John Carmack.  John is noted for his id Software founder status and 3D engine work on Wolfenstein, Doom, and Quake.  He single-handedly saved OpenGL from the demise of Microsoft, and basically invented the 3D first person shooter game genre.  Through vast amounts of fortune, Palmer met John, and a VR demo emerged at the 2012 Electronic Entertainment Expo (E3), and catapulted VR back into public consciousness. Next thing we know Facebook buys Oculus for two billion dollars.

In 2013, I started design work on our medical simulation system called PeriopSim that offers a virtual reality experience and iPad trainer app. We could train in full immersion, and later recap the training on an iPad. We could maximize the memory retention, making learning a natural experience.  The result is a trained practitioner, ready to step-up with a feeling of having actually done this all before.  This saves time and cost of ramping new perioperative staff in hospitals.

I travelled to several locations on the hunt for VR and simulation technology that would add to the user experience.  I tried William Steptoe’s research lab in London, UK, and was impressed by his augmented reality research. Later I stepped into the Trent Simulation Centre in Nottingham UK.  It was there I understood where VR and medical simulation should eventually meet.  Finally on my return to Canada, I met Jamie Kelly at VRcade in Seattle and experienced true “presence” with motion captured VR.  This was a life changing experience.

Back in Vancouver, I wanted to focus on usability and how we could pull VR significantly forward with good design and our knowledge of mobile technology.  This year the PeriopSim iPad app was deployed for the first time in June at the Canadian Neurological Surgeon Rookie Camp held in Halifax, and we completed a PeriopSim VR experience that was shown at SIGGRAPH.

Thinking about the fact that the display isn’t the only piece of the VR puzzle; we also have millions of people walking around with mobile phones packed with sensors. To scale up for more complicated usage, we made a series of investments, partnerships and developed drop-in components for building a VR experience in Unity and (thanks to Athomas Goldberg) UE4, hand interaction using Playstation Move controllers, and a pair of Mocap VR facilities for building things.

VR is best for looking at information and things with a spatial shape and size, what makes Mocap VR special is we’ve completely eliminated motion sickness from the VR experience.  Just place the headset on and look around, pick things up, and work in a physical space.

Any new medium unlocks new ways for us to rethink how to do everyday things. Many critical services and operations remain unnecessarily trapped by PCs, these could be rethought in a mobile + Mocap VR context.  The command centres for any dispatch operator, disaster scenario coordinator, and military command offer great examples where a Mocap VR environment would enable commanders to see from all angles, giving better comprehension of the situation, and make better strategic decisions.

By leveraging mobile on the ground with other data feeds and control tools in a Mocap VR space, we can re-envision work environments. At last we can leave behind the sedentary confines of our mouse and keyboard PC existence.  We can stand up and walk around both our virtual world and the real world, and leverage enormous strengths of both.

Leave a Reply

Your email address will not be published. Required fields are marked *