We’re constantly being asked by our clients and partners how virtual and mixed reality (VR and MR, respectively) technologies could be used to transform their businesses and enhance human experience in the built environment. Stimulant’s interests lie specifically in how these technologies can (or can’t) be used to provide entertainment, information and/or utility in real space (i.e. a physical environment that can accommodate many users simultaneously in the same space), as opposed to virtual space (i.e. an experience accessed through a device that is agnostic of the user’s current environment, such as a mobile device).
In short, VR has already become a valuable tool for our workflow, but its very nature – private, decoupled from the real-world environment, and carrying practical risks around injury and hygiene – doesn’t make it compelling for the public, multi-user, shared digital experiences that we create, although it has become a key part of our internal design process. MR, on the other hand, has a lot of possibilities for smart spaces, digitally enhanced environments, and a very bright future indeed — even if current implementations are rough around the edges, based on our early experiments and investigations.
VR at Stimulant: It’s a Deliverable
While VR has inherent social, contextual, and experiential limitations, it is still getting heavy use at Stimulant. We’re using it as both a prototyping tool and a deliverable.
Our clients and partners rely on us to explain and visualize experiences at scale, in physical environments, and there are precious few ways to really give clients a sense of size and resolution for large-scale interactive installations short of actually building them out. Many of our clients have not participated in projects that involved creating architectural-scale digital experiences, and many of the best practices we’ve developed that guide our design decisions aren’t fully understood until viewed in proper scale and context. VR uniquely solves this issue by essentially letting us digitally “stage” our experiences at scale without screens, projectors, or rigging.
Whether it’s a meeting with Oculus Rift headsets in our studio or bringing Google Cardboard viewers to a presentation, VR allows us to pre-visualize installations, forms, apparent/viewed resolutions, and more, to make smarter decisions earlier on projects. Being able to showcase how large something really is, and even being able to simulate exact pixel resolutions, helps with setting client expectations, determining sight lines in a complex environment, guaranteeing content legibility, understanding ergonomic and human-factors limitations, and much more.
Unity provides an easy way to mock up large displays to assess scale and legibility in virtual reality.
MR at Stimulant: It’s an Experience
The very premise of mixed reality – holographic overlays on the real world, and real-world objects with digital properties – is directly in line with the kinds of experiences our clients ask us to create. In this context, we see MR as very exciting indeed, even if now, in 2016, it’s not quite ready for prime time.
The MR market is still maturing and evolving, and is several years behind VR. Magic Leap hasn’t released its platform at the time of this writing. Microsoft HoloLens has a maddeningly narrow field of view. Only HoloLens is, so far, untethered. Scanning and tracking are as good as they’ve ever been, but improvements will always be welcomed.
If we take a step back from what we currently think of as MR – head mounted displays, worn like goggles – and think more broadly about the principle of mixed reality and virtual overlays on the physical world, MR can be done with transparent OLED displays, which already exist (at the time of this writing, they are brand new and few exist in the rental or retail channels). Mixed with head or gaze tracking, single users can have near-perfect parallax shifts based on their movement, and observers can actually share their view, even if their perspectives don’t perfectly match. This opens up interesting shared observation experiences without a wearable.
Mapping of the environment (left) makes mixed reality visual wayfinding (right) a snap.
The possibilities are only now becoming clear. What excites us even more, though, is combining mixed reality with other technologies. Imagine MR with active RFID, allowing the correlation of real-world objects in 3D space, mixed with visualizing hidden properties of those objects: When was that machine last serviced, or when was this component last replaced? Humans can be anonymously tracked in a space using computer vision systems, and their paths visualized after the fact, in the actual space: In front of which exhibits did my museum visitors dwell for the longest? Combining a calendar database with an office seating plan allows for rich inferences to be made that can’t be derived from just staring at someone’s empty chair: Has Jane just stepped away from her desk, or is she in a meeting? Integrating with GPS systems allows MR to move outside and overlay data on geotagged locations.
Combining mixed reality with other technologies allows for robust capture and playback of human movement in physical space (left) or the display of the invisible metadata of objects or people (right).
Where does this lead us? Visualizing geopolitical borders in real space? Perhaps handy to pilots navigating air spaces, but borders are not always universally agreed upon. How visible can one’s metadata be, and who has control over it? How open, accessible, or controlled are the “data shadows” of objects or people when they can be visualized by others? How much data do we really want or need to see overlaid on our world? How mediated will our perception of our environment become?
For our clients, many of these concerns are alleviated by the nature of the place their visitors inhabit, like museums, attractions, and briefing centers. But we believe that imbuing the perceived environment with overlays of data, context, and insight will, frankly, change the world…assuming the mechanisms of viewing mixed reality are lightweight, inexpensive, and either unobtrusive or socially acceptable.
Conclusion
Both virtual and mixed reality technologies hold a lot of promise, but the good and bad realities of these technologies are also mixed. From interpretive overlays in museums, to assistive overlays in medicine and industry, to real-world alternative-reality gaming, both VR and MR have bright futures. At Stimulant, our dedication to introducing people to ideas, brands, and each other in realspace makes VR a perfect fit for our workflow, and MR a great addition to our end-user experience toolkit.
