The annual ACM SIGGRAPH conference is the preeminent gathering of computer graphics pioneers in the world. Full stop. As such, it is something of a time machine, a visit to SIGGRAPH affords a glimpse at where industries that rely on computer graphics will be in over the next decade, depending on how close you’re willing to look.
For our immersive family this is the place to scope out what’s on the horizon in virtual and augmented reality, the broad collection of disciplines that is rapidly becoming referred to as “immersive tech.”
This series represents my notes on the things I’ve seen and experienced at SIGGRAPH 2019.
I’ve never hid the fact that I’m bearish on Magic Leap, the Florida Company augmented reality company whose pre-rollout buzz nearly eclipsed the sun.
The Big Idea behind Magic Leap is ambitious as hell: that realistic virtual objects and characters can be layered on top of physical reality through the use of a lightweight wearable device. Of course, there’s a lot of moving parts to that idea and tons of innovations not only in technology but in experience design that need to come to pass if the vision is to work.
So far I’d only encounter one experience with the Magic Leap: a third party demo that adapted an AR game for phones and tablets to the device. I saw the potential, and was impressed with how lightweight and unobtrusive Magic Leap’s hardware is — relatively speaking — but the hard limitations of the tech were readily apparent.
On day one of the conference I caught three Magic Leap demos. Two from in house teams and one from Parallux and NYU Future Reality Lab.
The first in house demo I saw was a short version of the newly launched Undersea. This program opens a portal in a nearby wall to the ocean, and plants a piece of procedurally generated coral on the floor. Small fish swim out into the space around you, and if they come near can be made to follow your hand around.
Sudden hand movement can shatter the illusion, so it takes a quick moment to learn how to “play” with your new fish friend. This is something common to both AR & VR experiences. Motion and object tracking are far from flawless on any hardware at present, so there’s always a bit of a learning curve. I’m happy to report that once you they the knack of it, the interaction with virtual fish feels right: pleasant and playful.
The most impressive thing about the demo was how well it overcame the relatively small field of view that the Magic Leap has. While there are still bands and the top and bottom of the lenses — you can kind of think of the Magic Leap One as “bifocals for alternate reality” — the images in Undersea draw the eye “deeper” into the frame. This feels like a conscious design choice, managing the user’s attention to overcome the hardware’s limitations. In short: it works.
Get Noah J Nelson’s stories in your inbox
Join Medium for free to get updates from this writer.
SubscribeSubscribe
I’m still not sure how compelling Undersea would be over the long haul, but its certainly a good example of the kind of world building that can be accomplished already in Magic Leap’s brand of AR.
Mica, the virtual being that caused a stir at the company’s own Leap Con, was also on hand. Mica, while still clearly a digital character, is dialed in so that her “physical presence” is somewhere north of the Uncanny Valley. The illusion that she’s looking at you, and moreover that she’s deeply curious about what she’s seeing, is compelling. For this demo we couldn’t talk to Mica. Our sole form of interaction was mimicking her — grabbing a controller and interacting with virtual objects which we used together to make a collage. It was something out of a sci-fi silent movie routine. The controller didn’t always work properly, but that wasn’t as interesting as Mica “herself.”
The experience design teams at Magic Leap clearly understand the need to create a dynamic between the user and the virtual being. What’s impressive is that they demonstrate the ability to nail down the small details while framing the dynamic in ways that allow the user to slip into the fictional reality. Mica’s physicality suggests curiosity and enthusiasm, and if you’re open to the experience even a little bit your mirror neurons start doing their bit to get you into the mood. That’s potent stuff, and hopefully the hardware will keep getting better and give these artists room to grow.
My final Magic Leap demo of day one was Mary and the Monster, a fictionalized account of the creation of Mary Shelley’s Frankenstein. A cartoony recreation of the famous bet between Mary Shelley and Lord Byron to create the most horrifying story, the whole thing plays out on a table as doll-sized holograms perform a kind of AR theatre for us. We were told that the story was visible in both AR and VR, with the VR version presumably blowing the characters up to life size.
To be blunt: I didn’t love this. Maybe I would have found it more interesting in VR, with life-sized characters. The technical side of things — Parallux’s tech meant that a half-dozen of us were able to see the show at the same time — is certainly interesting in terms of the potential for creative collaboration. Yet I don’t see all that much value to staging proscenium style works as AR mini theatre. That’s not what makes immersive tech compelling. There’s certainly a lot of skill going into the product, but if anything a passive narrative experience cuts against the grain of AR’s advantages. A mismatch of form and content.
So where did all this leave me on the Magic Leap?
Still a bit bearish, but I’m warming up to the hardware as a platform. It’s certainly one of the most comfortable headsets I’ve screwed onto my head in the past year. Yet it’s the in-house design teams that are giving me the most hope that Magic Leap’s hype might be able to be lived up to, at least a little bit.
To create a real sense of immersion designers have to focus on the connection of the user to, well, everything around them. Immersive means medium as totality. While it’s not necessary to have photorealistic quality inside an experience, the design needs to have a weight to it. A sense of reality where the relationship between action and consequence is real.
The sparks of that are definitely here.
NoPro is a labor of love made possible by our generous Patreon backers. Join them today!
In addition to the No Proscenium web site, our podcast, and our newsletters, you can find NoPro on Twitter, Facebook, YouTube, Instagram, in the Facebook community Everything Immersive, and on our Slack forum.
Office facilities provided by Thymele Arts, in Los Angeles, CA.
Discussion