Immersive theatre meets VR in this joyous collaboration between Baobab Studios and Mathias Chelebourg

It starts with, of all things, a broom.

I’ve been thrust into a ramshackle home on stilts. It’s a single-room house nestled amongst the trees in a thick, fog-filled forest. And I’m getting yelled at by my mother, who is an animated frog. She’s sweeping because our house is a mess, and won’t I be a good son, and please help out? She thrusts the broom into my hands. I reach out and am surprised to feel a real-life broom in my hands.

My hands are green, because, naturally, I am also an animated frog. A frog named Jack.

“Go sweep under the stove, Jack,” commands Mother Frog. I attempt to clean up in the kitchen but don’t see any dirt there. I start to move around the tiny home, but Mother Frog tells me I’m doing a terrible job sweeping. She then grabs the broom out of my hand and tells me to sit down on a stool.

Cautiously, I follow her command and am startled that the piece of furniture feels surprisingly solid under my hands. Mother Frog begins a tirade about she wants to sell an old radio, and how the hawker will be any minute now, and I should sell it to him, but only for a good price, and do I understand what I am supposed to do, son?

Silence.

“Well, Jack?” she says.

It occurs to me that I not only can I speak in this interactive piece, but also that I should speak. Because this is a real-life performer, and she’s responding to what I do and what I say during this one-on-one within a virtual environment. I’m not just an observer in this world, I’m at the center of the story, and anything I say and do can change what happens.

“Y — Y — Yes, ma’am,” I say, taken aback. “I’m going to sell the radio to the hawker when he gets here.”

“For what?” says Mother, impatiently.

“Uh…” I respond, dumbfounded.

“For the best possible price. The best possible price. You got that Jack?”

Mother leaves, muttering about my inadequacies, and I get up from the stool. I immediately bump a lightbulb hanging from the ceiling. Ack.

Then I pick up the old radio which Mother wants to sell.

Damn. It’s real, too.

As expected, the transaction with the Hawker doesn’t go as planned and Mother is not enamored of the magical bean I’ve acquired (which, as an aside, smells terrible). So when the beanstalk suddenly starts to grow, taking the house up with it, I find myself gripping the edges of the bed, afraid to get up, lest I tumble into the darkness.

My brain is on fire, trying to process what is happening.

Jack: Part One comes to life via an exuberant performer in a motion capture suit who plays all of the different characters in the piece (I “acted” with Lee Delong when I went). The actor also manually triggers the next scene based on the participant’s responses, which accounts for a lot of the piece’s fluidity. The creators also use a multitude of cameras and sensors to track the positions of objects in the environment like the bed, broom, stool, coffeepot, and more. And a few helpers in the back playfully trigger effects like wind and other motions, on cue. (Sometimes the story or underlying technology doesn’t work as expected, but I observed the quick-thinking of the performer help the plot get back on track or smooth over any bumps in the road; the technology remains quick finicky at the current stage.)

The entire whimsical story takes place within a small physical footprint, but inside the headset, the world of Jack: Part One feels like there’s so much more to explore. I wanted to climb aboard the hawker’s ship and see what other treasures he might have hidden. I wanted to lie down in the bed and star up at the stars, daydreaming the hours away. I wanted to pick up the jar of fireflies and go exploring. I wanted to walk out of the home into the forest to see what would happen.

The visceral experience I had in Jack: Part One felt indelibly real and magical. For the first time in a long while, I was like a kid again, plopped gleefully into a world of infinite possibilities.

I’ve spent a lot of time in VR headsets over the last week. I’ve spent a lot of time in line, talking to filmmakers and journalists and producers and curators and students and enthusiasts of all types. It’s been great to see just how many people are excited by this current wave of immersive experiences; the demand for many of these has been off the charts at the Virtual Arcade, particularly for any exhibit with a high amount of agency and a physical component.

And this is just the vanguard. VR makers are just starting to leverage the whole spectrum of ways they can combine the tactile and the digital, whether it’s making space for interactive live performance inside of rendered environments, or creating multi-sensory experiences where the participant can walk around, feel sensations of heat or wind, or even reach out and (literally) touch someone, with a headset on.

And it’s not just at the Tribeca Film Festival — between Draw Me Close, Tree, Terminal 3, Hero, THE VOID’s Secrets of the Empire (now at 5 locations and counting), Jack: Part One, Dreamscape’s Alien Zoo, Carne y Arena, and Holojam in Wonderland, I get the sense that there’s something in the air. Something wonderful that connects all of the aspects of immersive design, regardless of medium.

So wherever we’re headed to next with VR, count me in.

Tribeca Film Festival’s Virtual Arcade continues through April 28. Tickets are $40 for each 3-hour time slot. Capacity is limited and many exhibits will have long waits; it is suggested that visitors arrive 30+ minutes early before each session.

Catch up on all of our Tribeca Immersive 2018 diary entries.

No Proscenium is made possible by people like you: join us on Patreon today to support more content like this.

In addition to the No Proscenium website, our podcast, and our newsletters, you can find NoPro on Twitter, Facebook, YouTube, Instagram, in our online community Everything Immersive and in our Slack forum.