Inside the Montreal company engineering virtual-reality pop star Maya Kodes.

A trip to the South by Southwest festival in Austin, Tex., this month could mark the start of a mixed-reality phenomenon.

Near the turn of the millennium, Yves St-Gelais had a vision. A part of the founding team behind Montreal entertainment juggernaut Cirque du Soleil and a fledgling animation producer, St-Gelais conceived a mixed-reality future where celebrities were not fallible objects of fetish, but ubiquitous, geo-customizable and interactive avatars.

At the heart of this vision was a pop star – a force for good who would speak primarily through music, the world's lingua franca, as well as interact with her audience in their mother tongue, a feat he describes as "bringing art to the soulless." He would name her Maya Kodes and he envisioned a superhero's arc, with story beats similar to that of Spider-Man. Early demos proved promising but fruitless. Never mind the animation quality or the music, computers simply were not fast enough to create a viable, likeable, real-time reactive humanoid.

Two decades later, St-Gelais believes the tech has caught up with his vision.

Armed with a story and a songwriting team, 12-person production crew headed by an Emmy Award-winning animator, her own A&R man, a mountain of proprietary software and hardware, a touring holographic spectacle and two EPs of chart-ready pop songs available on iTunes and Spotify that bring to mind Sia and Lady Gaga, Kodes (pronounced "codes" as in computer, not "kode-es," like a Russian spy) is already the world's first interactive virtual singer. This month, starting with a trip to the South by Southwest festival in Austin, Tex., St-Gelais will set out to discover if Kodes can become its first mixed-reality phenomenon.

On a recent morning, a cheery crew of technicians, programmers and performers arrived at the office of Neweb Labs, which St-Gelais founded in 2015, to run a dress rehearsal for the touring show. A small ground-floor unit in a sprawling commercial park surrounded by body shops in Montreal's industrial Côte-Saint-Paul neighbourhood, the studio space – bifurcated into a test theatre and a work area – is hardly the slick cradle of innovation one might expect from such a company, but charmingly reflects the scrappy mentality of its occupants.

Trickling in, the team sipped coffee "borrowed" from an upstairs cafeteria while setting up their respective work stations. For the technology-inclined, this meant helming computers, assuring programs were booting and code was running smoothly, while others slipped into bodysuits strapped with ball-sensors or donned a head rig resembling the skull of an aardvark.

Gently prodded by Neweb's artistic director, Véronique Bossé, the group – all women save for a technical artist – exchanged knowing glances as the hum of processors grew louder. Before long, a blond humanoid appeared on multiple screens inside the studio and began shadowing a body-suited performer's routine.

"There she is," Bossé exclaimed, addressing the nearest screen. "Hello, Maya."

On the surface, a virtual pop star may seem tame compared to the recent holographic resurrections of legendary artists such as Tupac or Bob Marley. What sets Kodes apart is that she's being performed and rendered in real time.

Kodes's team performs this "soul insertion" through the synchronization of highly guarded proprietary software and a puppeteering duo: Adancer in a motion-capture rig controls her body, while a stationary singer handles her facial movements and voice; both have audience monitors allowing them to react and interact with whatever's happening in the theatre. This occasionally allows Kodes, who is projected onto a holographic screen stretched out across the stage, to slip out of human form and interact with her real and digital surroundings, allowing her to participate in elaborate, otherworldly stage routines – multiplying herself in a psychedelic haze, for example – then answer off-the-cuff audience questions.

At Neweb, Kodes is referred to as if she were actually in the room, not unlike when addressing Santa Claus around children. And, like Santa, she comes with her own convenient if slightly confusing back story: Created to combat the Y2K bug, a glitch turned her code to human. Greeted to the world with music, the new form henceforth documents its existential anthropological study via song.

Arriving at the tour dress rehearsal fresh from the Pollstar Live! convention in Los Angeles, St-Gelais is reservedly excited. In his late 50s, soft-featured with dark-rimmed glasses, he appears like a cross between Walt Disney and late-era Steve Jobs and speaks softly with a pronounced Québécois accent.

A veteran of the entertainment field, St-Gelais now splits his time between Neweb and its big sister, Vox Populi Productions, which has produced the Quebec political satire series ICI Laflaque for the past 15 years. It was working on Laflaque, which uses quick-turnaround 3-D animation to skewer politicians on a weekly basis, that convinced the budding producer that creating real-time, interactive animation was not only possible, but likely the future of entertainment.

In that sense, Maya Kodes is very much St-Gelais's Mickey Mouse, a likable emissary he can use to capture the imagination of a new generation and launch an empire of interactive holograms and AI bots. But first, he has to create his Steamboat Willie. To do this, he's bet big, putting his own money alongside that of private investors into Neweb, as well as receiving $1.2-million from the Canadian Media Fund to have Kodes and her team perform at industry showcases and 1,000-person-capacity venues in Switzerland, Sweden, Germany, Canada and the United States. The future, he says shortly after the rehearsal wraps, is interactive, global and beyond the capabilities of a single human. After all, there can only be one Britney Spears, Beyoncé or Taylor Swift, and each comes with their own very human problems. In contrast, multiple versions of Kodes can work 24/7 around the world without the fear of burnout or language issues.

"We had every big name in the Canadian music business pass through here. They loved Maya, but they said they didn't know what to do with her," St-Gelais explains. "The traditional music-industry model is just that: traditional. [Kodes] is fundamentally an attraction. The market model is not Warner, Sony or Universal, it's Cirque du Soleil."

If Neweb's mission appears out of movies, books or literature, it's because it more or less is. A 2013 episode of Black Mirror titled The Waldo Moment featured an interactive animated avatar running for Parliament; in the 2002 film S1M0NE, a successful director secretly creates an interactive character to star in his films; and, perhaps most famously, Dr. Frankenstein created a humanoid experiment in Mary Shelley's updated Prometheus myth. In all cases, the creations – often born of frustration and their creator's own existential dread – ended up turning on their makers, leaving them desolate or dead.

When I bring this up to St-Gelais, he laughs it off. Kodes is not an extension of himself as much as an evolution of a movement, he explains. The next generation of tech-savvy consumers will not only want to be entertained but also be part of their entertainment – whether through augmented, virtual or mixed reality.

And judging by market trends, St-Gelais has a point.

According to industry forecaster Digi-Capital, augmented reality and virtual reality combined will be a US$110-billion business within five years. And the entertainment sector, second only to gaming, has been a maven in the industry. Although headset technology is still too nascent to create a full-on tipping point, industry trends indicate music will be a leading way in for Generation Z, due to both its sensory immersion and short running time. But while some forward-thinking artists, such as Future Islands, Black Eyed Peas and Run the Jewels, have released VR experiences, for the most part, marquee pop stars have stayed out of the sphere – preferring to keep as much manicured control as possible.

"The technology is a factor, but so is the marketing," explains Jon Riera, one half of Combo Bravo, the Toronto-based creative team behind VR videos for A Tribe Called Red and Jazz Cartier. Riera argues that until the marketing budgets match those of traditional video and there's a centralized digital location where audiences know they can turn to to find music-based VR, the medium will always suffer from the "cheesy-wow" factor. However, he points out that at least one big-name, owl-loving Toronto artist has expressed an interest in working in the medium. "He just needs Apple to release its own VR headset."

Harold Price at Occupied VR, one of the leading companies working in the field in Canada, sympathizes with Riera's argument. We've already accepted the tools to this future – AI assistants and ARKit-enabled phones – into our home, he says. "What we need is a 'must-see' cultural moment. We need our Star Wars."

St-Gelais believes Kodes could be such a moment. "Look at Hatsune Miku," he says. "She plays to 200,000 people in Japan. And she is just a playback."

Miku, whose name literally translates to the voice of the future, is Kodes's most obvious predecessor, and much of the expectations of avatar-based musical success can be traced back to her effect on the industry. Invented as a marketing avatar in 2006, Miku is a "voicaloid" – meaning her voice is competently synthesized – who shot to fame in the late aughts, selling out stadiums in Japan, opening for Lady Gaga's 2014 North American tour and even appearing on The Late Show with David Letterman. As St-Gelais points out, Kodes, who currently has no comparative competition in the market, can not only do everything Miku can, but also be adapted for different geopolitical regions, while feeling just as organic and interactive in each one.

"For me, holography is not the last platform," St-Gelais says, revealing that plans are already in place for a weekly 30-minute talk show with live Q&A on Facebook. "The next step is connecting her to an artificial intelligence, so it's possible to have Maya on your phone or virtual reality."

First though, the Neweb team has to focus on bringing Kodes to the world "in person." The exercise almost already feels antiquated, but St-Gelais has been waiting nearly 20 years for the technology to be ready and he's not going to let this moment pass.

"We're on her ninth body," Bossé says, pointing out the speed in which the software develops, while mentioning there's already talk among the crew that the 10th will be a little more reflective of "real" women.

While the pomp and circumstance surrounding Kodes and Neweb's ambition is admirable, the question of her success still rests on finding a dedicated audience. And, for all the talk of connecting to international, tech-savvy youth waiting for tech to catch up to its demand, for me, Kodes's and Neweb's success lies squarely in its ability to appeal to children. Always the salesman, St-Gelais is keen to discuss the strong international reaction so far.

But it's when I ask him why he created Kodes that his mouth curves up and he affects a Disney glint. "I have children," he says in his Québécois accent. "I make her for my children."

© Copyright 2020 The Globe and Mail Inc. All rights reserved.