Respeecher CEO on working on Star Wars, Book of Boba Fett Luke and more

If you’ve ever watched a Star Wars show on Disney Plus, there’s a good chance you’re familiar with the work of tech startup Respeecher, whether you know it or not. The Ukrainian company’s AI-powered voice-cloning platform delivered Mark Hamill’s aged vocal performance in both The Mandalorian and Boba Fett’s bookas well as for an as yet unidentified character in Obi Wan Kenobi. Lucasfilm has asked Respeecher to keep this character’s name a secret for now – and with so many franchise veterans returning for the series, there’s certainly no shortage of potential candidates.

Polygon spoke to Respeecher CEO Alex Serdiuk to better understand a process that undoubtedly borders on sacrilegious for many fans: using technology to create fully customized performances for one (or possibly even two) of the most iconic characters in the Star Wars franchise. to create saga. From the start, Serdiuk emphasizes the human component behind the platform itself. “With our technology and services, we can create a digital copy of a given voice and enable another person to speak with that voice,” he explains. “And that’s why we make it possible [studios] Scaling voices, aging voices and even reviving voices for some projects.”

So a far cry from the mental image that terms like “artificial intelligence” and “voice cloning” conjure up – that of a sound engineer running lines of dialogue through a computer algorithm, which then spits out audio files. Respeecher’s work on Star Wars is surprisingly performance-oriented. While Darth Vader himself is now more machine than human if the Ukrainian startup provides his voice (and remember, we said “if”), the essence of the character’s voice is still very much flesh and blood.

A masked Darth Vader (Hayden Christensen) is lit by flames against a dark background in Obi-Wan Kenobi.

Image: Lucasfilm

“There’s no AI yet, and I don’t think there would be, that would allow us to just use it on a turnkey basis to create the performance we want to create. […] We need another human voice [to provide] Input because that human voice provides all the key, accent, speaking style and tempo that the AI ​​is not good at generating,” Serdiuk points out. He adds: “Our system requires performance on input, so it can be done by the same person who is de-aging, for example, or by someone else. […] It takes the whole performance, the whole acting of a so-called ‘source voice’ and then we do the transformation.”

Additionally, Respeecher’s pipeline allows actors like Hamill to record different takes, just like on a real set, which the company’s experts can later adjust at their end, based on notes from showrunners like Jon Favreau or directors like Deborah Chow.

“With studio projects and films, they might record thousands of takes for each line, and that means we’d have to convert all of those [into the younger voice], send them back and maybe send different versions because we used to train different models with different settings,” he says. “And sometimes we also need to meet creative expectations so they can just guide us, Can you try to make [a line reading] sounds a bit more like that? and we would work on making it sound a little bit more like what they’re asking.”

So the ultimate goal is to recreate a traditional performance by non-traditional means—in the case of The Mandalorian or Boba Fett’s bookas if Hamill’s lines were somehow carried straight from the 1983 set Star Wars: Return of the Jedi? “The goal is to make it sound like it was recorded in the studio yesterday by the target voice himself,” agrees Serdiuk.

Of course, with voice-cloning technologies like Respeecher’s, there’s still a risk that the synthesizer performance they produce will sound artificial, although viewers aren’t quite sure why. Serdiuk even admits that Hamill’s aged voice sounds “much, much better.” Boba Fett’s book as it in The Mandalorian, thanks to small but significant improvements to the way the Respeecher AI model was “trained” to emulate the actor’s singing. At the same time, the CEO is also quick to point out that many fans have complained about the visual effects used to portray 20-year-old Luke Skywalker The Mandalorian Finale, few noticed that the Jedi Master’s voice was also synthetic, until Lucasfilm spilled the beans in one Disney gallery “Making of” special a few months later.

Serdiuk mentions that Hamill’s convincingly decrepit vocals come into play The Mandalorian and Boba Fett’s book were all the more impressive considering the quality of the legacies Respeecher received from Lucasfilm. “That [data] was quite old, so we had something from cassettes, we have some old ADR recordings, something from a video game,” Serdiuk recalls. “And the thing is, you have to train that data in your model so that it can produce the quality of output that would fit into modern production. In many projects that deal with aging or resurrection [performer’s voices]this might be the biggest challenge as the lack of data and the quality of the data introduce additional blockers in terms of good sound.”

Respeecher’s CEO claims that overcoming these data-related hurdles has paid off now that industry heavyweights like Lucasfilm have taken their jobs. “[We] started with the idea of ​​building a synthetic language [platform] at the level where it would go through sound engineers and Hollywood studios and end up in big productions. So if they accept our sound, if they say something good about the sound we’ve been able to produce – and that’s a very complicated and difficult technical challenge to create synthesized speech at a level where it can’t be produced from a real recording In cases like this, it really encourages us and helps us grow,” he says.

I told Serdiuk that the growing acceptance of voice cloning technology could mean that studios will also stop using talented soundalikes to fill in for deceased actors. For example not Rogue One: A Star Wars Story — in which Guy Henry imitated the voice of the late Peter Cushing as Grand Moff Tarkin — be the kind of project that would automatically land on Respeecher’s desk now? Not necessarily, according to Serdiuk, who sees Respeecher’s voice cloning technology as one of several viable options available to filmmakers.

Rogue One - Governor Tarkin / Death Star

Governor Tarkin looks at the Death Star
Credit: Lucasfilm/Disney

“There are always different ideas about how things should be done in the industry and fans have different ideas about how it should be done. I would not say that [Respeecher] is very well suited to be a leader or judge [which approach is best],” he says. Serdiuk also made it clear that should Lucasfilm ever ask Respeecher to recreate the voice of a dead actor, the company would only do so with the consent of that actor’s estate. And while Serdiuk’s response will put voice actors at ease for now, the fact suggests , that Lucasfilm is a regular Respeecher customer, indicated that the startup has made serious strides in voice cloning technology.

In fact, Serdiuk already has a vision for the future of Respeecher that goes beyond the voices of aging actors, though he firmly believes the company’s plans will expand, not limit, the filmmakers’ creative horizons. He talks about democratizing technology so smaller film and television studios and video game developers can use it to further stretch their budgets. He also speaks enthusiastically about the breakthrough healthcare applications of Respeecher’s platform – even naming a case where the company is working with a voice actor who has lost his voice so he can perform again.

However, looking to the future doesn’t mean that Serdiuk has lost sight of what it means for Respeecher to be part of a certain space opera set long ago in a galaxy far, far away. “It’s something special. I mean you are part of this story. We can safely say that Star Wars broke quite a bit of ground for Hollywood, right? They’ve disrupted the industry technically from the start, and the way they make their films is extraordinary. It is therefore a great honor to be able to work with these people and learn from them.”

https://www.polygon.com/star-wars/23178636/respeecher-star-wars-ceo-interview-luke-vader Respeecher CEO on working on Star Wars, Book of Boba Fett Luke and more

Charles Jones

24ssports is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – admin@24ssports.com. The content will be deleted within 24 hours.

Related Articles

Back to top button