Dubbing Doctor Who into German with AI
The borderless nature of the Internet has driven an explosion in interest and engagement with content in multiple languages. Streaming platforms are regularly introducing popular international TV shows, and consumers are watching more entertainment in foreign languages than ever before.\
Film production and dubbing companies use various workflows to dub content into other languages. Ranging from fast-paced dubbing through AI-generated voices to meticulous word-by-word work by human translators and voice actors.
Core to both of these approaches is the ability to isolate the original dialogue track so it can be removed and replaced with the translated dialogue. But what if you don’t have all of the audio files of the content separated? How can you remove dialogue in one language and replace it with another, if all of the audio is “stuck” together? For older or vintage films and TV shows, AI can be of great help. This was the case when production studio Pandastorm Pictures set out to dub the original Doctor Who series in German. Since the British series was originally filmed and produced in the 1970s, the production team’s only audio source contained the English dialogue, music, and sound effects all layered on top of one another.
“As a film distributor, we’re often confronted with the problem of missing music and effect audio tracks. Especially when dealing with older productions. In 2015, we acquired the rights for the BBC’s classic ‘Doctor Who’ franchise. But there was only the English mix available, which left us with two options: either composing new music and creating the effects, a very costly and work-intensive process or cutting out the English dialogues and reproducing only the sections missing music and effects. But this would only work for episodes with a simple music score.” — Dennis vom Berg, Senior Product Manager, Pandastorm Pictures
Fortunately, Pandastorm was able to use AudioShake’s AI, which can separate any audio source into individual tracks. From the audio of an episode of Doctor Who, our technology could distinguish between and separate the dialogue track from the background music and sound effects. This gave Pandastorm the ability to remaster the original track, remove the English dialogue, and layer in the new German dubbing.
“When we came across AudioShake AI, this seemed like a possible solution… A test with the AI showed us that we could extract the existing English dialogues, leaving us with a very good music and effect track on top of which we could add the recorded German dialogues and complete the German audio track for the show.” — Dennis vom Berg, Senior Product Manager, Pandastorm Pictures
After testing AudioShake, Pandastorm Pictures used its AI to help dub the entire season into German, which will be released later this year for the series’ 60th anniversary. The team is busy working on the “Genesis of the Daleks” now, also using AudioShake.
The work of Pandastorm Pictures and AudioShakes hints at the bigger opportunity that AI has in making content more accessible to international audiences. Especially for content old enough to have lost or always lacked audio files that engineers could isolate or manipulate while dubbing.
In this case, stems pulled from AudioShake’s AI provided engineers with better files to work from and gave German audiences a more immersive experience with Doctor Who in their native language. Future applications of AudioShake’s AI can continue to expand content for more audiences, making dubbing and accessible content features more easily available.