Amazon has unveiled an artificial intelligence technology on Prime Video capable of adapting actors' lip movements to match the audio of dubbed versions of films and series. The solution aims to make synchronisation between speech and facial movements more natural, reducing one of the most noticeable discrepancies when content is watched in a language other than the original.
The technology combines AI models with visual effects to analyse dubbed audio and adjust actors' mouth movements to match the words being spoken. When a character speaks in a language different from the original, the lip movements can be digitally modified to follow the new audio track, creating an experience closer to that of a production originally made in that language.
Amazon is currently testing the solution on the English-language version of the German series Maxton Hall. The company intends to evaluate the tool's performance before extending its use to other titles available on Prime Video. In the future, the technology could facilitate the adaptation of films and series to different markets, enabling productions to be made available in more languages without significantly compromising the synchronisation between voice and image.
Amazon's approach also distinguishes itself from systems that rely exclusively on AI-generated voices. In this case, the company uses audio produced through traditional dubbing and deploys AI alongside visual effects to adapt facial movements. Human involvement is therefore maintained in the audio production process, while the technology is used primarily to improve the correspondence between speech and image.
The move comes at a time when major streaming platforms are increasingly turning to AI to accelerate the translation and localisation of productions for different markets. Beyond Amazon, companies such as Meta and YouTube have also been developing automatic dubbing and lip-sync tools using artificial intelligence.
For the audiovisual industry, these technologies have the potential to reduce the time and costs associated with adapting productions to different languages, while also facilitating international distribution. The use of AI in this process does, however, raise questions regarding translation quality, the preservation of actors' performances, and the use of technologies capable of digitally altering a person's image.
Source: Diário Económico
Original article: https://www.diarioeconomico.co.mz/?p=539524
