Revolutionizing Portraits: Microsoft AI Brings Mona Lisa to Life Through Rap

Thanks to new artificial intelligence technology from Microsoft, the Mona Lisa can now do more than just smile.

Microsoft researchers recently introduced a groundbreaking AI model that can generate a lifelike video of a person speaking by using a still image of their face and an audio clip. These videos, whether of photorealistic faces, cartoons, or artwork, feature realistic lip syncing and natural facial and head movements.

Researchers demonstrated in a demo video how they used technology to make the Mona Lisa recite a funny rap performed by actress Anne Hathaway.

The results generated by the AI model, VASA-1, are entertaining and somewhat unsettling in their lifelike appearance. Microsoft mentioned that this technology could have practical applications in education or assisting individuals with communication difficulties. It could also potentially be used to develop virtual companions for people. However, there is also a concern about the misuse of this tool to impersonate others.

This AI-generated image, from an OpenAI Sora video, shows woolly mammoths treading through snow.

This AI-generated image, from an OpenAI Sora video, shows woolly mammoths treading through snow.

Sora/OpenAI

Related article

OpenAI is now offering a new feature that allows users to generate videos based on verbal instructions. This advancement in technology raises concerns beyond just Microsoft. With the rise of tools that can create realistic AI-generated images, videos, and audio, experts are concerned about the potential for misuse leading to the spread of misinformation. Additionally, there are fears that this technology could disrupt creative industries such as film and advertising even further.

Microsoft has announced that they do not have immediate plans to release the VASA-1 model to the public. This decision is similar to how Microsoft partner OpenAI is handling the release of its AI-generated video tool, Sora. OpenAI introduced Sora in February, but it is currently only available to a select group of professional users and cybersecurity professors for testing purposes.

In a blog post, Microsoft researchers stated their opposition to the creation of misleading or harmful content involving real people. They emphasized that the company will not release the product to the public until they are confident that the technology will be used responsibly and in compliance with regulations.

Creating realistic facial movements

Microsoft has developed a new AI model that has been trained using a variety of videos showing people's faces in motion. This model is specifically designed to identify and replicate natural face and head movements such as lip motion, expressions, eye gaze, and blinking. As a result, when VASA-1 animates a static photo, the video appears more lifelike.

In a demo video, you can see someone looking agitated while playing video games. The person's face shows furrowed brows and pursed lips.

The AI tool is capable of creating a video where the subject is directed to look in a specific direction or convey a particular emotion.

Upon closer inspection, there are still indications that the videos are created by a machine, like occasional lack of blinking and overly dramatic eyebrow movements. However, Microsoft claims that its model is superior to other similar tools and sets the stage for interactive experiences with realistic avatars that mimic human conversational patterns.

Editor's P/S:

The emergence of AI technology like VASA-1 raises both excitement and concerns. While it holds immense potential for entertainment, education, and communication, its misuse can lead to misinformation and impersonation. Microsoft's decision to withhold its release until responsible use can be ensured is commendable. However, it is crucial for all stakeholders, including policymakers and industry leaders, to address the ethical implications and develop guidelines to prevent the harmful application of such technology.

The potential impact of AI-generated media on creative industries cannot be ignored. As tools like VASA-1 become more sophisticated, they may challenge traditional filmmaking and advertising methods. It is essential to explore the opportunities they present while safeguarding the livelihoods of creative professionals and ensuring the authenticity of human expression. The future of AI-generated media depends on a balanced approach that promotes innovation while mitigating potential risks.