Adobe’s AI tool changes the entire video by editing frames

At its annual Max conference, Adobe unveiled a suite of experimental AI-powered tools that offer new and intuitive ways to edit photos, videos and audio. These experimental projects, known as “Sneaks,” include tools that can propagate changes applied to a single frame to the entire video, easily manipulate lighting in images, and even correct mispronunciations in audio files.

One of the most impressive of these projects is called “Project Frame Forward”. This tool allows video editors to add or remove any object or person from a video without the need for a time-consuming masking process. In the presented demo, this Adobe artificial intelligence tool easily identifies, selects and removes a woman in the first frame of the video and fills her empty space with a natural and harmonious background; A function similar to Photoshop tools such as Context-aware Fill. This change is then automatically applied to the entire length of the video with just a few clicks. Users can also add new objects to the video by drawing an outline and providing a text description, which will intelligently interact with the environment.

Another tool is Project Light Touch, which uses generative artificial intelligence to rearrange light sources in photos. The tool can change the direction of light, adjust its color, and even create dynamic lighting that moves live around objects and people. For example, you can light up a pumpkin from the inside or change the atmosphere of a room from day to night.

In the audio section, “Project Clean Take” was introduced. This new editing tool can use text commands to change the way words are expressed or even the emotion behind a speaker’s voice and replace words without affecting the main characteristics of the speaker’s voice. This technology completely eliminates the need to re-record audio or video. These projects are currently in the experimental stage and there is no guarantee that they will be publicly available, but many of the current features of Adobe products have been introduced in the past.

Source: The Verge