New app lets you remove a person from a video

The Shelf of Deepfake Tools Has Arrived - Jason Mays, Machine Intelligence Specialist at Google, has developed a new tool for retouching images on the screen. It allows you to completely remove the image of a person from the frame, to create the illusion of his absence. The tool is built with Javascript and TensorFlow and can be tried out through the Disappearing People webcam app.

The method consists in the fact that initially the camera shoots an empty room, a detailed map of the visible image is created. Then, when a person appears in the frame, the application calculates which pixels it occupies and replaces them with pixels from the reference map. The work is carried out in real time, the algorithms are still imperfect and traces of human movements are still noticeable. However, he himself is not visible in the retouched video, there are no clues left to identify the visitor.

Mays designed the app out of creative and scientific interest, he does not elaborate on the scope of its application. However, the demo video immediately gives a hint - when a person moves on a soft surface, it typically bends under him, but the application does not restore these fragments of the frame. It looks like an invisible man is walking nearby. This is the first line of work - inexpensive special effects for cinematography.