Artificial intelligence has learned to create detailed portraits of people from the roughest sketches

This year, due to the pandemic, the SIGGRAPH conference will be held in virtual mode, so its participants present their new products in advance. Among the most promising is the DeepFaceDrawing technology developed by scientists from the Chinese Academy of Sciences and the University of Hong Kong. It is designed to redraw simple sketches of human faces into detailed, photorealistic portraits.

Formally, DeepFaceDrawing does nothing new; similar tasks are already being successfully solved using programs such as pix2pixHD, Lines2FacePhoto and iSketchNFill. The original image in the form of strokes and dots is divided into basic components such as eyes, nose, mouth, etc., after which the AI ​​replaces them with fragments of real faces using a selection method based on an extensive database. The output is an analogue of a photographic portrait, in which all parts are as harmonious as possible to each other.

The difference between DeepFaceDrawing technology is in complete disregard for the quality of the source material. This program tries to "think", implicitly simulate the shape space in order to get believable variants based on the original image. If other programs need at least a minimal set of exact initial parameters, then there is no such limitation. The new AI sets a framework for itself in order to approximate the input data and synthesize the most realistic face within them.