This AI system can make old photos look brand new

Tencent researchers build innovative system off Nvidia’s StyleGAN-2.

Move over DALL-E 2, a new AI model is grabbing attention – GFP-GAN.

The model, whose full name is Generative Facial Prior-Generative Adversarial Network, can restore damaged and low-resolution pictures.

Developed by researchers from Chinese company Tencent, the tool is free to use and can be downloaded via GitHub.

The tool uses both Tencent’s own model and a pre-trained StyleGAN-2 model from Nvidia – similar to the system used to develop GauGAN, Nvidia’s image generative model.

In a paper outlining how the model works, Tencent’s AI team used the two models to effectively fill in the missing elements of an old image. In just seconds, the combined power of two models can turn low-quality images into new and better ones.

Figure 1:

The paper suggests that previously, image restoration required a reference point to recreate specific details. GFP-GAN, however, works by combining pre-trained faces from Nvidia’s model with the data input from the photo being restored to create an image that has “a good balance of realness and fidelity.”

“Thanks to the powerful generative facial prior and delicate designs, our GFP-GAN could jointly restore facial details and enhance colors with just a single forward pass, while GAN inversion methods require image-specific optimization at inference,” the paper reads.

"Extensive experiments show that our method achieves superior performance to the prior art on both synthetic and real-world datasets."

The paper suggests that the model “performs well on most dark-skinned faces and various population groups” due to a combination of pretrained data and data from the input image.

Tencent's team did note that the color of the person in an input portrait may appear lighter than the original skin tone from a gray-scale image as “the inputs do not contain sufficient color information.” To rectify this further, the paper’s authors suggest the need for a diverse and balanced dataset to fully realize the potential of the model.

AI and images

GFP-GAN comes as DALL-E 2 made waves on social media for generating images from text prompts.

Developed by OpenAI, the model has been used to generate alternative versions of Johannes Vermeer's Girl with a Pearl Earring, the cover image for an issue of the magazine Cosmopolitan and images of ketchup for condiment brand Heinz.