Tencent researchers build innovative system off Nvidia’s StyleGAN-2.

Ben Wodecki, Jr. Editor

August 1, 2022

2 Min Read

Tencent researchers build innovative system off Nvidia’s StyleGAN-2.

Move over DALL-E 2, a new AI model is grabbing attention – GFP-GAN.

The model, whose full name is Generative Facial Prior-Generative Adversarial Network, can restore damaged and low-resolution pictures.

Developed by researchers from Chinese company Tencent, the tool is free to use and can be downloaded via GitHub.

The tool uses both Tencent’s own model and a pre-trained StyleGAN-2 model from Nvidia – similar to the system used to develop GauGAN, Nvidia’s image generative model.

In a paper outlining how the model works, Tencent’s AI team used the two models to effectively fill in the missing elements of an old image. In just seconds, the combined power of two models can turn low-quality images into new and better ones.

Figure 1: 9312.jpg

The paper suggests that previously, image restoration required a reference point to recreate specific details. GFP-GAN, however, works by combining pre-trained faces from Nvidia’s model with the data input from the photo being restored to create an image that has “a good balance of realness and fidelity.”

“Thanks to the powerful generative facial prior and delicate designs, our GFP-GAN could jointly restore facial details and enhance colors with just a single forward pass, while GAN inversion methods require image-specific optimization at inference,” the paper reads.

"Extensive experiments show that our method achieves superior performance to the prior art on both synthetic and real-world datasets."

The paper suggests that the model “performs well on most dark-skinned faces and various population groups” due to a combination of pretrained data and data from the input image.

Tencent's team did note that the color of the person in an input portrait may appear lighter than the original skin tone from a gray-scale image as “the inputs do not contain sufficient color information.” To rectify this further, the paper’s authors suggest the need for a diverse and balanced dataset to fully realize the potential of the model.

AI and images

GFP-GAN comes as DALL-E 2 made waves on social media for generating images from text prompts.

Developed by OpenAI, the model has been used to generate alternative versions of Johannes Vermeer's Girl with a Pearl Earring, the cover image for an issue of the magazine Cosmopolitan and images of ketchup for condiment brand Heinz.

About the Author(s)

Ben Wodecki

Jr. Editor

Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like