Available on GitHub under Apache 2.0 license

Wireless and video R&D company InterDigital has created a software platform for artificial intelligence-based video compression research.

‘CompressAI’ has been released to the video compression research and standards communities as an open source tool, in the hopes it will help speed up the process of designing, training, and testing a new generation of AI-based codecs.

Keeping it small

With the amount of data exchanged online continuing to grow, researchers are increasingly turning to AI models to see if they can be used to compress images and videos.

“Developing these AI-based codecs involves several complex, time-consuming, and tedious steps,” said Jean Bolot, vice president of the AI Lab at InterDigital.

Every time researchers develop AI-based codecs, they have to augment them with other neural network layers, find image and video sequences with which to train their network, and then compare the results against existing compression standards.

CompressAI aims to minimize the time required for these steps by automating some of the more repetitive processes, “allowing teams to test and evaluate AI-based codecs quickly and accurately,” Bolot explained.

One module helps load the data used to train neural networks, pointing towards publicly available datasets. Another formats the data so that it can be easily understood by neural networks.

Then there's reference layers, a module that includes potentially useful specialized video coding neural network layers, including entropy coding – a lossless data compression scheme. An additional package includes pre-trained modules, including those from InterDigital, as well as other open source deep image and video compression models.

Finally, there's the all-important benchmarking, using metrics such as PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity Index Measure).

The tool is available freely on GitHub. “By making our platform publicly available, we hope to be able to support the incredibly important work research teams are doing to bring the next generation of video compression standards,” Bolot said.

“Someone’s compression will save the world from data-geddon”

With data storage and data transfer both costly endeavors, the promise of using AI to reduce file sizes without noticeably impacting quality is a tantalizing one.

In February, Disney Research and the University of California, Irvine debuted a new compression model they said was 'AI-enhanced.'

The research product had less distortion and smaller bits-per-pixel rates than current coding-decoding algorithms such as H.265 when trained on specialized video content, they claimed.

“Ultimately, every video compression approach works on a trade-off,” Stephan Mandt, research lead and UCI assistant professor of computer science, said at the time.

“If I’m allowing for larger file sizes, then I can have better image quality. If I want to have a short, really small file size, then I have to tolerate some errors. The hope is that our neural network-based approach does a better trade-off overall between file size and quality.”

The system is far from ready for prime time, however – part of the current challenge is that the neural network itself needs to be compressed.

“Because the receiver requires a trained neural network for reconstructing the video, you might also have to think about how you transmit it along with the data,” Mandt said. “There are lots of open questions still. It’s a very early stage.”

Although it’ll be a while before such a system is in use on a platform like Disney+, others are already deploying AI in their compression tech

"AI technology is dramatically improving video compression for broadcast and OTT content delivery," Harmonic senior product manager Jean-Louis Diascorn said.

The head of AI at the streaming solutions company gave a detailed talk last year at the Society of Motion Picture and Television Engineers conference, explaining how AI could change compression.

"AI is very good at predictions and decisions," he said. "This, in fact, is very useful and very impactful for video compression."

AI is already being used to provide higher video quality at any bit rate, higher density using less computing resources, and to improve “the quality of experiences,” Diascorn said. “This can be accomplished with AI through Dynamic Encoding Style (DES), Dynamic Resolution Encoding (DRE), and Dynamic Frame rate Encoding (DFE).”

The field is growing, and – given the stakes – has a lot of high profile interest. One initiative to watch is the annual 'Challenge on Learned Image Compression,' sponsored by Netflix, Disney, Amazon, Nvidia, and Google, that awards prizes to those applying neural networks to the problem of compression.

Some of the work from the challenge is likely to be shown at next year's Data Compression Conference, sponsored by Microsoft, and featuring Alibaba, Dolby, and ByteDance among its committee.

"We have an exciting future ahead of us," Diascorn said last year. "We're just starting."