Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!
December 6, 2023
Want to improve your AI image generations using older models? Researchers have come up with a way to upgrade older diffusion plugins to make them compatible with current-gen models like Stable Diffusion XL without retraining.
Dubbed X-Adapter, the tool is designed to control the upgraded model with a new text-image data pair – it universally upgrades plugins, making them directly compatible with the upgraded model.
It effectively makes a copy of the old model, preserving connectors for different plugins and adds trainable mapping layers that bridge decoders from models of different versions, allowing for feature remapping. The remapped features are then used as guidance for the upgraded model.
Examples of results on SDXL and SD 2.1 using X-Adapter and pre-rained SD 1.5 plugins and the related prompts.
It's like playing older video games on a new console – enhancing graphics (or in this case images) while retaining the core aspects of the original system. For example, the foundational qualities of Stable Diffusion 1.5 can be retained while still leveraging the power of Stable Diffusion XL.
The team behind X-Adapter is from the AI lab of Chinese tech giant Tencent, Show Lab at the National University of Singapore and Fudan University in China. They experimented with plugins like ControlNet and LoRA on Stable Diffusion 1.5, and then, using X-Adapter, upgraded them so they could work with Stable Diffusion XL.
The creators of X-Adapter said the concept could “facilitate wider application in the upgraded foundational diffusion model.”
There were some issues with X-Adapter. The paper outlining it suggested that some plugins failed to maintain the identity of personalized concepts. The researcher said this was because the custom plugins “work on the text encoder rather than the feature space concepts that are not directly injected into the upgraded model other than fused as guidance because of the custom plugins.”
At the time of writing the code for X-Adapter is not available, however, it’s due soon. It will be accessible on the X-Adapter GitHub page.
Read more about:ChatGPT / Generative AI
Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.
You May Also Like