The Maker of Stable Diffusion Shifts Toward Language Models

Stability AI, the embattled firm commercializing Stable Diffusion, is doubling down on language models in an attempt to turn its fortunes around.

Stability has unveiled the first model in its ‘Stable LM 2’ family of models. Called ‘Stable LM 2 1.6B,’ the inaugural model is just 1.6 billion parameters in size but it is a powerful language model.

Stable LM 2 1.6B was trained on two trillion tokens for two epochs, or cycles through the training data. It was fed multilingual data across seven languages, including English, Spanish and French, among others.

Stability’s newest model is designed to lower hardware barriers, with the company saying it would allow “more developers to participate in the generative AI ecosystem.”

The model outperforms other under 2B parameter models on most tasks, surpassing popular smaller systems like Microsoft’s Phi-1.5, TinyLlama 1.1B and Falcon 1B.

“By releasing one of the most powerful small language models to date and providing complete transparency on its training details, we aim to empower developers and model creators to experiment and iterate quickly,” Stability said.

It introduced both the base model and an instruction-tuned version. It also released data details and the final pre-training checkpoint before the cooldown, including the optimizer states to help developers continue pre-training and fine-tuning more smoothly.

Related:Stability AI Seeks Sale as Investors Lose Confidence in CEO

Shift for Stability

Stability has been shifting towards language model work for the past few months. Last December, it released the StableLM Zephyr 3B model, while the initial StableLM came out nine months ago.

The company looks to be changing direction following reports that it was facing increased pressure from investors amid money troubles.

Reports surfaced last November that it was up for sale, with the likes of Cohere and Jasper reportedly interested in buying the company.

Stability is reportedly spending millions of dollars on computing costs and salaries while bringing in a fraction of the amount in revenue. However, the company is said to be focusing on R&D.

A shift to language-based systems would complement its image and video generation while putting it in competition with text-based model providers.

Access Stable LM 2 1.6B

Stable LM 2 1.6B can be used both commercially and non-commercially – however, you will need a Stability AI Membership for commercial uses.

Non-commercial memberships are free but they are only for personal and research uses. The professional tier − for creators and developers with less than $1 million in annual revenue, $1 million in institutional funding and one million monthly active users − is $20 per month. Anything above those stipulations is considered enterprise level, which requires negotiations with Stability.

Related:Stable Diffusion Adds Video Generation

You can also try out the model for free via Hugging Face.

To be sure, the model does exhibit common issues such as hallucinations and the potential to output toxic language. Stability said it wants the community to “take appropriate measures to ensure they are developing responsibly” when building atop the model.