Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!
January 23, 2024
Stability AI, the embattled firm commercializing Stable Diffusion, is doubling down on language models in an attempt to turn its fortunes around.
Stability has unveiled the first model in its ‘Stable LM 2’ family of models. Called ‘Stable LM 2 1.6B,’ the inaugural model is just 1.6 billion parameters in size but it is a powerful language model.
Stable LM 2 1.6B was trained on two trillion tokens for two epochs, or cycles through the training data. It was fed multilingual data across seven languages, including English, Spanish and French, among others.
Stability’s newest model is designed to lower hardware barriers, with the company saying it would allow “more developers to participate in the generative AI ecosystem.”
“By releasing one of the most powerful small language models to date and providing complete transparency on its training details, we aim to empower developers and model creators to experiment and iterate quickly,” Stability said.
It introduced both the base model and an instruction-tuned version. It also released data details and the final pre-training checkpoint before the cooldown, including the optimizer states to help developers continue pre-training and fine-tuning more smoothly.
Stability has been shifting towards language model work for the past few months. Last December, it released the StableLM Zephyr 3B model, while the initial StableLM came out nine months ago.
The company looks to be changing direction following reports that it was facing increased pressure from investors amid money troubles.
Reports surfaced last November that it was up for sale, with the likes of Cohere and Jasper reportedly interested in buying the company.
Stability is reportedly spending millions of dollars on computing costs and salaries while bringing in a fraction of the amount in revenue. However, the company is said to be focusing on R&D.
A shift to language-based systems would complement its image and video generation while putting it in competition with text-based model providers.
Stable LM 2 1.6B can be used both commercially and non-commercially – however, you will need a Stability AI Membership for commercial uses.
Non-commercial memberships are free but they are only for personal and research uses. The professional tier − for creators and developers with less than $1 million in annual revenue, $1 million in institutional funding and one million monthly active users − is $20 per month. Anything above those stipulations is considered enterprise level, which requires negotiations with Stability.
You can also try out the model for free via Hugging Face.
To be sure, the model does exhibit common issues such as hallucinations and the potential to output toxic language. Stability said it wants the community to “take appropriate measures to ensure they are developing responsibly” when building atop the model.
Read more about:ChatGPT / Generative AI
Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.
You May Also Like
Generative AI Journeys with CDW UK's Chief TechnologistFeb 28, 2024
Qantm AI CEO on AI Strategy, Governance and Avoiding PitfallsFeb 14, 2024
Deloitte AI Institute Head: 5 Steps to Prepare Enterprises for an AI FutureJan 31, 2024
Athenahealth's Data Science Architect on Benefits of AI in Health CareJan 19, 2024