New Falcon AI Model: A Small Yet Powerful Challenger to Meta’s Llama 3

The team behind the open source Falcon AI model has released the latest version and claims it outperforms Meta’s Llama 3.

Developed by the Technology Innovation Institute (TII), Falcon 2 stands at 11 billion parameters and comes in two versions: A base version and another optimized for visual workloads.

The Falcon 2 team said they aimed to create smaller yet equally powerful models to enable developers and businesses to run the model more cost-effectively.

For comparison, the original Falcon was 40 billion parameters in size. There was also a much larger 180 billion parameter version that was susceptible to “problematic” outputs due to its size.

Both Falcon 2 models can efficiently run on a single GPU, according to TII, enabling businesses and developers to scale the model without requiring substantial infrastructure. Users can also run them on laptops.

“AI is continually evolving and developers are recognizing the myriad benefits of smaller, more efficient models,” said Hakim Hacid TII’s executive director. “In addition to reducing computing power requirements and meeting sustainability criteria, these models offer enhanced flexibility, seamlessly integrating into edge AI infrastructure, the next emerging megatrend.”

Falcon 2 11B VLM is the first multimodal version of the model, capable of handling images and returning text.

Related:Falcon 180B: The Powerful Open Source AI Model … That Lacks Guardrails

It can identify objects in images and visuals, making the model suitable for use in industries including health care, to help clinicians inspect scan results like X-rays or in finance to sift through charts and graphs to identify trends.

Credit: TII

Both models are multilingual and can handle tasks across a variety of languages including English, French, Spanish, German and Portuguese, among others.

Falcon 2 v Llama 3: Battle of the Open Source AI Models

When Falcon 2 was released, TII promoted the model as surpassing Meta's Llama 3.

Falcon 2 11B surpassed the performance of the 8 billion parameter version of Llama three on popular evaluation benchmarks.

The TII achieved scores of 78.30 on the WinoGrande benchmark, 82.91 on the HellaSwag test and 52.56 on the TruthfulQA benchmark, compared to Llama 3 8B’s 77.35, 82.09 and 43.09, respectively.

The smaller Meta model did however beat Falcon on the MMLU test, scoring 66.6 compared to Falcon 2’s 58.37.

Credit: TII

It’s important to note that when launching Llama 3, Meta only published the 8 billion version and a much larger 70 billion parameter version. The company previously dropped a 13 billion parameter version when it launched Llama 2, which would be comparable in size terms with Falcon 2 but a model’s size doesn’t always reflect its performance levels.

Related:6 Latest AI Models from Meta, OpenAI, Apple and More

“While Falcon 2 11B has demonstrated outstanding performance, we reaffirm our commitment to the open-source movement with it, and to the Falcon Foundation,” said Faisal Al Bannai, the Advanced Technology Research Council’s secretary generation and advanced technology affairs advisor to the UAE president. “With other multimodal models soon coming to the market in various sizes, our aim is to ensure that developers and entities that value their privacy, have access to one of the best AI models to enable their AI journey.”

Falcon 2 is open source, meaning anyone can use it. Businesses can use the model to power their AI applications so long as they agree to its acceptable use policy.

The base version of Falcon 2 can be accessed from Hugging Face.

The visual version of the model will be available from May 14.

Falcon 2 MoE?

TII plans to expand Falcon 2 by creating a Mixture of Experts (MoE) version of the model.

MoE models produce a series of responses based on the inputs of multiple smaller systems working in tandem to come up with an answer.

TII didn’t provide any details on the MoE version of Falcon 2 but said such an approach would improve the model’s accuracy and boost its decision-making abilities.

Adopting an MoE approach would help Falcon 2 overcome the limitations of the current transformer architecture most prevalent in language models, according to Seth Dobrin, Qantm AI founder and CEO.

Dobrin highlighted the original Falcon in his February appearance on the AI Business Podcast.

The former chief AI officer of IBM said that should TII adopt MoE for the latest models, it would “allow for improved performance without drastically increasing the number of parameters. This is a similar approach to what was used in GPT-4 and other large language models.”

Dobrin noted that despite the release of the second generation of Falcon, UAE-based TII is yet to offer a Falcon model to support Arabic.

“This presents a unique opportunity to enhance our status in the region and tap into a vast market,” Dobrin said. “They are still maintaining their Apache open-source license, making them the highest-performing truly open-source large language models.