UAE-based Falcon 3 challenges leading open source companies as demand for small AI models increases

The UAE government is supported Institute of Technology Innovation (TII) has announced the launch of Falcon 3, a family of open source small language models (SLMs) designed to operate efficiently on lightweight, single GPU-based infrastructures.

Falcon 3 has four model sizes – 1B, 3B, 7B and 10B – with Base and Instruct variants and promises to democratize access to advanced AI capabilities for developers, researchers and companies. According to Hugging Face’s leaderboard, the models already outperform or come close to popular open-source counterparts in their size class, including Metas Llama and category leader Qwen-2.5.

The development comes at a time when the demand for SLMswith fewer parameters and simpler designs than LLMs, is increasing rapidly due to their efficiency, affordability, and ability to be deployed on devices with limited resources. They are suitable for a range of applications in various industries, such as customer service, healthcare, mobile apps and IoT, where typical LLMs may be too computationally intensive to run effectively. Accordingly Evaluates reportsThe market for these models is expected to grow, with a compound annual growth rate of nearly 18% over the next five years.

What does Falcon 3 bring with it?

Based on 14 trillion tokens – more than twice as many as its predecessor Falcon 2 – the Falcon 3 family uses a pure decoder architecture with grouped query attention to share parameters and manage memory usage for the key-value cache (KV) during the Minimize inference. This enables faster and more efficient processes when processing a wide range of text-based tasks.

At their core, the models support four major languages ​​- English, French, Spanish and Portuguese – and are equipped with a 32K context window so they can handle long inputs, such as strongly worded documents.

“Falcon 3 is versatile, designed for both general and specialized tasks, offering users tremendous flexibility. Its base model is perfect for generative applications, while the Instruct variant is suitable for conversational tasks such as customer service or virtual assistants,” notes TII website.

According to the Leaderboard At Hugging Face, all four Falcon 3 models perform quite well, but the 10B and 7B versions are the stars of the show, achieving state-of-the-art results in logical reasoning, language comprehension, command sequencing, and code and math tasks.

Among models in the size class with 13B parameters, the 10B and 7B versions of the Falcon 3 outperform the competition Google’s Gemma 2-9BMetas Llama 3.1-8B, Mistral-7Band Yi 1:5-9B. In most benchmarks – such as MUSR, MATH, GPQA and IFEval – they even outperform Alibaba’s industry leader Qwen 2.5-7B, with the exception of MMLU, the test for evaluating how well language models understand and process human language.

Cross-industry use

With the Falcon 3 models now available Hugging faceTII’s goal is to serve a wide range of users and enable cost-effective AI deployments without computational bottlenecks. With their ability to handle specific, domain-related tasks with fast processing times, the models can support various applications at the edge and in privacy-sensitive environments, including customer service chatbots, personalized recommendation systems, data analytics, fraud detection, healthcare diagnostics, supply chain optimization and training.

In addition, the institute plans to further expand the Falcon family by introducing multimodal models. These models are expected to launch sometime in January 2025.

Notably, all models were released under the TII Falcon License 2.0, a permissive Apache 2.0-based license with an acceptable use policy that promotes responsible AI development and deployment. To help users get started, TII has also launched a Falcon Playground, a testing environment where researchers and developers can try out Falcon 3 models before integrating them into their applications.