Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Take part in our daily and weekly newsletters to get the latest updates and exclusive content for reporting on industry -leading AI. Learn more
Liquid AI, the Foundation model startup based in Boston from the Massachusetts Institute of Technology (with), tries to move the technology industry about its dependence on the transformer architecture, which underpins the most popular large language models (LLMS), such as: Openais GPT Series and Google’s Gemini Family.
Yesterday the company announced “Hyenanic edge“A new, folding-based multi-hybrid model for smartphones and other edge devices in advance of International conference on learning representations (ICLR) 2025.
The conference, one of the leading events for machine learning, will take place in Vienna, Austria this year.
New folding -based model promises a faster and more memory -efficient AI on the edge
Hyäenkante is designed in such a way that it exceeds strong transformer baselines for both the arithmetic efficiency and the quality of the voice model.
In real tests on a Samsung Galaxy S24 Ultra smartphone, the model delivered a lower latency, a smaller memory expression and better benchmark results compared to a parameter adjustment transformer ++ model.
A new architecture for a new era of Edge Ai
In contrast to most small models that were designed for the mobile provision-SMOLLM2, the Phi models and Lama 3.2 1B, the hyenine edges are removed from traditional attention-based designs. Instead, it replaces two thirds of the GQA operators (group cross-attention) with gated convolutions from the Hyena-Y family.
The new architecture is the result of the synthesis of Greatored Architectures (Star) in Liquid AI, the evolutionary algorithms was announced in December 2024.
Star examines a wide range of operating compositions based in the mathematical theory of linear input variables to optimize several hardware -specific goals such as latency, memory consumption and quality.
Bench brand directly on consumer hardware
In order to validate the real willingness of Hyena Edge, Liquid Ai tests carried out directly on the Samsung Galaxy S24 Ultra smartphone.
The results show that the Hyäenkante has reached up to 30% faster and latency injuries compared to its transformer ++ counter, with the speed advantages increase in the case of longer sequence lengths.
Pre-discharged in short sequencing lengths also exceeded the transformer-based based on critical performance metric for reaction-fast on devices.
With regard to the memory, the hyena edge used in all tested sequence length consistently less RAM during the inference and positioned it as a strong candidate for environments with tight resource restrictions.
Outperformance of transformers on language benchmarks
Hyena Edge was trained on 100 billion tokens and evaluated by standard benchmarks for small voice models, including Wikitext, Lambada, Piqa, Hellaswag, Winogrande, ARC-Easy and ARC challenge.
With each benchmark, hyenas edge either voted on the performance of the GQA transformer ++ model with strange improvements in the confusion values to Wikitext and Lambada and higher accuracy rates for PIQA, Hellaswag and Winogrande.
These results suggest that the model’s efficiency gains do not apply at the expense of the predictive quality-a common compromise for many noble-optimized architectures.
Hyäenkante Evolution: A look at performance and operator trends
For those who are looking for a deeper immersion in the development process of Hyänen Edge, a newer development process is Video Walkthrough Offers a convincing visual summary of the development of the model.
The video shows how the most important performance metrics – including preletation, decoding of latency and memory consumption – have improved compared to successive generations of architectural reinforcement.
It also offers a rare check of the scenes behind the scenes, how the inner composition of the hyenäänkante has changed during development. The spectators can see dynamic changes in the distribution of operator types such as self-assembly mechanisms, various hyenene variants and Swiglu layers.
These shifts offer insights into the principles of architectural design, which have contributed to the model to achieve its current level of efficiency and accuracy.
By visualizing the compromises and the operator dynamics over time, the video offers a valuable context for understanding the architectural breakthroughs, which are based on the performance of Hyänen Edge.
Open source plans and a wider vision
According to Liquid Ai, a number of Liquid Foundation models, including Hyänen Edge, are planning too open in the coming months. The aim of the company is to create capable and efficient all-purpose AI systems that can scale from cloud data centers to personal edge devices.
The debut of Hyena Edge also underlines the growing potential of alternative architectures to challenge transformers in practical environments. Since mobile devices are increasingly expected to carry out demanding AI workloads native, models such as Hyena Edge could set a new basis for the AI optimized by Edge.
The success of Hyena Edge – both in the case of shouting metrics and in the presentation of an automated architecture design – positions liquid AI as one of the aspiring players who can observe in the developing AI model landscape.
