What are Liquid Neural Networks? And why should you care

What are Liquid Neural Networks?
And why should you care?

Pascal Brier

Oct 25, 2024

• Facebook
• Linkedin

Earlier this week, our friends at Liquid.AI introduced their first products powered by Liquid Neural Networks (LNNs). This is a new generation of AI models that promise to achieve state-of-the-art performance at every scale, while maintaining a smaller memory footprint, more computing efficiency and much better transparency.

But what are Liquid Neural Networks exactly? And why should you care?

To understand, let's consider the classical Large Language Models we've been building over the past few years.

LLMs, like ChatGPT, are great statistical learners. Meaning they have to ''memorize'' trillions of variations and patterns from an immense training dataset in order to coherently mimic those patterns in their outputs. This is why models are becoming better but also exponentially larger with each iteration. This reliance on scale is why LLMs have grown into models with trillions of parameters. To produce increasingly complex and nuanced outputs, LLMs need always more parameters, which means more data, more computational power, and a larger model size.

This approach is becoming extreme, as constantly increasing the number of parameters to improve performance is both resource-intensive and costly. In our race to develop Generative AI, we are also increasingly scaling black boxes with little explainability.

In contrast, Liquid Neural Networks (LNNs) offer the promise of fundamentally more adaptive and efficient model architecture.

Instead of relying on larger and larger networks of simple neurons, LNNs use smaller networks of more capable neurons that adjust in real-time to new inputs. In simplified terms, these neurons are mathematical formulas that are adaptive - they can change their behavior based on new inputs. They adjust their connections and processing methods dynamically, like a formula that updates itself as new information comes in.

Since these neurons are not static, they continuously evolve based on the information they process, allowing LNNs to learn on the go and adapt to new environments without needing retraining. This adaptability means that LNNs can perform complex tasks with far fewer parameters. As a result, LNNs are better suited to handling dynamic, unpredictable situations, such as real-time decision-making in autonomous systems or robotics, where flexibility and efficiency are key.

This is why LNNs have immense potential. Same or maybe better performance (future will tell) , but far less computational power, energy, and cost. This has an obvious impact on the sustainability profile of AI, but also opens up many more deployment options and use cases. LNN-based architectures enable AI deployments on smaller edge devices-such as mobile phones, vehicles, smart-home systems, airplanes, and industrial machinery-without relying on massive, cloud-based computing resources.

Try to imagine a fully offline automotive AI system that runs efficiently on a standard PC CPU without needing specialized hardware like GPUs. Or an industrial robot that continuously adapts to new tasks and surroundings, making real-time adjustments as it learns from ongoing interactions.

The implications of Liquid Neural Networks are profound. Their ability to deliver state-of-the-art performance with fewer resources and real-time adaptability represents a significant add-on in the evolution of AI. For business leaders and CxOs interested in staying ahead in the AI race, keeping a close eye on Liquid Neural Networks evolution is not just advisable-it's essential. This could very well be the future of sustainable, efficient, and explainable AI.

Group Chief Innovation Officer, Capgemini

Pascal Brier was appointed Group Chief Innovation Officer and member of the Group Executive Committee on January 1st, 2021. Pascal oversees Technology, Innovation and Ventures for the Group in this position. Pascal holds a Masters degree from EDHEC and was voted "EDHEC of the Year" in 2017.