MIT solved a century-old differential equation to break 'liquid' AI's computational bottleneck

GURPS · Nov 22, 2022

On Tuesday, MIT researchers announced that they have devised a solution to that restriction, not by widening the data pipeline but by solving a differential equation that has stumped mathematicians since 1907. Specifically, the team solved, “the differential equation behind the interaction of two neurons through synapses… to unlock a new type of fast and efficient artificial intelligence algorithms.”

“The new machine learning models we call ‘CfC’s’ [closed-form Continuous-time] replace the differential equation defining the computation of the neuron with a closed form approximation, preserving the beautiful properties of liquid networks without the need for numerical integration,” MIT professor and CSAIL Director Daniela Rus said in a Tuesday press statement. “CfC models are causal, compact, explainable, and efficient to train and predict. They open the way to trustworthy machine learning for safety-critical applications.”

So, for those of us without a doctorate in Really Hard Math, differential equations are formulas that can describe the state of a system at various discrete points or steps throughout the process. For example, if you have a robot arm moving from point A to B, you can use a differential equation to know where it is in between the two points in space at any given step within the process. However, solving these equations for every step quickly gets computationally expensive as well. MIT’s “closed form” solution end-arounds that issue by functionally modeling the entire description of a system in a single computational step. AS the MIT team explains:

Imagine if you have an end-to-end neural network that receives driving input from a camera mounted on a car. The network is trained to generate outputs, like the car's steering angle. In 2020, the team solved this by using liquid neural networks with 19 nodes, so 19 neurons plus a small perception module could drive a car. A differential equation describes each node of that system. With the closed-form solution, if you replace it inside this network, it would give you the exact behavior, as it’s a good approximation of the actual dynamics of the system. They can thus solve the problem with an even lower number of neurons, which means it would be faster and less computationally expensive.