Microsoft's Phi-3 model is cool tech, but local LLMs are useless
Running an LLM on-device is a technological dead end for a very nuanced reason
Microsoft researchers just claimed a breakthrough in AI efficiency with phi-3-mini, a language model that approaches the performance of industry leaders like GPT-3.5 while being small enough to run on a smartphone. On the surface, this is an exciting development - who wouldn't want a pocket-sized AI assistant that can engage in deep conversation and analysis without even needing an internet connection?
But once you look past the novelty factor, it becomes clear that on-device LLMs are a technical curiosity, not a game-changing advance. In a world of ubiquitous connectivity and cloud computing, deliberately limiting an AI to run locally is almost always going to be a handicap, not a feature. Useful language model applications almost always rely on access to live information (or for you to provide live info in the context window) and the ability to interface with other systems, which fundamentally assumes a networked environment.
In this post, I'll break down the key claims of the phi-3-mini paper and explain why I believe on-device LLMs are ultimately a technological dead-end, at least in terms of real-world utility. Impressive efficiency gains aside, a model that can't leverage the full resources of the cloud is going to be severely limited in the value it can provide to users.
This full analysis - covering an overview of the model, its performance, how it works, and why I think it’s all for nothing because it misses something critical not about how AI works but about how technology works - is only available to you if you’re a paid subscriber. Let’s begin.
Keep reading with a 7-day free trial
Subscribe to AIModels.fyi to keep reading this post and get 7 days of free access to the full post archives.