2026-09-15 11:22 UTC
The "local LLM" idea doesn't scale.
- If all LLM users would move to local LLMs, it would make the problem worse because your local compute is never as efficient as that of a specialised data centre. Both hardware utilisation and performance per Watt would be much worse.
- If the local LLM was really as capable as its cloud variant, the cloud operator would be running the "local" version, and it would do so more energy-efficiently.
#FrugalComputing (1/2)
Replies (1)
-
@wim_v12e@scholar.social 2026-09-15 11:23
Also, a local model is a distilled version of a large model. They can only exist because the large models exists. (2/2) #NoToAI