2026-09-01 00:48 UTC
I don’t like conspiracy theories but hugging face was bought, tens of thousands of Mac’s were bought, RAM is “indefinitely “ high priced.
Starting to feel someone doesn’t want the self hosted AI to get legs.
Replies (1)
-
@boonhet@sopuli.xyz 2026-09-01 06:32
Self hosting also requires a ton of RAM because each individual user has to load a (usually slightly smaller) model + KV cache, whereas for hosted, they load the huge-ass model once (per cluster) and then store KV caches for a bunch of users, which has better scaling. The memory companies win whether you self-host or pay a subscription. They might win more if you self-host because look at the cost of self-hosting, it’s actually pretty steep if you don’t have the hardware already from the before times.