2026-02-07 12:15 UTC
@oli@social.tchncs.de I have to disagree on one point: it takes surprisingly little hardware resources to run #GenAI efficiently for say 20 ppl. The hardware required for this is now quite affordable for a reasonably sized company as well as for enthusiasts (excluding the current crazy RAM prices). And decentralised operation neither requires a nuclear power plant nor an ocean for cooling, it works perfectly well with renewable energy.
Decentralisation is the key, one could perhaps even call it democratisation.
Replies (2)
-
@oli@social.tchncs.de 2026-02-07 14:02
@riaschissl@sigmoid.social I'd argue that you can't run infra for 20 ppl as efficiently as infra for 10 million people, resulting in even worse environmental net impact. And your small rig may be able to run inference but not training. My main problem is that we're throwing non-deterministic "solutions" at deterministic problems in the first place.
-
@dzervas@infosec.exchange 2026-02-07 16:16
@oli@social.tchncs.de @riaschissl@sigmoid.social even for 20 people, what would run and on what hardware? even deepseek ~0.5T parameter model needs a tremendous amount of vram (around 1G/B params) and the price of it is in the ballpark of "let's buy a house that will be useless in 5 years"