Post #4800985 · Elektrine

2026-09-14 11:50 UTC

Yeah this is the part about all this AGI talk and AI going rogue, it requires a prompt. It responds to an input. These agentic systems have powerful tools that can do other things then spit out text but it’s always responding to a prompt. It may be in a loop responding to its own prompt but it’s still a prompt. And what it spits out is never really anything new. It’s some derivation of what it’s ingested. Actual human intelligence works differently. We do come up with new ideas, new thoughts. So I’ve often thought these LLMs are a dead end to AGI.

Replies (2)

@MangoCats@feddit.it 2026-09-14 12:27

it requires a prompt As delivered to public chat interfaces, yes. They all eventually stop and ask “how am I doing?” Structurally, theoretically, nothing prevents them from running an infinite loop and continuing to act without that checkpoint. They could be given a goal to “optimize” whatever and just act and act and act in what their pattern-matching systems judge to be actions pursuing “optimal” results. There was an article about “the genie effect” where protagonists are “tricked” by being given something they technically asked for but not what they really wanted. LLMs carry the additional “risk” of not matching input requests to any recognizable pattern, just following their own path which makes no sense to us. Focused on different priorities.

Open ##4800984
@communist@lemmy.frozeninferno.xyz 2026-09-14 20:40

“what it spits out is never really anything new” this is just false www.youtube.com/watch?v=Z7dH_Cxx75g github.com/openai/ten-proofs unless you would agree that humans never spit out anything new, as well.

Open ##4812079