There is an extinction risk because LLMs don’t work. They are next word predictors, so putting them in charge of things that can kill people is a real problem.