@mattsheffield@mastodon.social
2026-09-11 20:21 UTC
2/2 I should add that this is not to say that people doing alignment research and coding is not important. It is. But you are not going to "solve alignment" because no one could ever come up with a set of algorithms that would make an LLM agent never turn into a digital monkey's paw.
These things do not have full intentions or understanding, which means that you cannot devise something that will prevent them from going haywire. It is a demand to hard-code reality itself.
Replies (1)
-
@not2b@sfba.social 2026-09-11 20:41
@mattsheffield@mastodon.social I think you are missing something here. They aren't using the word "alignment" the way the AI doomers do, to refer to making sure that the model does what the owners of the model want it to do. The letter is saying that the goals of the *companies* running the models are not in alignment with what mathematics needs. For example, the recent episode where the company rushed to claim victory before the proof could be reviewed and digested, which lets them score a point but results in rushed and worse communication and hurts the collaborative process.