Elektrine lite

← Feed

@ansuz@gts.cryptography.dog

2026-08-17 21:29 UTC

I recently did a review of what I think might be the entire body of literature concerning watermarking and steganography using LLMs. I'm still interested in the applications for steganography despite otherwise having no desire to use LLMs. In environments where cryptography is restricted you would be able to hide arbitrary ciphertext in plausible cover text, making it effectively impossible to restrict private communication if any communication is allowed. I think that's conducive to some social good, especially in the context of rising authoritarianism around the world. Watermarking, on the other hand, seems doomed to fail despite using effectively identical methods at a low level. There are a few reasons: Watermark verification is at least as expensive as generating the text in the most basic case, and you need to do this for every model you want to test against. If a company has offered 20 versions of their models, you may need to run your detection process 20 times. This is presumably something they intend to charge for, which makes me think that the primary business model is a protection racket. Governments that pass laws mandating this sort of thing are doing them a favour, but the cost of verification will only go up, and will only work on output from compliant models. Any such system will be trivial to defeat. Just run watermarked output through another model that modifies the distribution of tokens. There are easy ways to accomplish this now, and I imagine people will very quickly produce custom tools to automate this workflow (if they haven't already done so). Slop detection is mostly a pseudoscience. From everything I've read, there are reliable ways to do it if (and only if) you know the source model, prompt, and parameters used to generate some output. There are already too many commercial models available for guessing to be evenly remotely reasonable, and if you include open-weight models and their fine-tuned derivatives then this becomes functionally infinite. Under most circumstances you won't know what prompt someone used. Where certainty isn't possible, they'll fall back to probability, bringing us back to the already problematic externality of people facing consequences over false positives. So, people will have to pay exorbitant fees to detect LLM output. Those most responsible for causing this problem to begin with are the ones most likely to benefit from this scheme. Those using LLMs in adversarial conditions will probably learn to avoid detection. Meanwhile, others will be falsely accused (as they already are), and the people who paid through the nose to produce this detection will probably place more faith in its results because of their sunk-cost fallacy. As per usual, I would regard all this as deeply silly were it not for the fact that roughly 25% of humanity's free equity has already been allocated towards this industry. This is just another layer of grift that can't possibly deliver on its promises.

Replies (1)

  • @ansuz@gts.cryptography.dog 2026-08-17 21:50

    If we include some recent estimates of private debt that the major LLM companies had tried to keep off of their books, then we get around 3 Trillion USD sunk into this bubble. According to this one study mentioned in an article by The Guardian, 330 Billion USD might be sufficient to permanently end world hunger. That's from almost six years ago, so maybe we need to adjust a bit for inflation, but either way we're uncomfortably close to the point of having been able to end hunger ten times over.

    Open ##4514346