2026-09-23 14:20 UTC
LLMs are also deterministic and the weights for a model could be ‘hard-coded’ into a chip, no?
Replies (1)
-
@trebach@sh.itjust.works 2026-09-23 14:40
They’re actually not because they have randomness built in. Since we are working with probabilities, it won’t always pick the next token that has the highest probability and the randomness can be tuned via a “temperature” setting to make it more or less likely that it will choose the most probable token. The weights for the model could be stored in a firmware chip but you still need RAM because it pulls all the weights into RAM in order to perform the calculations.