LLMs are notoriously non-deterministic, and even at temperature zero you still can't predict exactly where the weights will take you next.
An LLM can be made to be completely deterministic. I use them in this mode so I can reproduce test cases. Of course it requires complete control over the model, etc. but this myth that a computer program is non-deterministic needs to end.
You can 100% predict where the weights “will take you” given a set of inputs.
>Floating point matrix calculations are non-deterministic.
This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.
The software standard is. GPU matrix calculations, are not. The hardware, has tiny shifts that rarely matter, except in high finance and... AI modeling.
By "can't predict exactly where the weights will take you next" I meant with your brain. The blind chess analogy suggests you can predict, using your own thought process, the exact output of a prompt.
I think when people say non-deterministic what they mean is closer to chaotic, like https://en.wikipedia.org/wiki/Chaos_theory as in very small changes in conditions can produce completely different output making predictions difficult
You can 100% predict where the weights “will take you” given a set of inputs.