Why Deterministic Inference Should Be a User-Facing Choice
When Thinking Machines Lab launched their new blog Connectionism, their first post “Defeating Nondeterminism in LLM Inference” - framed the problem in elegant technical terms: how do we reduce unpredictability in large language model outputs - and how do we make that choice user-facing?
That word - connectionism - stayed with me. Because the deepest challenge in AI isn’t only about connecting nodes in a neural net. It’s about connecting systems to the people who use them.
I’ve experimented with this many times in various ways. One example: I asked an AI to draft my bio using specific information I provided. Instead, it invented work I’d never done, wrapped in language I never gave it. I gave it my story; it handed me back a fiction. The gap wasn’t just technical - it was personal. It felt less like collaboration and more like being in an unresponsive classroom. Like when a child says, “I need help,” only to be told, “No, you mean something else,” the system rewrote me instead of listening.
This is where I thought about the idea of a gap protocol: Mind the gap, then build the bridge. In human terms, the gap is the distance between what we expect and what we actually experience.
In AI, it’s the unpredictable space where nondeterminism can turn into mistrust.
The bridge? Deterministic inference - but only if it’s more than a backend safeguard. To build real trust, it has to be user-facing - a choice people can see, toggle, and control.
Because defeating nondeterminism isn’t just a technical achievement. It’s about trust. And trust is built not when the system fills the silence, but when it acknowledges the space - and hands us the choice.
cc: hashtag#Apertus, Thinking Machines Lab, Horace He
Kind regards,
Amber Eltaieb
PS
Reference this work as Eltaieb, Amber, “Why Deterministic Inference Should Be a User-Facing Choice", Starstuff International: StarStuff Architecture Lab, Sep 2025
Reference:
He, Horace and Thinking Machines Lab, "Defeating Nondeterminism in LLM Inference", Thinking Machines Lab: Connectionism, Sep 2025.