tl;dr: In the universe of chaos, the definition of deterministic is different than thehuman universe. Since the human can't control/measure every variable, it's not deterministic.
Compilers preserve semantics. That is part of their contract. Whether the output has instructions in one order or another does not matter as long as the output is observationally/functionally equivalent. Article does not do a good job of actually explaining this & instead meanders around sources of irrelevant "stochasticity" like timestamps & build-time UUIDs & concludes by claiming that LLMs have solved the halting problem.
Full determinism is also highly prized by compiler writers because it massively simplifies the task of reproducing and debugging problematic executions.
I wrote "We have not remotely solved the halting problem in the formal sense", which does not read like a claim that LLMs have solved the halting problem to me, but I'm open to
rewording it. How would you put it?
We haven't solve it in the formal sense b/c it is formally unsolvable so hedging adds no semantic content. It's not a formal article so you can phrase things however you want but an uncritical reading will leave the reader confused about what exactly you were trying to explain.
C and C++ compilers are limited to preserving semantics for data-race free code only, though.
They are allowed to turn a single load into multiple loads, or even a store into multiple stores: things that won't affect anything if you have only one thread accessing memory but for multithreaded programs, changing compiler or just making seemingly unrelated changes and recompiling can make existing data-race bugs have effects or not.
Attempting to get consistent results from floating-point code is another rabbit hole. GCC and clang have various flags for "fast math" which can enable different optimisations that reduce precision.
Before SSE, fp on x86 was done by the "x87" FPU which always had 80-bit precision, even if the type in the source code was 32 or 64 bits — and it used to be accepted to sometimes get more precision than asked for.
Java got its "strictfp" mode mainly because of x87.
Data races are undefined behavior¹ so in that case the compiler is still technically preserving semantics but if you use the proper primitives to remove undefined behavior (atomic operations, locks) for any shared state then the compiler will not generate code w/ undefined behavior & all modifications/mutations will be serialized in some order. You can then further refine the code if you want the operations to happen in a certain order. At the end of the day you must assume that the compiler will preserve your intended semantics otherwise we'd still be writing assembly b/c no high level specification would ever mean what it was intended to mean for the low-level executable machine model/target of the compiler.
Yes, but the spooky thing is still that the code with the bug and the code affected by a change in the compiler that triggers the effect of the bug could be in two different code modules.
> The computer science answer: a compiler is deterministic as a function of its full input state. Engineering answer: most real builds do not control the full input state, so outputs drift.
To me that implies the input isn't deterministic, not the compiler itself
You're not wrong but I think the point is to differentiate between the computer science "academic" answer and the engineering "pragmatic" answer. The former is concerned about correctly describing all possible behavior of the compiler, whereas the latter is concerned about what the actual experience is when using the compiler in practice.
You might argue that this is redefining the question in a way that changes the answer, but I'd argue that's also an academic objection; pragmatically, the important thing isn't the exact language but the intent behind the question, and for an engineer being asked this question, it's a lot more likely that the person asking has context for asking that cares about more than just the literal phrasing of "are compilers deterministic?"
It matters a lot. For instance, many compilers will put time stamps in their output streams. This can mess up the downstream if your goal is a bit-by-bit identical piece of output across multiple environments.
And that's just one really low hanging fruit type of example, there are many more for instance selecting a different optimization path when memory pressure is high and so on.
> ... the important thing isn't the exact language but the intent behind the question ...
If we're not going to assume the input state is known then we definitely can't say what the intent behind the question is - for many engineering applications the compiler is deterministic. Debian has the whole reproducible builds thing going which has been a triumph of pragmatic engineering on a remarkable scale. And suggests that, pragmatically, compilers may be deterministic.
> To me that implies the input isn't deterministic, not the compiler itself
or the system upon which the compiler is built (as well as the compiler itself) has made some practical trade offs.
the source file contents are usually deterministic. the order in which they're read and combined and build-time metadata injections often are not (and can be quite difficult to make so).
I mean, if you turn off incremental compilation and build in a container (or some other "clean room" environment), it should turn out the same each time. Local builds are very non-deterministic, but CI/CD shouldn't be.
Either way it's a nitpick though, a compiler hypothetically can be deterministic, an LLM just isn't? I don't think that's even a criticism of LLMs, it's just that comparing the output of a compiler to the output of an LLM is a bad analogy.
Also, most real build systems build from a clean directory and checkout.. so, outside of a dev's machine they should be 100% reproducible, because the inputs should be reproducible. If builds aren't 100% reproducible that's an issue!
I’ve felt like a good response to the vibe coding thing is that customers, product managers, etc ask for features and don’t read the code. You don’t need to read the code of something to build a level of trust about what it does and whether that matches your expectations. It is not that wild that you can have a setup where you get an application and without reading the code decide if it solves your problem to your satisfaction.
There will absolutely be systems of the future that are entirely LLM written. Honestly they will probably be better quality than the standard offshore team output.
But lets all hope these are not vital systems we end up depending on.
I'm using Claude every day and I am correcting the output all the time, by reading the code, because it does bad things and breaks things. I feel like the people saying that it doesn't matter either aren't building anything real or they're outsourcing their work onto reviewers or they have a time bomb on their hands.
And what happens when you change one word in the specification and the app completely changes?
Sure everything you have unit tests for might stay the same, but unless your unit tests are testing all observable behavior (and if they are they’ll be 100x longer than the code) users will notice incredibly confusing differences in every build.
Vibe coding doesn't mean that it's rebuilt from the spec from scratch each time, nor does it mean nobody looks at the application. It originally meant nobody looked at the code at all, although that's much more diluted I feel now - things are called vibe coding that sit on a larger spectrum.
My point was that "didn't look at the code, looked at the app and gave feature requests and tried the results" is a way of building applications that essentially anyone who doesn't code has been doing all along.
I have actually encountered multiple people proposing that. People in the camp of LLMs are the new high level compiler and code is assembly language.
That’s what prompted this entire are compilers deterministic article.
But ignoring whether it’s regenerated from scratch, when you ask an agent to add minor functionality, it will regularly change existing unrelated functionality.
Some of that is accidental, and could theoretically be solved. For example agents when refactoring won’t copy and paste existing code, they will rewrite it from scratch and frequently change it. Some of it is a required because systems are intertwined.
If the application is small enough you can get away with just trying the results to build confidence. However for any non-trivial application combinatorics explosion means that this stops working very quickly. This has been understood for decades.
> My point was that "didn't look at the code, looked at the app and gave feature requests and tried the results" is a way of building applications that essentially anyone who doesn't code has been doing all along.
That is not building applications. That is delegating building applications to a trusted team of humans. If that trusted team of humans are essentially junior programmers with no agency or ability to learn and improve, your app will fail.
Imagine being a product manager with a team of 100 off shore junior developers that rotate out every day. You could 100% build something useful with that setup. But anything more complex than a todo app would eventually collapse under the chaos.
They are able to extrapolate behaviour from single examples, because the implementor has common human sense, and produces an algorithm, that is continuous, and doesn't exhibit completely different behaviour for slight input changes. That is precisely the problem here.
> . I’m AI-pilled enough to daily-drive comma.ai, and I still want deterministic verification gates around generated code. My girlfriend prefers when I let it drive because it’s smoother and less erratic than I am, which is a useful reminder that “probabilistic system” and “operationally better result” can coexist.
When did the girlfriend enter the discussion? Did I miss something?
comma.ai is an autonomous driving add-on (fully open source software and hardware IIRC) for cars that don't have (full) autonomous driving but do have hardware like radar cruise control, etc. that can let software control the accelerator, brakes, steering, etc. so that the add-on can take control of the car.
The OP brings up testimony of someone other than himself who prefers when software drives their car rather than him.
It's not uncommon to have a regression test for compilers that are written in their own language (e.g. some C compilers): compile each new version with itself, then use that to compile itself again, then use the result on unit tests or whatever, which should yield the same results as before.
The point being that determinism of a particular form is expected and required in the instances where they do that.
(I'm not arguing for or against that, I'm simply saying I've seen it in real life projects over the years.)
If the output has problems, do you usually rerun the compilation with the same input (that you control)? I don't usually.
What is included in the 'verify' step? Does it involve changing the generated code? If not, how do you ensure things like code quality, architectural constraints, efficiency and consistency? It's difficult, if not (economically) impossible, to write tests for these things. What if the LLM does not follow the guidelines outlined in your prompt? This is still happening. If this is not included, I would call it 'brute forcing'. How much do you pay for tokens?
I thought to myself that I do this pretty frequently, but then I realized only if I'm going from make -j8 to make -j1. I guess parallelism does throw some indeterminancy into this
If parallelism adds indeterminacy, then you have a bug (probably in working out the dependency graph.) Not an unusual one - lots of open source in the 1990s had warnings about not building above -j1 because multi-core systems weren't that common and people weren't actually trying it themselves...
Whenever I traced them, those bugs were always in the logic of the makefile rather than in the compiler. A target in fact depends on another target (generally from much earlier in the file) but the makefile doesn't specify that.
> This comes up now as “is vibecoding sane if LLMs are nondeterministic?” Again: do you want the CS answer, or the engineering answer?
Determinism would help you. With a bit of engineering, you could make LLMs deterministic: basically, fix the random seed for the PRNG and make sure none of the other sources of entropy mentioned earlier in the article contribute.
But that barely impact any of the issues people bring up with LLMs.
And determinism isn’t particularly helpful with compilers. We expect adherence to some sort of spec. A compiler that emits radically different code depending on how much whitespace you put between tokens could still be completely deterministic, but it’s not the kind of tool we want to be using.
Determinism is a red herring. What matters is how rigorous the relationship is between the input and the output. Compilers can be used in automated pipelines because that relationship is rigorous.
And the reason that relationship can be regiorous is because compilers by definition translate one formal language to another. You can’t have a compiler that translates English to machine code in a rigorous, repeatable manner because English is ambiguous.
The problem you pointed out is real, but determinism in compilers is still useful!
Suppose you had one of those widely unstable compilers: concretely if you change formatting slightly, you get a totally different binary. It still does the same thing as per the language spec, but it goes about it in a completely different way.
This weak determinism is still useful, because you can still get reproducible builds. Eg volunteers can still audit eg debian binary packages by just re-running the compiler with the exact same input to check that the output matches. So they can verify that no supply chain attack has fiddled with the binaries: at least the binaries belong to the sources the are claimed to.
If you've got a fixed GPU that doesn't degrade at all during the process, I think? If you switch GPUs (even another one of the same model) or run it long enough the feed-forward of rounding will produce different results, right?
The rounding itself is, but the operations leading to what gets rounded are not associative and the scheduling of the warps/wavefronts isn't guaranteed.
Even the specific example in the article, the non-determinism was treated as a bug and was fixed (since by 2004 that was definitely a regression - we put a lot of work in, in the mid to late 1990s, to get bit level reproducibility - and even before that, those little details like timestamps were still deterministic variations, we had binary diff tools that could filter them out.)
Compilers aren't deterministic in small ways, timestamps, encoding paths into debug information, etc. These are trivial, annoyances to reproducible build people and little else.
You cannot take these trivial reproducibility issues and extrapolate out to "determinism doesn't matter therefore LLMs are fine". You cannot throw a ball in the air, determine it is trivial to launch an object a few feet, and thus conclude a trip the moon is similarly easy.
The magnitude matters, not merely the category. Handwaving magnitude is a massive red flag a speaker has no idea what they're talking about.
And that result of that magnitude is the paradigm of operation is just completely different. Good programmers create inputs, check outputs, and build up a mental model of the system. When the input -> output is not well defined you can't use those same skills.
GCC and LLVM consider it a bug if the compiler is non-deterministic. If re-running the compiler generates different output because of things like address differences for example then it's something that needs to be fixed. So yes they are deterministic.
I feel like this is kind of missing the point of the argument around this. People love to say "Well you don't check your compiler output do you?" (never mind that some of us actually do for various reasons). When's the last time a compiler introduced a bug into your code? When's the last time an LLM introduced a bug into your code? There you go.
Excuse me. I curse at compiler writer's on a regular goddamn basis. There are these things called "optimizations" that I assume they spend a bunch of time hmmming and hawwwing over, but will regularly take a shit on how the structure of how the resulting assembly comes out. It is downright infuriating.
Maybe you don't build or tinker with things enough to have warranted making a dartboard out of the gcc contributor graph, but damnit, some of us do. That compiler is not magic, continually floats around, and when you're just trying to get something from the stage of "doesn't exist at all" to "exists", does absolutely throw curve balls your way. I start wit -O0 -g and then crank up the optimization level once everything works. Otherwise come debug time, shit's missing, stuff happens at weird times, etc. If you don't treat the compiler as spooky, you haven't paid enough attention to it.
While I agree undefined behavior is annoying and a huge UX issue, end of the day the bug is in your code not in the compiler.
Also having an -O0 debug build is standard practice.
My point isn't that compilers are super easy to use and never frustrating, my point is that the notion that LLMs "compile" english to code is a bad analogy. Compilation is a translation from one formal representation to another. LLMs are an interpretation of informal language into a formal language. They just are not at all the same thing.
Unrelated to your actual point, with which I agree with, the problem here is not just UB. It's that timing behaviour is not part of the language specification, so compilers don't necessarily uphold it, same for other machine specific semantics, like reachability of symbols or access patterns to random addresses.
If we’re talking about “can we ignore the code the way we mostly ignore assembly and treat prompts as the new high level language”, determinism isn’t the hard problem.
The real issue is prompt instability (chaos). A one word change to a prompt/spec will produce a drastically different program. Until that is solved there’s no world where we just check in the prompt and almost no one ever has to worry about the code.
If you think that’s bad you should see how non-deterministic the alternative is (human programmers). Thankfully LLMs can iterate on the code they write, anyone who is using them to generate the same code from scratch each time a change is made needs some extra education. They are not code generators, they are junior programmers.
Yes, lots of supervision. So humans are still useful in the loop. LLMS are definitely not algorithmic transformers (ie compilers). Well, they are more like “coders” when the term meant the secretaries who translated the mathematician and scientists (the “programmers”) notation into machine code that could run on computer, by hand.
The analogy I’ve found most helpful is that it’s like having access to a team of 100 offshore junior developers that rotate out completely every few hours.
There are certainly useful things you can do with that.
But give that team to a PM who can’t read the output and anything beyond a todo app would collapse under the chaos.
A good LLM is a junior developer who is somehow really proficient at doing unreliable research. They won’t say no, I don’t know how, like lots of junior developers, but maybe sometimes they should? They also follow instructions much better than junior developers does, and they don’t mind being micro managed.
I think most people perceive program semantic not only to be defined for the whole program as a whole, but also towards individual subexpressions. If the "LLM compiler" changes the output completely when a single word is added, then it is not deterministic for the subexpressions, that don't contain that addition.
Granted that is not the rigorous definition of determinism. Also some existing compilers are non-deterministic with such a definition, namely when "undefined behaviour" would be triggered. That's precisely the problem with UB.
A related property is whether particular kinds of changes to the inputs have proportionally sized changes to the output. Adding a print statement shouldn't change the behavior of the function it's in (sans I/O), for example. Using calling the same function from two different callsites shouldn't change the behavior either. A new compiler version shouldn't change the observable behavior. Etc.
I think this is the more important property and I'm not sure if it has a well-known name. The article obliquely calls it reliability, but regardless it's the key difference from LLMs. Compilers mostly achieve it, ignoring an endless list of exceptions you learn with experience.
LLMs usually don't, even with 0 temperature and floating point determinism.
cyanydeez | 21 hours ago
measurablefunc | 21 hours ago
markatto | 21 hours ago
measurablefunc | 21 hours ago
adonovan | 21 hours ago
[OP] fragmede | 21 hours ago
I wrote "We have not remotely solved the halting problem in the formal sense", which does not read like a claim that LLMs have solved the halting problem to me, but I'm open to rewording it. How would you put it?
I added in a bit about compiler contract, wdyt?
measurablefunc | 21 hours ago
Findecanor | 20 hours ago
Attempting to get consistent results from floating-point code is another rabbit hole. GCC and clang have various flags for "fast math" which can enable different optimisations that reduce precision.
Before SSE, fp on x86 was done by the "x87" FPU which always had 80-bit precision, even if the type in the source code was 32 or 64 bits — and it used to be accepted to sometimes get more precision than asked for. Java got its "strictfp" mode mainly because of x87.
measurablefunc | 19 hours ago
¹https://cppreference.com/w/cpp/language/multithread.html
Findecanor | 15 hours ago
nu11ptr | 21 hours ago
To me that implies the input isn't deterministic, not the compiler itself
saghm | 21 hours ago
You might argue that this is redefining the question in a way that changes the answer, but I'd argue that's also an academic objection; pragmatically, the important thing isn't the exact language but the intent behind the question, and for an engineer being asked this question, it's a lot more likely that the person asking has context for asking that cares about more than just the literal phrasing of "are compilers deterministic?"
jacquesm | 20 hours ago
And that's just one really low hanging fruit type of example, there are many more for instance selecting a different optimization path when memory pressure is high and so on.
roenxi | 19 hours ago
If we're not going to assume the input state is known then we definitely can't say what the intent behind the question is - for many engineering applications the compiler is deterministic. Debian has the whole reproducible builds thing going which has been a triumph of pragmatic engineering on a remarkable scale. And suggests that, pragmatically, compilers may be deterministic.
DemocracyFTW2 | 10 hours ago
a-dub | 20 hours ago
or the system upon which the compiler is built (as well as the compiler itself) has made some practical trade offs.
the source file contents are usually deterministic. the order in which they're read and combined and build-time metadata injections often are not (and can be quite difficult to make so).
overgard | 18 hours ago
Either way it's a nitpick though, a compiler hypothetically can be deterministic, an LLM just isn't? I don't think that's even a criticism of LLMs, it's just that comparing the output of a compiler to the output of an LLM is a bad analogy.
overgard | 18 hours ago
IanCal | 21 hours ago
system2 | 21 hours ago
acedTrex | 21 hours ago
But lets all hope these are not vital systems we end up depending on.
[OP] fragmede | 19 hours ago
overgard | 19 hours ago
sarchertech | 19 hours ago
Sure everything you have unit tests for might stay the same, but unless your unit tests are testing all observable behavior (and if they are they’ll be 100x longer than the code) users will notice incredibly confusing differences in every build.
IanCal | 9 hours ago
My point was that "didn't look at the code, looked at the app and gave feature requests and tried the results" is a way of building applications that essentially anyone who doesn't code has been doing all along.
sarchertech | 9 hours ago
I have actually encountered multiple people proposing that. People in the camp of LLMs are the new high level compiler and code is assembly language.
That’s what prompted this entire are compilers deterministic article.
But ignoring whether it’s regenerated from scratch, when you ask an agent to add minor functionality, it will regularly change existing unrelated functionality.
Some of that is accidental, and could theoretically be solved. For example agents when refactoring won’t copy and paste existing code, they will rewrite it from scratch and frequently change it. Some of it is a required because systems are intertwined.
If the application is small enough you can get away with just trying the results to build confidence. However for any non-trivial application combinatorics explosion means that this stops working very quickly. This has been understood for decades.
> My point was that "didn't look at the code, looked at the app and gave feature requests and tried the results" is a way of building applications that essentially anyone who doesn't code has been doing all along.
That is not building applications. That is delegating building applications to a trusted team of humans. If that trusted team of humans are essentially junior programmers with no agency or ability to learn and improve, your app will fail.
Imagine being a product manager with a team of 100 off shore junior developers that rotate out every day. You could 100% build something useful with that setup. But anything more complex than a todo app would eventually collapse under the chaos.
1718627440 | 7 hours ago
Smalltalker-80 | 21 hours ago
drivingmenuts | 21 hours ago
When did the girlfriend enter the discussion? Did I miss something?
sprinkly-dust | 18 hours ago
The OP brings up testimony of someone other than himself who prefers when software drives their car rather than him.
DougMerritt | 21 hours ago
The point being that determinism of a particular form is expected and required in the instances where they do that.
(I'm not arguing for or against that, I'm simply saying I've seen it in real life projects over the years.)
einrealist | 20 hours ago
What is included in the 'verify' step? Does it involve changing the generated code? If not, how do you ensure things like code quality, architectural constraints, efficiency and consistency? It's difficult, if not (economically) impossible, to write tests for these things. What if the LLM does not follow the guidelines outlined in your prompt? This is still happening. If this is not included, I would call it 'brute forcing'. How much do you pay for tokens?
bandrami | 20 hours ago
[OP] fragmede | 19 hours ago
eichin | 19 hours ago
bandrami | 19 hours ago
eru | 20 hours ago
Determinism would help you. With a bit of engineering, you could make LLMs deterministic: basically, fix the random seed for the PRNG and make sure none of the other sources of entropy mentioned earlier in the article contribute.
But that barely impact any of the issues people bring up with LLMs.
wat10000 | 20 hours ago
Determinism is a red herring. What matters is how rigorous the relationship is between the input and the output. Compilers can be used in automated pipelines because that relationship is rigorous.
sarchertech | 19 hours ago
eru | 17 hours ago
Suppose you had one of those widely unstable compilers: concretely if you change formatting slightly, you get a totally different binary. It still does the same thing as per the language spec, but it goes about it in a completely different way.
This weak determinism is still useful, because you can still get reproducible builds. Eg volunteers can still audit eg debian binary packages by just re-running the compiler with the exact same input to check that the output matches. So they can verify that no supply chain attack has fiddled with the binaries: at least the binaries belong to the sources the are claimed to.
wat10000 | 8 hours ago
eru | 7 hours ago
No clue whether that's a good argument.
bandrami | 20 hours ago
eru | 17 hours ago
bandrami | 15 hours ago
pizlonator | 20 hours ago
Lots of engineering effort goes into making this be true.
TFA argues that you can't control the inputs perfectly, and so the behavior may differ if you fail to control the inputs. Yeah sure.
But the answer to the clickbaity question in the title is simply "Yes".
eichin | 19 hours ago
nickelpro | 20 hours ago
Compilers aren't deterministic in small ways, timestamps, encoding paths into debug information, etc. These are trivial, annoyances to reproducible build people and little else.
You cannot take these trivial reproducibility issues and extrapolate out to "determinism doesn't matter therefore LLMs are fine". You cannot throw a ball in the air, determine it is trivial to launch an object a few feet, and thus conclude a trip the moon is similarly easy.
The magnitude matters, not merely the category. Handwaving magnitude is a massive red flag a speaker has no idea what they're talking about.
groundzeros2015 | 20 hours ago
bandrami | 20 hours ago
pertymcpert | 19 hours ago
overgard | 19 hours ago
gaigalas | 19 hours ago
A compiler, making my job harder by being unpredictable? All the time.
So did other programmers, users with creative input, random parallel processes running at the same time.
LLMs are actually kind of tame in comparison.
salawat | 19 hours ago
Maybe you don't build or tinker with things enough to have warranted making a dartboard out of the gcc contributor graph, but damnit, some of us do. That compiler is not magic, continually floats around, and when you're just trying to get something from the stage of "doesn't exist at all" to "exists", does absolutely throw curve balls your way. I start wit -O0 -g and then crank up the optimization level once everything works. Otherwise come debug time, shit's missing, stuff happens at weird times, etc. If you don't treat the compiler as spooky, you haven't paid enough attention to it.
overgard | 18 hours ago
Also having an -O0 debug build is standard practice.
My point isn't that compilers are super easy to use and never frustrating, my point is that the notion that LLMs "compile" english to code is a bad analogy. Compilation is a translation from one formal representation to another. LLMs are an interpretation of informal language into a formal language. They just are not at all the same thing.
1718627440 | 7 hours ago
sarchertech | 19 hours ago
The real issue is prompt instability (chaos). A one word change to a prompt/spec will produce a drastically different program. Until that is solved there’s no world where we just check in the prompt and almost no one ever has to worry about the code.
gurjeet | 19 hours ago
seanmcdirmid | 19 hours ago
sarchertech | 9 hours ago
You’re agreeing with my point though which is that LLMs aren’t higher level compilers. LLMs aren’t abstraction, they are delegation.
seanmcdirmid | 9 hours ago
sarchertech | 8 hours ago
There are certainly useful things you can do with that.
But give that team to a PM who can’t read the output and anything beyond a todo app would collapse under the chaos.
seanmcdirmid | 7 hours ago
virgil_disgr4ce | 19 hours ago
So in other words, determinism (or lack thereof) is the hard problem!
ddtaylor | 19 hours ago
ChadNauseam | 19 hours ago
1718627440 | 7 hours ago
Granted that is not the rigorous definition of determinism. Also some existing compilers are non-deterministic with such a definition, namely when "undefined behaviour" would be triggered. That's precisely the problem with UB.
AlotOfReading | 19 hours ago
I think this is the more important property and I'm not sure if it has a well-known name. The article obliquely calls it reliability, but regardless it's the key difference from LLMs. Compilers mostly achieve it, ignoring an endless list of exceptions you learn with experience.
LLMs usually don't, even with 0 temperature and floating point determinism.
1718627440 | 7 hours ago