Industry insights15 min read

Deterministic AI and AI Risk: Why Process Industries Cannot Run on "Probably"

Ask a general-purpose AI model the same question about the same pump data twice, and you might get two different answers. In an office, that is an annoyance. In a refinery, a mill, or a concentrator, it is a risk someone has to sign for.

Abstract industrial graphic: a soft probabilistic wave of alternate sensor paths on the left resolves into one sharp, repeatable time-series path on the right against a faint plant silhouette.

Key takeaways

  • Generative AI predicts likely next words. That makes it fluent, and it makes it probabilistic.
  • AI hallucination is a known statistical outcome of how language models are trained and graded. It is not a rare glitch.
  • "Deterministic" settings do not make a language model repeatable. One study measured accuracy swings of up to 15% across identical runs.
  • Aviation, autonomous vehicles, and process safety arrived at the same design: AI interprets and proposes, while fixed, testable logic calculates and checks.
  • Deterministic AI applies that design to process data. The model handles the conversation, tested algorithms handle the math, and every result carries an audit trail.
In this article

That is why AI risk now shapes company policy. In McKinsey's 2025 global survey, inaccuracy was the AI risk organizations most often said they had experienced. Many engineers hear what they may not do with AI long before anyone shows them what they can.

The root issue is easy to state. Most generative AI is probabilistic. The industries that run physical processes are built on determinism: the same inputs give the same outputs, every time, with a record of how. This article explains what deterministic AI is, why AI hallucination is structural, and what air traffic control and self-driving vehicles teach about using AI where the stakes are physical.

What is deterministic AI, and why is generative AI not deterministic?

A deterministic system returns the same output for the same input, every time. A PID loop, an interlock, a spreadsheet formula, and a mass balance are all deterministic. You can test them once, document the result, and rely on it next month.

A large language model (LLM) works differently. It is trained to predict the next word across huge volumes of text, as OpenAI describes. When it answers, it assigns a probability to each possible next word fragment and picks from that distribution. The result is fluent language, produced one probable step at a time.

Three research findings matter for anyone who works with numbers.

  1. Outputs vary between identical runs. Researchers tested five LLMs on settings meant to be deterministic. Accuracy varied by up to 15% across runs, and no model was repeatable on every task. The authors concluded there is "far too much uncertainty in a realm where robust engineering is the expectation."
  2. Math is pattern matching. Apple researchers found that LLM scores on grade-school math fell when only the numbers in a question changed. Adding one irrelevant clause cut performance by up to 65%.
  3. The explanation may not match the method. Anthropic's interpretability team traced how its own model adds two numbers. Internally, it ran a rough estimate and a last-digit calculation in parallel. Asked how it got the answer, it described the textbook carry-the-one method.

That third finding should give pause to anyone who reviews calculations. If a system can describe a method it did not use, reading its explanation is not an audit.

Newer models make fewer of these errors. The mechanism underneath — picking from a probability distribution — has not changed.

Modern AI assistants also reduce the arithmetic problem by writing and running code. That moves the variability without removing it. The model still decides, on the spot, which columns to use, which rows to filter, and which method to apply. Ask again and it may decide differently.

Why do AI models hallucinate?

An AI hallucination is a confident statement that is false. The U.S. National Institute of Standards and Technology calls it confabulation and lists it among 12 risks that are unique to or made worse by generative AI. NIST traces it to the same statistical process that produces correct answers. The model generates what is likely, and likely is not the same as true.

OpenAI's 2025 research goes further. It argues that hallucinations persist because training and testing reward guessing. A model graded only on accuracy scores higher by guessing than by saying "I don't know." OpenAI also states that accuracy will never reach 100%, because some real-world questions cannot be answered from the information available.

Two details from the research carry straight over to plant data.

  • Invented reasoning. NIST warns that a model can generate logic or citations that appear to justify a wrong answer, which leads people to trust it more than they should.
  • Recognition without knowledge. Anthropic found that its model declines to answer by default. A hallucination can occur when the model recognizes a name, treats the subject as known, and suppresses that refusal.

Consider what that means for a tag name. To a language model, FIC-2301 looks like thousands of flow controllers it has read about. It knows nothing about yours unless the data are in front of it — and even then, it may fill gaps with what is typical.

When a general-purpose model analyzes process data, an error can enter at two points: when it writes the analysis code, and when it explains the result.

OpenAI's proposed fix is to reward models for admitting uncertainty. That is progress, but an abstention is not an analysis. An engineer who needs the 95th percentile of discharge pressure needs the number, calculated the same way each time.

Accountability does not transfer to the model either. In 2024, a British Columbia tribunal held Air Canada liable after its website chatbot gave a customer wrong fare information. The sum was small. The principle was not: an organization answers for what its AI says.

Why determinism matters in process industries

Process plants are deterministic by design. Control loops, interlocks, and safety instrumented systems respond the same way every time the same conditions occur. That predictability is what lets engineers test them, document them, and trust them at 3 a.m.

Engineers are not afraid of probability. Reliability engineering and process safety are built on it. What they require is that a probability be quantified and bounded: a failure rate, a confidence interval, a verified safety integrity level. The chance that a language model is wrong about your specific question is neither.

A wrong number is worse than no number

On March 23, 2005, an explosion at BP's Texas City refinery killed 15 workers and injured 180. A distillation tower was overfilled during startup. The U.S. Chemical Safety Board found that the tower's level indicator was malfunctioning and misled the operators. A separate high-level alarm failed to activate.

The causes ran far deeper than one instrument. The CSB's final report names safety culture, regulatory oversight, process safety metrics, and human factors as the key issues.

Texas City had nothing to do with AI. It shows what happens when skilled people act on a plausible reading that is wrong. Generative AI is a new way to produce plausible readings.

Where repeatability is not optional

  • Incident investigation and root cause analysis. A second engineer must be able to rerun the analysis and get the same result.
  • Anomaly detection. If the same data raises a flag on Monday and not on Tuesday, operators stop trusting the tool — the same way they learn to ignore nuisance alarms.
  • Management of change. A new setpoint or operating window needs a documented, traceable basis.
  • Environmental and production reporting. Numbers sent to a regulator or a joint-venture partner must be reproducible on audit.
  • Regulated manufacturing. Pharmaceutical and food producers must show how every reported value was derived.

What regulators now say about AI in operations

In December 2025, nine national cyber agencies, including CISA, the NSA, and the Canadian Centre for Cyber Security, published joint principles for AI in operational technology. The guidance is direct.

  • LLMs are usually built with randomness, so the same input may not give the same response.
  • AI can hallucinate, and LLMs "almost certainly should not be used to make safety decisions" in OT environments.
  • Humans remain responsible for functional safety.
  • Failsafes should revert to traditional automation or manual control.
  • Operators should first ask whether AI is the right tool, or whether an established capability already does the job.
  • Data should be pushed out of the OT network to the AI system, with no persistent access back in.

That last point describes the safe pattern for analytics. Data leaves the control environment, analysis happens elsewhere, and a person decides what to do with the result.

What air traffic control teaches about AI

Air traffic control is deterministic where it counts. Controllers separate aircraft using published minimum distances, typically 1,000 feet vertically or three to five nautical miles laterally under radar. Standard phraseology means an instruction carries the same meaning every time. Nobody estimates whether two aircraft are probably far enough apart.

Aviation has not rejected AI. It has insisted on assurance first. The FAA's Roadmap for Artificial Intelligence Safety Assurance sets out principles that translate well to industry.

  • Avoid personification. AI is a tool, not a crew member. It is not accountable. The people who design and maintain it are.
  • Separate learned AI from learning AI. A model trained offline and then frozen can be assured at design time. One that keeps changing in operation is still a research problem.
  • Go incrementally. Start with lower-criticality uses and build experience before moving up.

The FAA also explains why existing assurance does not carry over. Conventional methods assume a designer can explain every part of the design. With a learned model, the chain of traceability from requirement to implementation breaks.

The collision-avoidance example

ACAS X, the next-generation airborne collision avoidance system developed for the FAA, shows how the two worlds fit together. According to MIT Lincoln Laboratory, its logic is built offline using probabilistic models and optimization, then stored as lookup tables. On board, the aircraft does not improvise. It consults the finished tables once per second.

The statistics are done ahead of time, where they can be inspected and tested. What runs in the air is fixed. That is the pattern worth copying: probabilistic methods to build, deterministic logic to run.

The live test: the FAA's SMART system

The newest example is still unfolding. In September 2026, the FAA began using SMART (Strategic Management of Airspace, Routes and Trajectories), an AI-supported tool that combines about 200 data streams to forecast congestion and recommend schedule and route changes. Its design follows the pattern above. SMART advises, FAA specialists review its recommendations, and controllers stay responsible for separating aircraft.

The rollout has been contested all the same. The controllers' union says it had no part in the design, testing, or implementation, and more than two dozen U.S. lawmakers asked for a pause over safety evaluation, training, and oversight. On November 13, 2026, three days after SMART expanded to all 22 en route centers, a telecommunications outage forced a nationwide ground stop. As of that evening, no official had linked the outage to SMART and the cause was still under investigation. The speculation arrived anyway.

That is the lesson for industry. An AI tool can be advisory, human-reviewed, and possibly blameless — and still lose trust overnight. If the people who depend on it were not shown how it was tested, and cannot see what it did, they will assume the worst. The evidence has to exist before the rollout.

What self-driving vehicles teach about AI

Self-driving is the largest live experiment in letting learned models act in the physical world. Two lessons stand out.

One wrong classification, carried out with confidence

On October 2, 2023, a human-driven car struck a pedestrian in San Francisco and threw her into the path of a Cruise robotaxi. The robotaxi braked but hit her. It then began a pullover manoeuvre and dragged her about 20 feet.

An independent engineering review traced the manoeuvre to one error. The vehicle classified the collision as a side impact, so it pulled over instead of making an emergency stop. U.S. regulators later fined Cruise $1.5 million because two of its crash reports left out the dragging.

The technical failure was a wrong label. The corporate failure was an incomplete record. Both are arguments for systems that log what they did and why.

The leader adds an independent check

Waymo reports more than 200 million fully autonomous miles. It says that pure end-to-end neural systems risk black-box failures, because it is hard to see how they reach a decision.

Its answer is an independent onboard validation layer. That layer checks every proposed trajectory against physics-based constraints and traffic laws, and acts as a hard backstop if a plan breaks a limit. Waymo calls the design choice non-negotiable.

Other developers disagree and are betting on end-to-end models. Still, the company with the most driverless miles chose the same shape as aviation: a generative model proposes, and a separate layer with hard limits decides what is allowed.

What does deterministic AI look like in practice?

Deterministic AI does not mean no AI, and it does not mean no statistics. It means dividing the work so the language model never does the math.

In deterministic AI, the language model never does the math. Top path: Question, LLM writes code, code runs, LLM explains — methods can differ between runs. Bottom path: Question, LLM picks a tool, tested algorithm calculates the same every run with an audit trail, then LLM explains fixed numbers.
Read the lower path left to right. The model interprets the question and chooses from a fixed set of tested tools. The tool calculates. The model then explains a result it cannot change.
Two ways to answer a data question
AspectGeneral-purpose LLMDeterministic layer
Who does the calculationGeneral-purpose LLMCode the model writes for each requestDeterministic layerA tested, versioned algorithm
Same data, same questionGeneral-purpose LLMMethod and result can differDeterministic layerSame result every run
How you check itGeneral-purpose LLMRead the generated code, if it is shownDeterministic layerAudit trail of data, method, and parameters
Where errors can enterGeneral-purpose LLMWriting the code and explaining the resultDeterministic layerTool choice and wording, both open to review
Best suited toGeneral-purpose LLMDrafting, brainstorming, summarizingDeterministic layerNumbers people will act on

Three clarifications keep the term precise.

  • Forecasts are still estimates. A deterministic forecast is repeatable, not certain. The uncertainty belongs to the process, not to the tool.
  • Machine learning can be deterministic. Many methods use random numbers internally. Fixing the random seed and versioning the model makes them repeatable.
  • The explanation is commentary. The model still writes the narrative. The numbers, charts, and audit trail come from the deterministic layer, so check those.

A checklist for evaluating AI tools for process data

Ask any vendor, or your own data team, these eight questions.

  1. Repeatability. If I ask the same question of the same data twice, do I get the same numbers?
  2. Who does the math. Is it the language model, code it writes on the spot, or a tested algorithm?
  3. Traceability. Can I see the data range, method, and parameters behind each result?
  4. Reproducibility. Can a colleague rerun the analysis next month and match it?
  5. Versioning. Are algorithms versioned and random seeds fixed, so an update cannot quietly change old results?
  6. Honest gaps. What does the tool do when data are missing, sparse, or irregular? Does it say so?
  7. Data path. Where does my data go, and does the tool need any access into the control network?
  8. Human decision. Does the tool inform a person, or does it act on its own?

A tool that answers the first four well is fit for numbers people will act on. One that cannot should stay with drafting and summarizing.

How we approach deterministic AI at Grayson

That split is how we built Grayson TimeSeries. It adds what we call a trusted layer of deterministic analytics to the AI tools engineers already use, including Microsoft Teams, ChatGPT, Claude, Gemini, and Grok.

  • The AI handles the conversation. It takes the question, supplies context, and explains the result.
  • Tested models handle the calculation. Each answer comes from a tested statistical or machine-learning model, not from code written on the spot.
  • Every result can be rerun and audited. Answers arrive with charts, context, and the audit trail behind them.

Our four analysis agents cover the questions engineers ask most often of process data.

Grayson analysis agents
AgentWhat it looks forMethods
Anomaly detectionWhat it looks forSpikes and shiftsMethodsIsolation Forest, Z-score, IQR, variational autoencoder
ForecastingWhat it looks forWhere a variable is headingMethodsProphet, simple moving average, linear trend
CorrelationWhat it looks forWhich variables move togetherMethodsPearson, Spearman, rolling correlation, cross-correlation
Data hygieneWhat it looks forMissing values, irregular sampling, outliersMethodsData quality scoring

We built Grayson as an analytics layer for engineers and operators. It informs decisions. It does not control equipment or stand in for a safety system. That is the role the joint OT guidance recommends for AI: inform people, and leave control and safety functions to deterministic systems.

It is free to start, with no credit card and setup in under two minutes. Use Grayson Free and ask your process data a question today. For sector context, see our industry pages and pricing.

Frequently asked questions

What is deterministic AI?

Deterministic AI is a system design in which a language model handles the conversation and fixed, tested algorithms handle the calculation. The same data and the same settings produce the same result every time, with a record of how it was produced.

What is the difference between deterministic and probabilistic AI?

Probabilistic AI, including large language models, builds each output by picking from a probability distribution, so results can vary between runs. Deterministic AI routes calculations through fixed algorithms, so results repeat.

Can AI hallucinations be eliminated?

Not with today's general-purpose models. OpenAI's research says accuracy will never reach 100%, and that models hallucinate less when they are rewarded for admitting uncertainty. The practical answer is to keep the language model away from the calculation.

Does setting temperature to zero make an LLM deterministic?

No. A study of five LLMs found accuracy varied by up to 15% across identical runs at settings meant to be deterministic.

Is machine learning deterministic?

It can be. A trained model with fixed parameters, a fixed random seed, and a fixed version returns the same output for the same input. The FAA draws a similar line between learned AI, which is frozen, and learning AI, which keeps changing.

Why does determinism matter more in process industries than in office work?

Because the output drives physical decisions. A wrong sentence in an email gets edited. A wrong number about a pump, a tower level, or an emissions total can cost production, fail an audit, or put people at risk.

Should AI make safety decisions in a plant?

Joint guidance from nine national cyber agencies says LLMs almost certainly should not make safety decisions in operational technology, and that humans remain responsible for functional safety.

How can engineers use AI on process data safely?

Use AI for the question and the explanation, and a deterministic layer for the calculation. Keep data flowing out of the control network, keep an audit trail, and keep a person in the decision.

Sources

  1. OpenAI, Why language models hallucinate, September 2025
  2. Atil et al., Non-Determinism of "Deterministic" LLM Settings, arXiv:2408.04667
  3. Mirzadeh et al., Apple, GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models, arXiv:2410.05229
  4. Anthropic, Tracing the thoughts of a large language model, March 2025
  5. NIST, AI 600-1: Generative Artificial Intelligence Profile, July 2024
  6. McKinsey, The state of AI in 2025, November 2025
  7. CISA, ASD's ACSC, NSA, FBI, Canadian Centre for Cyber Security and partners, Principles for the Secure Integration of Artificial Intelligence in Operational Technology, December 2025
  8. Canadian Centre for Cyber Security, announcement of the joint guidance
  9. U.S. Chemical Safety Board, BP Texas City final investigation report, March 2007
  10. U.S. Chemical Safety Board, preliminary findings, BP Texas City, October 2005
  11. FAA, Roadmap for Artificial Intelligence Safety Assurance, Version I, 2024
  12. MIT Lincoln Laboratory, Airborne Collision Avoidance System X
  13. NHTSA, Consent order with Cruise, September 2024
  14. CNBC, report on the Quinn Emanuel and Exponent findings, January 2024
  15. Waymo, 10 AI Lessons from Driving 200+ Million Fully Autonomous Miles, August 2026
  16. McCarthy Tétrault, Moffatt v. Air Canada: A Misrepresentation by an AI Chatbot, February 2024
  17. AVweb, FAA Modernization Is Getting Smarter, Faster And More Expensive, September 2026
  18. FLYING, FAA's Rollout of SMART AI System Left Out ATC Union, September 2026
  19. Men's Journal, What We Know About SMART, the FAA's New AI Air Traffic Tool, After the Nationwide Ground Stop, November 13, 2026

Next step

Try Grayson free, or keep reading

Ask your process data a question today.

Free. No credit card. Set up in under two minutes.

Use Grayson Free