Decisions built on unjustified beliefs
Pritchard is not saying the machine is often wrong. He says something worse: that even when it is right, what you take away does not amount to knowledge, because there is nothing to hold on to in order to know why it was right. And we are making decisions on that...
The study concludes that generative AI does not meet the requirements of epistemic reliability, so using it as a source of information does not lead to knowledge. This implies that decisions based on its output may rest on unjustified beliefs.
In recent years, dependence on generative AI tools has soared, moving from experimental uses to everyday decisions in companies, media and governments. The speed with which these systems are being built into critical processes creates constant pressure to treat them as reliable sources of information, even though the academic evidence is still scarce.
The study published in *Synthese* took the epistemology of trust as its starting point, identifying the requirements any source must meet to be considered epistemically reliable. The authors reviewed the literature on trust in automated systems and applied a framework of safety, verifiability and accountability criteria to a representative sample of generative model outputs, evaluating whether those answers could stand as justified bases for knowledge.
The results showed that generative AI systematically fails the safety condition: its answers often mix verifiable facts with plausible inventions, without offering traceability or internal correction mechanisms. This mix of truth and fiction makes the system an unsafe source, which keeps the trust placed in it from leading to legitimate knowledge and produces beliefs that lack epistemic justification.
Those who lead organizations need to rethink how they integrate generative AI into their workflows. Instead of accepting its outputs as definitive data, they need to put in place layers of human validation, content audits and verification protocols before any important decision is based on them. Only then will the risk of building strategies on unreliable information be mitigated.
Conclusions
- Generative AI lacks the truth-guaranteeing mechanisms needed to be considered a reliable source.
- Its tendency to generate plausible but false content makes it an unsafe source of information.
- Relying on it to obtain knowledge does not justify true beliefs, only arbitrary opinions.
- Organizations must therefore complement its use with human verification or verified sources.