The inevitability of hallucinations

[jasciu]

I am not sure of this has already been discussed elsewhere, but I will bring it up here. I have read the article on the formal inevitability of hallucinations. Go here https://arxiv.org/abs/2401.11817

1. The paper formalizes the problem of hallucination in large language models (LLMs) and shows that it is mathematically impossible to completely eliminate hallucination in LLMs.
2. The authors define a formal world where hallucination is defined as inconsistencies between the LLM's outputs and a computable ground truth function. They then use results from learning theory to prove that LLMs cannot learn all computable functions, and will therefore inevitably hallucinate.
3. Since the formal world is simpler than the real world, the authors conclude that hallucination is also inevitable for real-world LLMs.
4. For real-world LLMs constrained by time complexity, the paper describes the types of tasks that are especially prone to hallucination.
5. The paper also discusses the potential mechanisms and efficacy of existing techniques to mitigate hallucination, and the implications for the safe deployment of LLMs.

In the context of the paper, the "computable ground truth function" would represent the definitive, provably correct information about some domain or task.

For example, if the task was to provide factual information about historical events, the computable ground truth function would be a mathematically representable function that returns the true, verifiable facts about those historical events for any given input.

So it's not just a series of known facts, but rather a fully computable mathematical function that can precisely map inputs to their corresponding ground truth outputs. The key idea is that this function represents the absolute, provable "truth" against which an LLM's outputs can be compared to assess hallucination.

The paper is essentially arguing that since LLMs cannot perfectly learn or emulate all possible computable ground truth functions, they will inevitably produce outputs that diverge from the true, provable information - i.e. they will hallucinate to some degree.

Just as Gödel revealed limitations in the power of formal logic and computation, this paper is suggesting there are inherent limitations in the ability of LLMs to fully capture and reproduce all computable functions and ground truths. The core issue seems to be the same - the inability of formal/computational systems to fully encapsulate mathematical reality.

But having multiple LLMs check and validate each other's outputs, is something that could potentially help address the hallucination issue raised in the paper.

The key insight is that rather than relying on a single, potentially flawed LLM to determine the "ground truth", you could leverage the collective knowledge and outputs of a diverse ensemble of LLMs. This could help compensate for the inherent limitations of any individual LLM.

Some potential benefits of this approach:

1. Cross-validation - If multiple LLMs independently arrive at the same result, that increases the confidence that the output is accurate and not a hallucination.
2. Complementary knowledge - Different LLMs may have been trained on slightly different corpora, giving them complementary knowledge bases. Aggregating their outputs could provide a more comprehensive and robust "ground truth."
3. Error detection - Discrepancies between LLM outputs could highlight areas where hallucination is occurring, allowing for further investigation and mitigation.
4. Iterative refinement - By comparing outputs and identifying inconsistencies, the LLMs could potentially refine and improve each other over time, gradually converging towards more accurate and reliable results.

The devil is in the engineering though. Is there any collaboration going on between the efforts at LLM's?