Hallucination

What is AI Hallucination?

AI hallucination occurs when a Generative AI Model generates false, misleading, or illogical information and presents it as fact. This issue arises because LLMs are trained on vast amounts of text data and use statistical patterns to predict the next word in a sequence, rather than understanding the underlying reality of the content they generate.

Why?

Large language models hallucinate—producing confident but false outputs—because their training and evaluation processes reward guessing over admitting uncertainty.

Why language models hallucinateOpenAI

Some Real Life Cases:

Case 1: Lawyer Citing Fake Cases Generated by ChatGPT

In 2023, New York lawyer Steven Schwartz utilized ChatGPT to draft a legal brief for a personal injury case. The brief included several fictitious court cases fabricated by the AI, which opposing counsel could not verify. When challenged, Schwartz and his colleague, Peter LoDuca, doubled down on their assertions until the court ordered them to provide the cases. Upon failing to do so, they were fined $5,000. Furthermore, the court mandated that any future filings involving generative AI content must explicitly disclose such use to ensure accuracy checks.

Case 2: AI-Generated Misinformation

AI-generated false information can spread rapidly, leading to public misunderstanding. An example of this occurred with Google’s Bard chatbot, which incorrectly claimed that the James Webb Space Telescope had taken the first image of an exoplanet. This misinformation quickly disseminated after its release, demonstrating the potential for AI to propagate erroneous news​.

https://www.theverge.com/2023/2/8/23590864/google-ai-chatbot-bard-mistake-error-exoplanet-demo

Mitigation Techniques

• Use Better Large Language Models: Whenever possible, utilize more powerful large models, as they tend to produce fewer hallucinations.

• Use AI Search Engines: Employ applications like Perplexity, which are optimized for search tasks using large models. These applications provide answers based on internet data content. (*Although they can still have hallucinations, the chances are significantly lower.)

• Human-in-the-Loop: Have humans meticulously verify AI-generated indexes, cases, and other content.

• Refine Your Prompts: Use more detailed and sophisticated prompts and prompt structures to obtain more accurate results.

• Use Advanced Reasoning LLM: Choose models specifically trained or fine-tuned for logical reasoning and step-by-step problem-solving tasks, such as GPT-4 with chain-of-thought (CoT) prompting or Claude 3, which systematically reduce hallucinations through explicit reasoning processes.

Here is a page on the OpenAI Cookbook that provides guidelines and examples for developing guardrails to prevent hallucinations in AI models.

Developing Hallucination Guardrails | OpenAI Cookbookcookbook.openai.com

In summary, always meticulously verify any content generated by large models before using it.