Debanjan Saha is CEO of DataRobot and a visionary technologist with leadership experience at top tech companies such as Google, AWS and IBM.
When using generative AI (GenAI) for marketing, advertising or entertainment, it might be acceptable to have the occasional response that is professionally written but factually inaccurate.
For the large majority of GenAI use cases, however, the stakes are higher. This lack of confidence in GenAI outputs is holding leaders back from using it in high-stakes external interactions such as healthcare and finance.
“Hallucinations” are just one of many challenges preventing teams from implementing GenAI: If you need to spot-check and research replies to ensure accuracy, you might just as well have done the job yourself.
While unexpected outputs can range from merely annoying and counterproductive to potentially dangerous, AI hallucinations—believe it or not—might be useful in revealing how modernized and enterprise-grade your AI processes, checkpoints and management might be.
The Root Causes Of LLM Hallucinations
First, let’s take a closer look at the root causes of LLM hallucinations.
1. Training Data Limitations: LLMs are trained on vast datasets consisting of text from the internet, books and more. These datasets might contain inaccuracies, biases or outdated information. The LLM can learn and replicate these flaws in its outputs.
2. Interpretation And Inference Errors: LLMs generate responses based on patterns and associations in the data they have been trained on. They can misinterpret a query or make incorrect inferences, leading to responses that are factually incorrect or nonsensical.
3. Lack Of World Knowledge: While LLMs can simulate understanding through the vast amount of information they’re trained on, they don’t possess true understanding or awareness. This can result in errors when the model attempts to generate information about new events, complex concepts or specific expertise areas outside its training data.
4. Overgeneralization: LLMs might overgeneralize from the training data, leading to responses that seem plausible but are incorrect or based on factually inaccurate training material.
5. Context Limitations: LLMs can struggle with maintaining and understanding context over longer conversations or texts. This might lead to responses that are inconsistent or not fully aligned with the initial query or the ongoing conversation.
6. Model Complexity And Opacity: The internal workings of LLMs are complex and not fully understood, even by their creators. This complexity can lead to unexpected behaviors, including hallucinations that are difficult to predict or explain.
Turning The Problem Of Hallucinations Into An Opportunity
AI hallucinations highlight gaps in AI’s build, governance and operation processes. CIOs and AI leaders need to examine each of these three critical areas with an eye toward reliability, stability and intervention to ensure that the outputs align with expected results.
To do so, AI leaders must approach hallucinations as an integral part of the AI development lifecycle. AI hallucinations let CIOs and AI leaders know where they need to invest to create state-of-the-art processes that are built to handle GenAI.
AI leaders, therefore, require real-time monitoring, logging and observability of GenAI outputs to detect anomalies. They also must create feedback loops for users and obtain expert reports on inaccuracies as well as hypergranular lineage of the prompt and the generated response to see where they need to augment the LLMs’ understanding of a topic.
As with all new tech, the focus and excitement is on building new GenAI applications. Still, the real value of GenAI will only be captured once CIOs and AI leaders can feel confident in the outputs.
This confidence requires that AI leaders focus not just on building, but also on lifecycle management, maintenance, oversight, governance and security to facilitate the early identification of potential issues. More importantly, leaders must ensure the continuous refinement of GenAI models through iteration and intervention.
That said, hallucinations may be much more than simple errors that need to be fixed. They may offer alternate approaches to problem-solving and creativity.
Synthetic Creativity
Perhaps the most intriguing aspect of AI hallucination is its potential to enhance and even provide a proxy for human creativity. Sophisticated hallucinations often involve GenAI providing unexpected combinations of ideas or reconfiguring patterns in ways not explicitly present in its training data.
This type of “mistake” is similar to the underpinnings of human creativity.
”Human” creativity involves activating diverse and often distant brain networks to recombine large amounts of information in novel ways.
AI, particularly in its use of neural networks, mimics this process by drawing on vast datasets to produce new patterns or ideas not present in its training data. The resulting AI hallucinations are like the brain’s creative leaps in connecting disparate ideas. These leaps hint at early-stage creativity within the GenAI, offering exploration, learning and problem-solving reminiscent of human imagination.
Embracing The Hallucination
First things first: CIOs and AI leaders need to have confidence in GenAI before they begin to use it to create solutions to complex problems.
It will take time and experience to recognize when GenAI “creativity” should be accepted, encouraged or reined in. Users will also have to be very clear on whether their GenAI is demonstrating “synthetic creativity” rather than offering factual outputs. CIOs and AI leaders will need to partner with users to ensure that GenAI isn’t offering creativity when trusted outputs are what’s called for.
To achieve the balance between creativity and confidence, CIOs and AI leaders must ensure that building, governance and operation are completely seamless and unified in the AI lifecycle. Fractured infrastructure will only leave you with fractured visibility and a lack of confidence in how your AI initiatives are performing. Investing in streamlining your AI lifecycle is the first crucial step that allows organizations to use GenAI with confidence in higher-stakes interactions.
Hallucinations hint at a huge and exciting opportunity for GenAI to provide “synthetic creativity” to solve problems in ways that are different, new and innovative. To do this, AI leaders need to embrace the current challenges around confidence and use errors to understand what areas need to be improved.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
